📰 Key Takeaways

IBM released LLMON (LLM Object Notation, pronounced like “lemon”), a markup language designed specifically for large language models. Alongside it, they put out two experimental preview models: granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview. Both are derived from the Granite 4.1 base models and trained to read and produce LLMON-marked text. There’s also a GitHub toolkit with example workflows and templates.

The motivation: in a long conversation, the context is a mix of many things — system prompts, past turns, instructions about retrieved documents, tool responses, output constraints, and reasoning chains. Most of that structure is implicit, or it gets stuffed into JSON, a format originally designed for parsers. That makes smaller models more error-prone, and it gives attackers room to pull off prompt injection because there’s no explicit structure to lean on.

LLMON’s approach is to mark semantically complete blocks with meaningful tags, using reserved special tokens as delimiters. It declares data structure and nesting in a way a causal language model can clearly understand. Every instruction gets a unique ID, so it can point at an instruction anywhere in the context and bind it to the data it’s supposed to handle. Conceptually, it’s like function calls and argument passing in programming languages.

The benefits listed in the original post include:

  • Indirect prompt injection hidden in user data is mostly ignored. On some benchmarks, the attack success rate drops to near zero, while ordinary instruction-tuned models get hijacked at a high rate. Overall capability is on par with Granite 4.1 Instruct.
  • Irrelevant passages, old conversation turns, and tool outputs don’t get mixed into the answer.
  • A converter can losslessly turn existing JSON into LLMON, and you can define custom tags and train LoRAs on top of it.
  • The Mellea framework supports it natively and automatically converts calls into LLMON.

The second half of the original post, which explains why not to just use JSON, was cut off in the summary. See the original link for the details.

💬 JudyAI Lab Take

IBM’s LLMON basically adds an explicit set of structural markers to an LLM’s context, and it ships with two preview models derived from Granite 4.1. It’s worth paying attention to because it goes straight at an old problem: long conversations get messy, and that mess is exactly what prompt injection exploits.

In a long conversation, the system prompt, chat history, retrieved files, tool responses, and output constraints have mostly been implicit in the text, or crammed into JSON that was built for parsers. That trips up small models and leaves gaps for attackers. LLMON’s idea is to carve out semantic blocks with meaningful tags and reserved special tokens, give each instruction a unique ID, and bind it to the data it should act on, much like function calls and arguments. The original post says indirect prompt injection hidden in user data is mostly ignored, attack success rates on some benchmarks fall to near zero, and overall capability still matches Granite 4.1 Instruct. For AI builders, this points to a design direction: safety and stability don’t have to come only from bolt-on guardrails. You can also separate “instructions” from “data” at the input format and model training level.

If you’re wiring up tools or retrieval pipelines, a good first step is to take stock of what in your context is an instruction and what is external data, and check whether your current format lets the model clearly tell the two apart.

📅 Source Info

🔗 Further Reading