DELM: Scaling Agentic Reasoning via Decentralized Shared Context

TL;DR

Decentralized Language Models (DELM) break the bottleneck of centralized multi-agent orchestration. By introducing a shared verified context and an asynchronous task queue, DELM allows any number of agents to share discoveries, avoid redundant failures, and reason over massive contexts (like entire codebases or document sets) more efficiently. The results are striking: a 50% cost reduction and a 10.5% performance jump on top-tier engineering benchmarks.

The "Central Controller" Bottleneck

Modern multi-agent systems typically work like a strict corporate hierarchy: a "Main Agent" breaks a problem into pieces, hands them to "Sub-agents," waits for them to finish, and then tries to summarize everything to decide the next move. This Scatter-Gather approach has two fatal flaws:

1. Serialization: The Main Agent becomes a communication bottleneck; as the number of agents grows, the time spent merging results exceeds the time spent solving the problem.
2. Information Dilution: When the Main Agent summarizes a sub-agent's findings, it often loses the "nuance"—a small technical detail or a specific failure reason that might be vital for the next thread.

Methodology: Coordination as State, Not Prompts

DELM’s core insight is that agents shouldn't talk to each other; they should talk to the State. This is achieved through three architectural pillars:

1. Compact, Global, Unfoldable Context

Instead of passing huge raw logs, agents write Gists—ultra-compact, 100-token summaries. If a future agent needs more detail, it can "unfold" that gist into a grounded summary, and eventually into the raw evidence. This mimics "demand paging" in operating systems.

2. Admission-Time Verification

To prevent "hallucination propagation," DELM acts as a gatekeeper. Before any update is added to the shared context, a verifier LLM checks it against the raw source or reasoning trace. If it's unsupported, it’s rejected or rewritten.

3. Asynchronous Task Queue

Agents grab tasks as they become available. If Thread A finds a "red herring" and documents it in the shared context, Thread B (which just started) will see that note immediately and avoid wasting budget on the same mistake.

Experimental Performance: Better, Faster, Cheaper

The researchers tested DELM on two "stress-test" scenarios: Software Engineering (SWE-bench Verified) and Multi-Document QA (LongBench-v2).

• Software Engineering: DELM turned parallel attempts into "shared exploration." By sharing negative results (e.g., "Changing the printer didn't fix the bug"), agents converged on the solution faster.
• Long-Context QA: By building a verified hierarchical view of the documents first, DELM achieved higher accuracy than models like GPT-5.4 and Claude Sonnet 4.6, as it was less likely to miss cross-document evidence.

Critical Insight: Why Does It Work?

The secret sauce is Failure Sharing. In most MAS, if an agent fails, that failure dies with the thread. In DELM, a failure is a "FACT" that informs everyone else. For example, in a Django-related task, one agent discovered that a specific optimization broke multi-value relations. Because this was written to the shared context, subsequent agents didn't try to "re-optimize" that section, directly leading to a successful patch.

Conclusion & Future Outlook

DELM proves that the next frontier of AI agents isn't just "smarter models," but better communication substrates. By treating the shared context as a curated, verified database rather than a messy chat history, DELM enables massive test-time scaling. This approach is particularly promising for Automated Research, where agents must cross-reference thousands of papers and experimental logs without losing track of the grounding truth.

For more details, check out the full paper at the DELM Project Website.