Can alternating multi-agent sessions outperform single-agent MLE workflows?

Longer single sessions buy less reasoning, not more

The paper starts from a measurement problem rather than a design idea. Across six models run in single-agent /goal mode for up to six hours of active runtime on 75 tasks with three repeats, the mean output tokens per recorded response fell by 29.8% to 45.0% between hours 0–2 and hours 3–5. The authors call this long-horizon deceleration: as score gains shrink, the agent spends fewer tokens reasoning about what to try next, not more.

That statistic is a proxy, and the paper treats it as one. It counts provider-reported output tokens divided by the number of responses in a trailing 30-minute window, pooling tasks and repeats within each configuration, and it excludes tool-return text, input tokens, and coordinator output. So the claim is about the agent's own response length over time, not about total compute spent or about the quality of its reasoning directly.

The practical consequence is that simply letting one agent run longer does not automatically convert extra time into extra exploration. The paper pairs the response-length decline with rapid saturation of prefix gains, which is why it looks for a fix at the workflow level rather than inside a single session.

Custom harnesses lose value as the backbone improves

The second motivating finding concerns the context-engineering layer that many MLE agents add: Markdown instructions, skills, and accumulated experience that steer the agent's next steps. The paper compares three recent harnesses (ScienceFlow, ML-Master 2.0, and MLEvolve) against the official DeepSeek Harness on a 16-task MLE-bench subset, with task-specific cold-start knowledge disabled for fairness, using DeepSeek-V4-Flash and DeepSeek-V4.1-Flash.

The pattern the authors report is harness staleness: the custom harnesses' advantage over the official harness shrinks as the backbone model improves. On the stronger DeepSeek-V4.1-Flash, the official harness reached a 77.1% medal rate on those 16 tasks, versus 68.8% for ScienceFlow, 37.5% for ML-Master 2.0, and 50.0% for MLEvolve. The authors read the official harness's strength as consistent with a possible model–harness alignment acquired during training, and they flag that as an interpretation rather than a demonstrated mechanism.

These 16-task numbers are a separate comparison from the 75-task main results and should not be mixed with them. Their role in the argument is narrower: guidance tuned to compensate for one model's weaknesses can become redundant or counterproductive for its successor, so a workflow that depends on such guidance may not transfer across model versions.

Alternation moves the fix from prompts to workflow

Humanize MLE Agents (HMA) leaves each native model–harness pair unchanged and coordinates complete agents instead. Two agents, A and B, work on one shared workspace that retains the evaluated candidates and the current best result. Each session is capped at k experiments, and at every handoff the agent's context is reset while the workspace and retained result persist. At the end of the budget, a review step selects the final candidate.

The formal framing is a context-reset semi-Markov decision process (CR-SMDP). Each agent session is modeled as an option: an initiation set, an internal policy, and a termination rule. Termination creates an option boundary where the workflow policy picks the next option; in HMA that boundary preserves the shared workspace and result but empties the context. This is what makes the approach a workflow-level intervention rather than another harness.

The theoretical claim is conditional, not universal. Under an assumption that the best result is never lost and an assumption of persistent cooperative access to high-quality states beyond what either single agent can reach, the paper proves that alternation can exceed the single-agent reference ceiling. The proof sketch tracks the probability of still being below a target quality and shows it shrinks geometrically across cycles. Readers should treat this as a sufficient-condition result: it says when alternation can help, not that it always will.

The budget bound explains why short caps can help

From the same analysis the paper derives an explicit experiment-budget bound. With the initial below-target probability set to 1, a fixed target success probability p0, and a small per-cycle success factor a, a sufficient budget scales as 2k/a times a log term in 1/(1−p0), up to lower-order corrections. The cap k enters directly, which is why session length is a design variable rather than an arbitrary setting.

A second theorem addresses what happens when the cap cuts a session short. Comparing the capped policy against natural-termination alternation, the paper shows that exceeding the single-agent ceiling is guaranteed when the natural-termination margin plus the expected net gain from interrupted options is positive. In other words, a cap is not automatically harmless; it preserves the advantage only when the value of restarting with a fresh context outweighs the value of continuing the current session.

The paper motivates short caps empirically rather than deriving an optimal k. That is a real boundary: the theory gives conditions and a budget form, but the specific cap used in experiments is a choice supported by results, not a closed-form optimum.

78.2% any-medal rate at a quarter of the budget

The headline result comes from 75 Kaggle tasks in MLE-bench, with each configuration evaluated in three independent repeats and rates computed from per-repeat medal counts before averaging. HMA reached a state-of-the-art any-medal rate of 78.2%, 8.0 points above the strongest open-source baseline, while using a quarter of that baseline's time budget.

The comparison baselines are not resource-matched, and the paper says so: ScienceFlow ran with a 24-hour budget, MLEvolve with 12 hours, and ML-Master 2.0 with 24 hours, on differing hardware allocations. That makes the time-budget advantage a statement about reported configurations, not a controlled efficiency measurement. The any-medal rate is also a coarse metric: it counts whether a task earned any medal, not which medal, so it does not by itself show that HMA produces better top-end solutions.

The paper reports additional metrics including gold percentage and mean percentile, but the any-medal rate is the figure the main claim rests on. Readers comparing HMA to a specific system should check whether that system's budget and resources match, because the 8.0-point gap is stated against the strongest open-source baseline under its own reported configuration.

Gains hold across five heterogeneous agent pairs

A single strong pairing could reflect one lucky combination of models. To test that, the paper runs six configurations spanning five heterogeneous agent pairs and compares HMA against the average of the two constituent single-agent baselines, weighted equally within each pair. Across those configurations, HMA improved the any-medal rate by 6.4 points on average over the constituent baselines.

This is the more informative generality claim, because the comparison is internal to each pair: the same two agents are measured alone and then together. The 6.4-point average is smaller than the 8.0-point gap against the external baseline, and the two numbers answer different questions. One says HMA beats the best reported open-source system; the other says alternation tends to beat the agents it is built from.

The paper also reports that HMA outperforms the stronger constituent in medal rate under equal six-hour budgets across the six configurations. That framing matters: the relevant comparison for a practitioner choosing between running one agent and alternating two is against the better of the two agents, not against their average.

One kernel case extends the claim, cautiously

Beyond MLE-bench, the paper reports a supplementary case on kernel optimization that reached 878 cycles. The authors compare this to the Paradigm community leaderboard as of October 8, 2026, where 878 cycles matches the 10th-ranked score and sits 14 cycles (1.62%) above the leading score of 864 cycles. They explicitly note that the leaderboard uses a separate nine-seed judge, so this is a comparison of reported scores rather than a verified leaderboard placement.

This case is a single run over a 48-hour horizon, and the paper labels it preliminary evidence of applicability beyond MLE-bench. It should not be read as establishing comparative gains outside MLE-bench; the authors list that as future work. The value of the case is that it shows the workflow operating over a longer horizon than the MLE-bench runs, not that it dominates on kernel optimization.

Taken together, the evidence supports a specific priority: when a single agent's response length is decaying and its harness was tuned for an older model, alternating complete agents with short capped sessions and context resets is the intervention the paper's results favor. The condition that most changes the answer is whether the cap preserves value at each handoff, which the interruption theorem makes an explicit check rather than an assumption.

Sources used in this answer