Can a dual-loop autonomous research agent improve EEG analysis and decoding?

An agent that diagnoses before it intervenes

Fixed EEG pipelines and reward-driven search both skip a diagnostic step: they do not explicitly work out how signal structure, representations, predictor behavior, and training dynamics each contribute to a performance or generalization gap. NeuroAct is built to make that choice of intervention explicit. It is an autonomous research agent framework spanning EEG foundation models, task-specific predictors, and combinations of models.

The framework runs two coupled loops. In the inner loop, a Coordinator splits a shared objective into parallel questions and assigns them to three roles: a Researcher that investigates and proposes, a Developer that implements and runs experiments, and a Reviewer that challenges the result. These roles draw on evidence about signal structure, representations, predictor behavior, and training dynamics, and they refine candidates using feedback on performance, stability, and cost. The outer loop then distills what happened into conditional, testable recommendations tagged with the task and model conditions under which they held, and validates or revises those recommendations on later tasks.

A 17.75-point lead on BrainBench

The first setting, EEG General Analysis, tests broad EEG knowledge and analysis across four BrainBench domains: foundations, sleep, neurocognition, and physiology. With the DeepSeek backend, NeuroAct reached a macro-average of 80.88, compared with 63.13 for BrainAgent, 61.02 for CodeAct, and 58.90 for BrainPilot. The gap to BrainAgent is 17.75 points.

The domain breakdown shows where that average comes from. NeuroAct scored 87.26 on foundations and 92.16 on physiology, while its weakest domain was neurocognition at 68.73. The authors also ran NeuroAct with two other LLM backends: Luna reached a 75.58 macro-average and Qwen 75.26, so all three backends stayed above 75. The comparison is therefore not a single lucky model pairing, though the DeepSeek configuration is the one used for the headline number.

Improving shared checkpoints across five targets

Cross-task Model Optimization asks whether the agent can revise one shared checkpoint that serves four classification tasks plus HBN age regression. The paper reports this for four encoder families. For EEGPT, NeuroAct raised mean balanced accuracy by 7.34 points and lifted HBN Pearson's r from 0.6894 to 0.7362. For REVE, HBN r rose from 0.616 to 0.719 and mean absolute error fell from 2.149 to 1.810.

The gains are uneven, and the paper's own tables show it. LaBraM's BCIC-IV-2a balanced accuracy moved only from 28.65 to 33.94, and its ADFTD result improved from 35.28 to 56.37, while EEGPT's ADFTD balanced accuracy actually slipped slightly from 46.51 to 45.98. In the framework comparison with DeepSeek and LaBraM, NeuroAct reached 65.29 balanced accuracy on Workload and 70.74 on HMC, against 50.00 and 36.34 for BrainAgent on the same tasks. The pattern is that the agent helps most where the starting checkpoint is far from adequate, not uniformly.

Open research and model combination add further gains

Single-task Open Research fixes the dataset and prediction protocol but lets the agent choose models and change implementations. On ADFTD, the selected ShallowFBCSPNet reached 71.26 balanced accuracy against 54.93 for the compared EEGNet, a 16.33-point improvement. On BCIC-IV-2a, the selected REVE reached 60.59 versus 52.60 for the compared CBraMod. On HBN regression, NeuroAct's REVE predictor reached r = 0.715, above the compared BIOT at 0.488.

Collaborative Prediction tests whether combining models beats the best single member and simple equal weighting. On Workload, NeuroAct reached 85.02 balanced accuracy, above the best single member at 79.69 and equal weights at 77.95. On HBN, its collaborative solution reached r = 0.780 with MAE 1.585, beating the best single member's r = 0.758 and MAE 1.661. The gains are smaller on tasks where the best single model was already strong: HMC moved from 73.12 to 75.14, and BCIC-IV-2a from 60.85 to 61.02.

Reused experience raises accuracy and cuts token cost

The outer loop's value is tested directly through experience reuse. When recommendations from earlier tasks were reused, BCIC-IV-2a balanced accuracy rose from 34.66% to 50.81%, while LLM token use fell by 64.02%. That pairing matters: the agent did not buy accuracy with more search, it reached a better result with less of the expensive LLM interaction.

This is the paper's clearest evidence that the outer loop carries something transferable rather than merely re-running the inner loop. The authors describe the mechanism as distilling outcomes into conditional recommendations with task and model conditions, then matching and adopting them on subsequent tasks. The token reduction is reported for this reuse study specifically, so it should be read as a property of that comparison rather than a fixed cost saving for every setting.

Benchmark gains, not clinical readiness

All four settings are evaluated on existing EEG datasets, and the study reports aggregate benchmark results without recruiting new participants. The authors state that NeuroAct is evaluated as a research framework and that clinical use would require independent validation plus assessment of demographic and recording-condition shifts. The reported balanced accuracy, Pearson r, and macro-average figures therefore describe benchmark performance under the paper's protocols, not diagnostic performance in a clinic.

Two further boundaries sit inside the results. The headline BrainBench comparison uses the DeepSeek backend, with Luna and Qwen reported separately at lower macro-averages. And the cross-task results vary by encoder family, with some tasks improving substantially and others barely moving or slipping. A reader comparing agents should match the backend, encoder, and task before treating any single number as a general capability score.

Read the per-task table, not the average

The paper's most useful signal is the split between tasks where the agent had room to improve and tasks where the baseline was already strong. Workload collaborative prediction gained 5.33 points over the best single member, ADFTD open research gained 16.33 points over the compared model, and BCIC-IV-2a collaborative prediction gained only 0.17 points. The same agent, on the same benchmark suite, produces very different magnitudes depending on the starting point.

That makes the conditional recommendations the outer loop produces the part worth watching, rather than the aggregate scores. The paper's claim is that NeuroAct accumulates validated recommendations across tasks and conducts later studies more effectively, evidenced by the BCIC-IV-2a reuse result of 50.81% balanced accuracy at 64.02% fewer tokens. Whether that transfer holds for a new task depends on whether its task and model conditions match a stored recommendation, which is exactly the condition the framework attaches to each one.

Sources used in this answer