How do brain foundation models perform across EEG and iEEG tasks?

A single testbed for EEG and iEEG models

Brain foundation models are pretrained encoders that learn transferable representations from electrical brain signals, then adapt to downstream tasks such as diagnosis or sleep staging. Before this paper, the authors argue, benchmarks covered mainly scalp EEG, included only a limited subset of available models, and reported little beyond downstream scores. Brain4FMs is presented as the first unified benchmark for evaluating these models jointly on EEG and iEEG, the intracranial recording used in clinically indicated patients.

The scope is concrete: 17 representative models and 21 public datasets spanning clinical diagnosis, sleep staging, communication, and affective computing. The benchmark is described as open and plug-and-play, with dataset-aware preprocessing, cross-subject evaluation, handling of heterogeneous multichannel recordings, and standardized downstream adaptation workflows. Those components matter because they determine whether two models are being compared under the same conditions.

414 model–setting pairs, plus two NeuroGPT variants

The evaluation covers 26 downstream classification settings. All 17 models are evaluated on 22 of them, while four heterogeneous multichannel iEEG settings support only 10 compatible models: BIOT, LaBraM, CBraMod, BrainBERT, SppEEGNet, BrainWave, MBrain, BFM, CodeBrain, and MVPFormer. That yields 414 model–setting pairs. For NeuroGPT, the authors additionally report an encoder-only variant and an encoder–decoder variant on the 22 applicable settings, bringing the total to 436 reported pairs.

The metric choice depends on the task. Binary tasks are reported with AUROC, accuracy, F1, and F2, which capture threshold-free discrimination and positive-class detection under imbalance. Multi-class tasks use accuracy, one-vs-rest AUROC, macro-F1, and Cohen's κ, which measure overall discrimination, class-wise performance, and agreement beyond chance. Results are means with standard deviations over n-fold cross-validation, and the best mean is marked using unrounded values. These are different measures of different things, so a model that leads on AUROC is not automatically the leader on F1 or κ.

The winner changes with the dataset

The headline finding is heterogeneity. Performance varies across tasks, signal modalities, and adaptation protocols, and no single model consistently dominates all evaluation scenarios. The per-dataset tables make this visible in specific cases. On CHBMIT, BrainWave reaches an AUROC of .90 ± .03, while MBrain, the strongest contrastive-group model there, reaches .75 ± .08 and BIOT falls to .56 ± .08. On MAYO, BrainWave again leads at .98 ± .01, with BrainBERT close at .97 ± .01 and NeuroLM at .64 ± .09.

The ordering is not stable across datasets. On ADFD, REVE leads at .84 ± .07, ahead of LaBraM at .83 ± .10 and BrainOmni at .80 ± .07, while BrainWave sits at .74 ± .05. On the iEEG settings the picture shifts again: on HUP-ECoG, BrainWave reaches .88 ± .07 and BrainBERT .81 ± .12, but LaBraM drops to .51 ± .06 with an F1 of .00 ± .00. On HUP-SEEG, BrainWave leads at .79 ± .11 and BFM reaches .61 ± .02. A reader comparing models on a single dataset would draw a different conclusion than one comparing across the full set.

Fewer models qualify for the iEEG settings

The iEEG settings are not a like-for-like extension of the EEG results. Only 10 of the 17 models are compatible with the four heterogeneous multichannel iEEG settings, so the comparison set shrinks and the rankings there cannot be read as a full 17-model contest. The reported numbers also show much weaker separation on some of these datasets: on Cogitate, LaBraM reaches .53 ± .01, CodeBrain .52 ± .01, BrainBERT .52 ± .01, and BrainWave .49 ± .01, all close to chance-level AUROC of .50.

This is where the benchmark's design choice becomes a finding in itself. The authors built dataset-aware preprocessing and heterogeneous multichannel handling precisely because montages and recording setups differ, and the iEEG tables show that transfer into these settings is uneven. A model's strong EEG score does not carry over automatically to intracranial data in this benchmark.

Adaptation protocol changes the ranking

The benchmark separates two adaptation protocols. In full fine-tuning, the pretrained model is updated along with the downstream head. In frozen fine-tuning, the pretrained encoder is fixed and only the classification head is trained. To keep the comparison fair, the authors reuse the classifier-head learning rate selected in the corresponding full fine-tuning run, and they add a head-learning-rate sensitivity analysis on representative pairs covering Bendr, EEGPT, SppEEGNet, and NeuroGPT on MDD-64, MAYO, ADHD-Adult, and ISRUC-G1.

That sensitivity check matters because a badly chosen head rate could make frozen fine-tuning look worse than it is. For most pairs, varying the head learning rate produced only minor changes in frozen performance. EEGPT on MAYO was the one clearly sensitive case: the main-experiment head rate of 5×10⁻⁴ gave a validation AUROC of 73.20, close to the best searched result of 75.13 at 10⁻³, and still far below the full fine-tuning result of 91.76. The authors conclude the shared head-rate setting does not materially disadvantage frozen fine-tuning or change its ordering relative to full fine-tuning.

Few-shot gains are real but uneven

A prototype-based transfer analysis tests whether giving a model more labeled examples improves performance. Seven models were run on six datasets under 1-, 3-, 8-, and 15-shot settings, with each box summarizing fold-level results from five repeated support-set samplings per fold. At the aggregate level, the shot–performance relationship is positive.

At the pair level it is not uniform. Among the 42 model–dataset pairs with all four shot settings available, 22 show a significant positive 1/3/8/15-shot trend after FDR correction, and for the 15-shot versus 1-shot comparison, 21 of 42 pairs show a significant improvement. Several models on SleepEDF show significant positive trends with relatively large 15-shot gains, and BrainWave improves particularly on ADHD-Adult, SZ-28, and ADFD. Other pairs show only modest or non-significant gains, including ADFD with BrainOmni, ADFD with MBrain, SZ-28 with REVE, ADHD-Adult with CBraMod, and MDD-64 with CBraMod. The aggregate trend therefore should not be read as a guarantee for any particular model–dataset combination.

Spatial, spectral, and discrete representations

Beyond downstream scores, the authors run exploratory analyses of model-specific properties in three areas: spatial, spectral, and discrete representations. The stated purpose is to understand the heterogeneous transfer behaviors rather than to rank models further. The paper also organizes the models by self-supervised pretraining paradigm, grouping them into contrastive, generative, and other categories in the results tables, and ranks models by AUROC within each group.

This grouping is a reminder that the benchmark compares models built on different pretraining objectives, not variants of one design. The tables label each model's paradigm, so a reader can see whether a strong result comes from a contrastive model such as MBrain or BIOT, a generative model such as BrainWave or LaBraM, or another paradigm. The exploratory representation analyses are presented as a way to interpret the variation, not as proof of a causal mechanism behind any single score.

Classification only, and what that excludes

The benchmark is deliberately bounded. The authors state that they focus on classification because it remains central to brain-signal analysis and is the primary downstream setting supported by existing brain foundation models. Generative tasks are not covered and are described as future work to be added as the field and available models mature. The results therefore speak to classification performance under cross-subject protocols, not to generation quality or to clinical deployment.

The practical priority that follows is specific to this evidence: when choosing among these models, match the candidate to the dataset, modality, and adaptation protocol you will actually use, and check whether your setting is one of the four iEEG settings where only 10 models are compatible. The per-dataset tables, not a single overall ranking, are the decision surface this benchmark provides.

Sources used in this answer