Can physiology-guided pretraining improve intracranial EEG foundation models?

Physiology supervision replaces implicit retention with explicit targets

Most recent iEEG foundation models learn by reconstructing or predicting their own input, which leaves the retention of clinically meaningful signal properties implicit. The paper's premise is that clinicians already read iEEG through directly measurable properties — waveform shape, frequency, amplitude, morphology — and that these properties carry much of the information downstream tasks need. PHASE therefore makes them explicit learning targets rather than hoping they survive a reconstruction objective.

The targets are computed from the raw signal, not from labels. At each token position the physiological decoder predicts three groups: multi-scale wavelet power, amplitude and local-variation measures, and short-window high-frequency Fourier power. Wavelet and amplitude targets supervise both masked and fully visible positions; local Fourier targets supervise only fully visible patches, and frequency-dependent targets contribute only within the recording's effective bandwidth. Targets are standardized with fixed training statistics shared across recordings, so no per-site normalization or expert annotation enters pretraining.

Two stages separate within-channel timing from cross-channel context

PHASE-T handles a single channel. Its encoder produces states on a 32-ms grid with a feature dimension of 768, combining shifted local attention over overlapping waveform patches with global attention over a pooled coarser sequence that spans inputs of up to one minute. A refinement decoder merges upsampled coarse context with projected fine features through a skip connection. The encoder is shared across channels and uses no electrode coordinates or fixed channel identities, which matters because electrode placement and coverage differ by participant and acquisition practices differ by institution.

PHASE-ST then updates these representations across synchronized channels while keeping the temporal encoder frozen. It applies self-attention across channels independently at each time position and adds a residual update in the temporal feature space, so parameters are shared across channels and one set of parameters serves any number of channels. Its pretraining adds relation supervision on channel pairs — phase-locking value, envelope correlation, broadband correlation, and signed lag — measured from clean waveforms, which extends physiological guidance from single channels to how activity couples and propagates between them.

Pretraining used heterogeneous recordings from 222 participants at nine clinical sites.

Frozen PHASE-T leads on all five clinical tasks

On the Omni-iEEG test set, linear probes on the same frozen PHASE-T handle five clinical tasks spanning very different timescales: high-frequency oscillation events lasting tens of milliseconds, ictal periods and sleep–awake states in one-minute windows, and the anatomical origin of a channel. In the frozen single-channel comparison, PHASE-T leads three foundation-model baselines on every task, with the widest margins on Region-12, Lobe-5, and Ictal, which depend on regional physiology and minute-scale context. Its Region-12 macro-F1 is 31% higher than BrainBERT's, the strongest of the three baselines.

Frozen PHASE-T also outperforms specialist models on HFO, Sleep, Lobe-5, and Channel, even though each specialist was trained for its particular task with fixed input formats and prediction targets. After fine-tuning, PHASE-T sets a new state of the art on all tasks; its Outcome AUC of 0.7713 exceeds PyHFO-Omni, an event-based model built on expert HFO annotations.

These are benchmark scores on one evaluation suite, not clinical deployment results. The comparison is between model families under a shared probing protocol, and the frozen-versus-fine-tuned distinction matters: the frozen numbers test representation quality without task-specific adaptation, while the fine-tuned numbers test the best achievable performance.

Ablations show the physiological targets carry the gain

The strongest internal check is a matched ablation in which all four variants are frozen after 60,000 pretraining updates and probed with the same protocol. Latent-only drops the physiological targets; Latent+Recon replaces them with reconstruction of each masked token's clean 128-sample waveform patch; No-skip removes the fine-stage skip connection.

Against Latent-only, physiological supervision adds the most macro-F1 on brief events (HFO, 0.6276 to 0.8138), sleep (0.5398 to 0.7413), and anatomical localization (Region-12, 0.1996 to 0.3398). Latent+Recon leaves HFO and Region-12 essentially at the Latent-only level, and full PHASE-T beats it on all five tasks. The paper reads this as physiological supervision being a more useful pretraining signal than sample-level waveform reconstruction for these tasks. The skip connection contributes separately: removing it costs 0.15 macro-F1 on sleep and 0.13 on ictal, indicating that fine-scale features complement the coarse pathway's context.

Because every variant shares the pretraining data, update count, and probe, these gaps are attributable to the changed component within this setup rather than to training budget differences.

Transfer holds at unseen institutions with few local labels

Generalization matters clinically because recording setups differ across sites and expert labels are scarce. Fitted only on Omni-iEEG and applied unchanged, the PHASE-T probe identifies pathological channels at Tohoku and NCNP better than every supervised model trained on the same data, most clearly at NCNP (ROC-AUC 0.6292 against 0.5528 for SEEG-NET).

For cross-participant pathology classification after fine-tuning on four local participants, PHASE-T reaches a binary F1 of 0.7337 at MAYO against the best published 0.40, and exceeds the published comparator at FNUSA (0.6656 against 0.51). The two settings answer different questions — zero-local-label transfer versus four-participant adaptation — and the reported comparator values are published results rather than reruns under identical conditions.

Frozen embeddings encode seizure spread and regional identity

Task scores show the embeddings are useful; a separate analysis asks what they contain, without updating the encoder or fitting a probe. Across 200 seizures from 53 participants at the Hospital of the University of Pennsylvania, the paper measures each channel's distance from its own pre-ictal baseline in a space of up to 32 whitened principal components, expressed as a robust z-score, and marks a departure when a channel first stays above z = 5 for five consecutive seconds. When an ictal period begins, channels in the clinically marked seizure-onset zone move farther from baseline and peak earlier than the rest, even though PHASE-T's pretraining contains no ictal recordings.

Departure times are also ordered along the array: channels farther from the first departing channel depart later, with a median Spearman correlation of 0.60 over 706 stereo-EEG shaft–seizure pairs at HUP and 0.66 over 75 ECoG strip–seizure pairs from three other institutions, against 0.34 and 0.39 when departure times are shuffled within each array.

A regional map appears as well. Averaged by brain region, frozen embeddings of 16,444 ECoG channels line up across institutions: 14 of 16 region matches between UCLA and Detroit are correct, about seven times the two expected at random, and none of 2,000 region-label shuffles reached that count (permutation test, p = 5×10⁻⁴). These are descriptive analyses of frozen features, not evidence that the model was trained to represent seizures or anatomy.

Cross-channel context helps most where channels interact

PHASE-ST's contribution is uneven across tasks, and the pattern is informative. It further improves seizure-onset-zone identification over PHASE-T, and relation targets account for most of that gain from concurrent context — consistent with SOZ identification depending on how activity couples across channels rather than on any single channel alone.

On naturalistic listening in BrainTreebank, frozen PHASE-ST reaches mean ROC-AUCs of 0.9450 for volume and 0.8822 for pitch across seven evaluation participants, above the strongest published results of 0.92 and 0.83; fine-tuning raises these to 0.9515 and 0.8886. The paper's own framing is that PHASE-T and PHASE-ST are complementary: the temporal stage carries the broad clinical benchmark, while the spatial stage adds value on tasks that depend on synchronized channels.

The evidence stops at one-minute context and separate stages

The framework pretrains its temporal and spatial stages separately rather than jointly, and context is limited to one minute. The paper names joint pretraining, extension toward hour-long recordings, and broader patient cohorts and downstream evaluations as future work. Those boundaries qualify how far the reported gains extend: the five-task benchmark, the cross-institution transfer, and the auditory decoding results all sit inside this design.

The practical priority the results support is a specific one. When the goal is a frozen backbone that transfers across institutions with few or no local labels, the ablation comparison points to physiological targets over waveform reconstruction as the pretraining signal to keep, and the frozen-versus-fine-tuned split tells readers which number to compare against their own baseline. When the task depends on relationships among simultaneously recorded channels, the spatial stage is where the additional gain appears.

Sources used in this answer