When do foundation models transfer effectively to electromyography tasks?

What the benchmark actually tests

EMG-FM-Bench is built to answer a transfer question rather than a leaderboard question: given a model pretrained on general time series or physiological signals, how well does it work on electromyography, and under what adaptation regime? The benchmark unifies 20 public EMG datasets containing over 1 million EMG segments and evaluates nine pretrained foundation-model configurations under one evaluation framework.

The datasets are deliberately heterogeneous. Fifteen are upper-limb gesture datasets, four are lower-limb locomotion datasets, and one is a sequence-to-sequence typing dataset. Sampling rates span 100 to 2400 Hz, channel counts span 6 to 128, and the classification datasets cover 3 to 52 gesture classes. That spread matters because the paper's central claim is about transfer across variation in users, sensing configurations, acquisition hardware, and downstream tasks, not about performance on a single clean dataset.

The nine evaluated configurations come from seven model families: MOMENT-S and MOMENT-B, Chronos-2-S and Chronos-2-B, TimesFM, Lag-Llama, PatchTST, UniTS, and BRANT. Parameter counts range from roughly 1M for PatchTST to over 500M for BRANT, which was pretrained on about 1.01 TB of intracranial EEG. The benchmark therefore mixes general-purpose time-series backbones with one electrophysiology-specific model.

Linear probing is not a shortcut to backbone choice

The first research question asks how pretrained models behave when frozen versus fully fine-tuned. Linear probing attaches a linear classification head to the representation produced by each backbone and keeps the pretrained weights frozen; full fine-tuning updates the backbone end to end. Both are evaluated under the same downstream protocols.

The paper's finding is that linear probing provides useful information about frozen representations, but does not fully reflect how the same models perform after end-to-end fine-tuning. Full fine-tuning can substantially change downstream EMG performance. In practical terms, a frozen-representation ranking is informative about the quality of the pretrained features, but it is not a reliable proxy for which backbone to deploy when the deployment regime involves full adaptation.

This distinction is easy to blur. Linear-probing accuracy and full-fine-tuning accuracy answer different questions: the first measures how linearly separable the frozen representation is, the second measures what the model can reach once its weights are allowed to move. The paper treats them as separate evidence rather than interchangeable scores.

Pretraining advantage is real but not universal

The second research question isolates the contribution of pretraining by comparing each pretrained model with the same architecture trained from scratch. This architecture-matched control is what makes the comparison interpretable: differences cannot be attributed to architecture or parameter count alone.

Across these comparisons, the benefit of pretraining varies substantially across models and is not universal. Pretrained initialization improves some models and datasets, while others perform similarly or better when trained from scratch. The discussion adds that pretraining gains are concentrated in a small subset of configurations and do not increase consistently with model scale.

That last point is a caution against reading strong fine-tuned performance as evidence of useful pretraining. A model can post a high downstream score after fine-tuning while its from-scratch counterpart performs comparably, in which case the pretrained weights are not the source of the advantage. The benchmark's contribution here is the control, not a single headline number.

Unseen users remain the hard case

The third research question concerns generalization to new users with limited labeled data. Performance decreases when models are evaluated on new users, and the paper reports that five-shot adaptation improves macro-F1 in 70.2% of evaluated model-dataset combinations, but recovers only part of the lost performance.

The denominator matters: 70.2% is a share of evaluated model-dataset combinations, not a share of users and not a guarantee of recovery. The paper also notes that limited labeled data does not remove substantial user-to-user variability, and the discussion recommends reporting subject-level variability and macro-F1 in addition to average accuracy, because class imbalance can obscure failure cases.

One dataset makes the evaluation protocol especially consequential. In the Mendeley Knee dataset, injury history is a participant-level label, so protocols in which the same participants appear during training and testing may let models exploit participant-specific signal characteristics. The paper states that its linear-probing, full-fine-tuning, scratch, and five-shot results for that dataset should be read as performance on seen participants, whereas held-out and leave-one-subject-out protocols give more informative estimates of generalization to unseen participants.

Performance tracks across EMG task types

The fourth research question asks how performance changes across different EMG tasks. The benchmark covers upper-limb classification, lower-limb classification, and continuous EMG-to-text decoding on the emg2qwerty typing dataset, which contains 5,262,671 keypress events across 1,135 sessions and 346.4 hours of recording.

Model performance is highly consistent between upper- and lower-limb classification and remains strongly correlated with continuous EMG-to-text decoding, although the strongest agreement occurs between the two classification categories. The paper reports a sensitivity check: excluding the Mendeley Knee dataset changes the upper- versus lower-limb rank correlation from 0.97 to 0.92 for accuracy and from 0.96 to 0.98 for macro-F1, without changing the qualitative conclusion.

The decoding task uses a different metric from classification. Models are trained with CTC loss and evaluated with character error rate, computed by concatenating predictions across windows to reconstruct full typed text. A correlation between classification performance and decoding performance is therefore a relationship between two different measurement scales, not a claim that one score can be substituted for the other.

The preprocessing pipeline sets the comparison

All datasets pass through a unified, sampling-rate-aware pipeline: resampling to a 1000-Hz grid, bandpass filtering at 20–450 Hz, 50/60-Hz notch filtering where the original sampling rate supports it, and per-channel, per-window Z-score normalization. Recordings acquired below 1000 Hz are bandpass filtered at their native rate before upsampling, with the upper cutoff constrained by the original Nyquist frequency.

That ordering is a stated limitation as well as a design choice. For recordings acquired below 1000 Hz, resampling harmonizes the temporal grid but does not recover frequency components above the original Nyquist frequency. The benchmark equalizes the interface across models; it does not equalize the underlying information content of the recordings.

Downstream tasks use non-overlapping 3-second windows, giving 3000 samples after resampling, with two exceptions. CapgMyo trials are about 1 second long, so each complete trial is used as one sample and temporally resampled to 3000 points. HD-EMG gait cycles are treated as single samples and reflect-padded to 3000 points when shorter. These exceptions preserve trial and gait-cycle boundaries rather than forcing every dataset into the same segmentation rule.

What the benchmark does not settle

The benchmark evaluates nine pretrained configurations under a common protocol, but the paper does not claim that this set exhausts the space of foundation models or EMG tasks. The BiLSTM reference included in the study is described as a representative conventional supervised reference rather than an exhaustive comparison with task-specific EMG architectures, and it is excluded from the linear-probing-versus-fine-tuning comparison because those settings require a pretrained checkpoint.

The results are also reported at the level of model-dataset combinations and task categories. Statements such as pretraining gains being concentrated in a small subset of configurations are aggregate observations across the benchmark, not predictions for a specific new dataset or a specific new user population. The paper's own framing is diagnostic: it is designed to show when and how transfer happens, not to certify a single best backbone.

The released artifacts are standardized datasets, preprocessing pipelines, and evaluation code, which is what makes the comparisons reproducible across the 20 datasets. The evidence for any individual claim in this article is the benchmark's own evaluation, not an external replication.

Match the adaptation regime to the deployment question

The paper's most decision-relevant lesson is a pairing rule. Frozen evaluation is useful for identifying broad performance tiers, but it is insufficient for selecting a backbone intended for full adaptation; models should be compared under the adaptation regime used at deployment. If the plan is to fine-tune end to end, a linear-probing ranking is the wrong selection signal.

The second rule follows from the architecture-matched controls: strong fine-tuned performance is not itself evidence of useful pretraining, so the comparison that matters is against the same model trained from scratch. The third is about user shift: because five-shot adaptation improved macro-F1 in 70.2% of evaluated model-dataset combinations while recovering only part of the lost performance, limited calibration should be treated as partial and task-dependent rather than as a fix for cross-user variability.

Read together, these rules point to a specific priority for anyone using this benchmark: decide first whether the deployment regime is frozen or fully adapted, then check whether the pretraining advantage survives the from-scratch control, and only then ask how much a small labeled set from new users actually closes the gap.

Sources used in this answer