What the 63.2% retrieval score actually measures
The headline number comes from a 200-way retrieval task on THINGS-EEG2: an EEG recording of a seen image is used to pick the matching image from 200 candidates. The paper reports average Top-1 accuracy of 63.2% and Top-5 accuracy of 90.3% across ten subjects, compared with 50.9% and 79.7% for UBP, the strongest listed baseline. The paper describes these gains as +12.3% and +10.6% relative improvements.
Two conditions matter for reading that score. First, this is the intra-subject setting, where each subject's data are used in that subject's own evaluation; the same table shows every subject improving over UBP, from 48.2% to 70.6% Top-1. Second, the comparison is against the baselines reported in this paper, not against an exhaustive field. The much harder inter-subject setting, where the model is tested on unseen individuals, gives a far lower 17.0% average Top-1.
Why a frozen CLIP leaves alignment on the table
CLIP is a pretrained vision-language model that maps images and text into a shared embedding space. In the EEG-to-image pipelines the paper describes, CLIP is typically treated as a static feature extractor: image features are computed once and an EEG encoder is trained to match them. The authors argue this overlooks how much the visual representation itself could be adjusted to fit neural signals.
They also name a second obstacle, the physiological-symbolic gap. Text carries explicit symbols that can be aligned with image content, whereas EEG encodes visual information implicitly as distributed neural activity across time and frequency, with no human-annotated intermediate representation. NeuroCLIP's design is a response to both problems: make the visual side adaptable, and soften the contrastive objective so it does not demand exact one-to-one matching between noisy EEG and image features.
Two levels of visual prompting inside the transformer
NeuroCLIP keeps the CLIP vision transformer frozen and inserts learned prompts at two levels. The instance-level path starts from a second, filtered copy of the input image. A lightweight network generates a content-adaptive filter, producing a filtered image whose patch tokens are fused with the original image's patch tokens through cross-attention. This lets image content itself modulate which patch representations reach the transformer, rather than applying one fixed transformation to every image.
The shared-level path adds visual prompt tokens that are common across samples and inserted into the transformer alongside the class token and the fused patch tokens. The paper describes these as global, modality-level prompts that interact with the adjusted patch tokens during training. The distinction is the point: instance-level prompts vary with each image, while shared-level prompts carry a learned, dataset-wide adaptation. Both are trained while the CLIP backbone stays frozen, so the method changes how the visual representation is assembled rather than retraining the vision model.
Softening the contrastive objective for noisy EEG
Standard CLIP training uses a contrastive loss that pulls each EEG embedding toward its paired image and pushes it away from all others, treating every non-matching pair as equally negative. The paper argues this is a poor fit for EEG, where signals are noisy and semantically ambiguous, so some non-matching images may still be perceptually related to the recorded stimulus.
NeuroCLIP therefore combines three terms. The original CLIP contrastive loss is retained, and two additional losses are added: a soft alignment loss that builds targets from within-modality similarity distributions and compares them with cross-modal distributions using symmetric KL divergence, and a relation-aware regularization term that removes the positive diagonal, renormalizes, and compares the remaining negative distributions. The total objective is a weighted sum of the three. The authors state this soft contrastive design is adapted from vision-language work and is, in their account, the first application of it to EEG-image alignment.
Inter-subject generalization improves but stays low
The inter-subject table is the more demanding test and the more informative boundary. There, NeuroCLIP averages 17.0% Top-1 and 40.3% Top-5, against UBP's 12.4% and 33.4%, a gain the paper reports as +4.6% Top-1 and +6.9% Top-5. The improvement is consistent in direction but uneven in size: Subject 2 rises from 15.5% to 31.8% Top-1 and Subject 10 from 16.0% to 29.8%, while Subject 9 falls slightly from 15.5% to 9.9%.
That pattern matters more than the average alone. A 17.0% Top-1 score in a 200-way choice is well above the 0.5% chance level, but it is far from the intra-subject result, and the per-subject spread shows the method does not transfer uniformly across individuals. The paper attributes the inter-subject difficulty to higher EEG variability across people, and its own framing treats generalization as the condition where alignment strategy matters most.
Where the gains come from and what they cost
The paper's parameter table compares CLIP vision backbones used with NeuroCLIP. The reported gains over the corresponding baseline shrink as the backbone grows: +1.57% for ViT-B/16, +1.54% for ViT-B/32, +0.86% for ViT-L/14, and +0.68% for ViT-H/14. Runtime rises steeply over the same range, from 6.53 minutes on a single subject for ViT-B/16 to 200.01 minutes for ViT-H/14, with per-batch retrieval time going from 1.2 seconds to 5.99 seconds.
Read together, these numbers describe a trade-off rather than a simple scaling story: the smaller backbones show the largest prompt-tuning benefit at the lowest cost, while the largest backbone shows the smallest benefit at roughly thirty times the single-subject training time. The paper also reports additional experiments on THINGS-MEG, stating competitive intra- and inter-subject performance and consistent inter-subject improvements over UBP, but the extracted results section does not give the MEG table's values, so those claims cannot be checked numerically here.
Where the claim holds and where it stops
The evidence supports a specific claim: prompt tuning a frozen CLIP vision transformer, combined with a softened contrastive objective, improves zero-shot EEG-to-image retrieval on THINGS-EEG2 relative to the baselines reported in this paper, in both intra-subject and inter-subject evaluation. The intra-subject gain is large and consistent across all ten subjects; the inter-subject gain is smaller and uneven across subjects.
Several limits sit close to that claim. The EEG encoder is described by the authors as relatively shallow and task-specific, and the prompting behavior emerges implicitly from the architecture rather than being controlled explicitly. The prompt representation is visual only, so the framework does not yet connect EEG-derived prompts to a text encoder. And the inter-subject results, not the 63.2% headline, are the relevant reference point for any use where a new person's EEG must be decoded without subject-specific training. The paper's own priority is therefore the physiology-aware adaptation of the visual side, with the size of that benefit depending heavily on which evaluation setting and which backbone are chosen.
Sources used in this answer
NeuroCLIP: Brain-Inspired Prompt Tuning for EEG-to-Image Multimodal Contrastive Learning
On THINGS-EEG2, NeuroCLIP reaches 63.2% average Top-1 and 90.3% Top-5 zero-shot image retrieval across ten subjects, versus 50.9% and 79.7% for UBP, and 17.0% Top-1 under inter-subject evaluation versus 12.4% for UBP.