Can vision-language models trace longitudinal evidence in brain MRI reasoning?

BrainTRACE scores the evidence chain, not the answer

Brain MRI interpretation is a longitudinal reasoning problem: radiologists compare serial studies, integrate findings across MRI sequences, localize them within volumetric anatomy, and translate that evidence into report-grounded assessments. BrainTRACE is built to test whether a vision-language model can trace that evidence structure, rather than produce a fluent description of one image.

The benchmark contains 7,273 scored VQA instances derived from 1,778 longitudinal patients, 7,299 MRI studies, and approximately 29k co-registered 3D MRI sequence volumes. Because items are scored against report-grounded references, a plausible-sounding answer is not automatically a correct one; the benchmark is designed to expose failures in the evidence chain that support clinical validity.

Five reasoning levels cross four evidence demands

BrainTRACE organizes questions by the clinical reasoning depth required, not as interchangeable visual recognition tasks. L1 covers image and acquisition recognition, such as sequence, plane, laterality, focal abnormality, and concise image description. L2 covers anatomical localization and single-study finding recognition, including lesion counting, size or volume categorization, signal and enhancement characterization, morphology, and asymmetry. L3 covers disease-specific interpretation and interval change, such as size change, new lesion appearance, resolution, enhancement evolution, mass-effect change, interval summaries, and volume change. L4 covers longitudinal trajectory or quantitative change, including progression pattern, first onset, peak timepoint, narrative trajectory, response category, and serial volume change. L5 covers case-level and integrated clinical judgment, including impression generation, disease-status assessment, comparative summaries, and differential diagnosis with supporting evidence.

Orthogonally, the benchmark annotates evidence demands: longitudinal comparison, report-grounded references, multi-sequence integration, and volumetric spatial evidence. This two-axis design matters because a model can pass a recognition item and still fail the same clinical target once the item requires serial comparison or report grounding.

Twenty configurations, one attempt, no leaking references

The evaluation spans 20 model configurations: open-weight general VLMs, open-weight medical VLMs, closed-source frontier VLMs, and available native-volume medical baselines. All models are evaluated zero-shot using only the permitted model-facing visual evidence, question, answer format, and allowed non-leaking metadata. Reports, segmentation masks, parsed findings, measurements, and hidden reference fields are never shown.

Each model receives one attempt per item, and malformed outputs, refusals, and abstentions remain in the denominator. For 3D evidence items, general VLMs use released rendered inputs, while native-volume medical baselines use their supported volume input interfaces. This means 3D evidence results compare supported interfaces on shared clinical targets rather than isolating native-volume representation as a controlled variable, so the 3D comparison should not be read as a clean test of volume representation alone.

Local recognition rarely composes into longitudinal synthesis

The headline result is not simply that current VLMs score low. The paper reports that current systems can identify isolated visual cues but rarely compose them into grounded longitudinal interpretations. In the authors' framing, their successes do not compose: local recognition rarely survives through longitudinal, report-grounded clinical synthesis.

That distinction changes how a reader should interpret any single high score. A model that correctly names a sequence or localizes a lesion on one study has demonstrated a local capability, not the ability to trace how that finding appears across sequences, changes across serial studies, or supports a case-level impression. The benchmark's level structure is what makes this separation visible.

Volumetric access alone does not close the gap

BrainTRACE supports rendered inputs compatible with standard VLM interfaces, a 3D-evidence condition, and a decomposed case-reasoning track that audits six steps in a longitudinal evidence chain. The gap between local recognition and grounded longitudinal interpretation persists under current 3D evidence and native-volume interfaces.

The authors interpret this as suggesting that volumetric access alone is insufficient without stronger multi-sequence, temporal, and evidence-grounded reasoning. Because the 3D comparison uses each baseline's supported interface rather than a controlled native-volume variable, the defensible reading is that adding volume input did not by itself produce the missing longitudinal composition.

Cohort breadth supports the benchmark but is not prevalence

The retained cohort is not restricted to a single demographic or acquisition slice. Patient-level demographics show broad age coverage and a balanced sex distribution (54.6% female, 45.4% male). Follow-up depth is substantial: 28.4% of patients have 3-12 month gaps, 23.4% have 1-2 year gaps, 29.4% have 2-5 year gaps, and 15.2% have gaps of 5 years or more, with 3.6% under 3 months. Most patients have three studies (55.2%), followed by four (19.7%), five (9.6%), and six or more (15.5%).

Sequence availability is high for T1w (100.0%), T2w (99.8%), FLAIR (98.8%), and SWI (91.5%), while MRA is present in only 0.8% of patients. Study-level acquisition metadata spans Philips (52.3%), Siemens (43.0%), and GE (4.8%), with 51.6% at 3.0T and 48.5% at 1.5T. The paper states these percentages describe retained-cohort breadth rather than population prevalence, so they should not be read as epidemiology.

Watch the level transition, not the aggregate score

For readers comparing brain MRI VLMs, the most decision-relevant signal in this paper is where performance breaks down across the five reasoning levels and four evidence demands, not a single aggregate number. A model that performs well on L1 and L2 but degrades at L3 through L5 has demonstrated recognition without the longitudinal, report-grounded composition that BrainTRACE is designed to test.

The decomposed case-reasoning track, which audits six steps in a longitudinal evidence chain, is the paper's most useful instrument for that question, because it localizes which step fails rather than only reporting that the final answer was wrong. The benchmark specification, evaluation lists, scoring implementation, scoring rubrics, and audit-record format are released to support reproducible progress, so future comparisons can be made at the same level and evidence-demand granularity.

Sources used in this answer