Can aligning multimodal patient evidence with biomedical knowledge graphs improve clinical LLMs?

MM-KG aligns patient observations to biomedical concepts

Clinical questions often hinge on a link between a patient's own measurements and general biomedical knowledge. The paper quantifies how common that is: across 28,582 MIMIC-IV ICU stays, 29.9% show a laboratory-evident organ state that never appears in the coded diagnoses while the patient receives a medication that PrimeKG lists as contraindicated for that state. Answering such items requires reasoning about the current measurement and the drug-specific relation together; either source alone leaves the question undecidable.

MM-KG (Multimodal Knowledge Graph) puts both sides in a single typed graph. Modality-specific harmonizers convert EHR text, imaging, genomic, and biospecimen data into typed observation nodes rather than one fused vector: a vision-language model turns chest radiographs into calibrated textual findings, brain MRIs are segmented into regional abnormality findings under a normative model, and codes, measurements, and report text are resolved by vocabulary lookup, value-aware rules, and entity linking. Observations are mapped to UMLS concepts, and a route-prioritized aligner links those concepts to a biomedical knowledge graph. Provenance edges keep each observation's source and timestamp.

The design choice that matters for evaluation is that the alignment is explicit. Because each relation is a separately addressable object, the patient–relation correspondence can be disrupted while the patient evidence and nearly all of the graph text stay in place. That is what makes the contribution testable rather than merely architectural.

A 2x2 design separates patient evidence from knowledge

The authors build MM-KGs for MIMIC-IV and ADNI and evaluate them with a 2x2 design that crosses patient evidence with biomedical knowledge, isolating each source and their interaction. On questions that require both sources, neither source alone performs far above chance, whereas their combination yields a drug-controlled AUROC interaction of +0.194 on MIMIC and +0.299 on ADNI.

The factorial numbers on the MIMIC five-candidate ranking benchmark (n=1,244 independent patients, chance Hits@1 0.200) show the same pattern in a different metric. Patient evidence alone reaches Hits@1 0.304, PrimeKG alone 0.278, neither channel 0.293, and the combination 0.575. The gap between the combined condition and either single channel is therefore the quantity the paper is arguing about, not the absolute score of any one configuration.

These are two different measures on two different cohorts and should not be merged: the AUROC interaction is a drug-controlled effect estimate, while the Hits@1 figures come from a five-candidate ranking task with a 0.200 chance baseline. Both point the same way, but they answer different questions.

Deleting the answer-bearing edge erases the gain

If the benefit comes from the correspondence between patient state and relation, then breaking that correspondence should remove it. The authors apply three disruptions to the combined condition on the MIMIC five-candidate benchmark: deleting the single answer-bearing relation from the answer candidate's packet, replacing that packet with a length-matched packet from another candidate, and shuffling packets across the five candidates.

All three fall below the combined condition (p < 0.001). Deleting the edge returns Hits@1 to 0.299, the level of the no-knowledge condition, and shuffling drives it to 0.185, below the 0.200 chance level. Matched wrong packets land at 0.237. Because the decoys are matched on prescription frequency, graph degree, packet length, and drug class, the drop reflects the loss of the correspondence rather than the loss of graph text.

The ADNI 70-item test set shows the same ordering after Holm correction: removing the decisive edge lowers AUROC from 0.860 to 0.714, matched control packets give 0.548, shuffled packets 0.531, and keeping node names while removing their relations returns performance to 0.558, the level of the no-knowledge condition. The paper also reports that on the ADNI validation set all 85 attempted KG citations referred to relations in the supplied packet, and after the decisive edge was removed the model never cited it.

The interaction localizes to the evidence block that matters

The patient side is not uniformly responsible. On MIMIC, removing laboratory values or report findings reduces Hits@1 by 0.148 and 0.123 (p < 0.001), whereas removing imaging findings or coded history has no measurable effect. Removing medications actually raises Hits@1 by 0.036, and removing provenance headers raises it by 0.027.

On the ADNI validation set, medical history alone with the knowledge graph reaches AUROC 0.831 against 0.853 for the full record, and it is the only patient block whose enablement estimate (+0.294) exceeds the prespecified minimum detectable interaction of 0.137.

The authors' reading is that the interaction arises from the evidence block containing the condition the question concerns, and that mismatched packets return both cohorts to the no-knowledge level. The provenance result is worth separating from the alignment result: provenance makes supplied evidence inspectable, whereas correct semantic alignment is what supports the decision. In a separate audit under an earlier five-packet rendering, 98.2% of outputs were valid JSON and 79.5% of cited edges existed in PrimeKG, but only 16.5% reproduced the exact answer path, with evidence precision 0.312 and recall 0.447.

Query-conditioned retrieval matches accuracy with 6.8x less context

Retrieving the correspondence is itself a cost problem. On the patient-disjoint MIMIC test split (2,398 items, 1,079 patients), scored once after all policies were frozen, MM-KG reaches AUROC 0.731 with 1,206 input tokens, 6.8x fewer than the 8,157 used by the strongest generic policy, SapBERT vector retrieval at 0.712. The remaining policies score 0.062 to 0.204 lower, and when capped at MM-KG's budget they recover the answer path for at most 26% of positive items and fall to 0.526–0.638.

On the drug-balanced sensitivity subset (n=1,418), whose candidate drugs appear in both label classes, the two leading policies tie at 0.695 against 0.693 while the token gap remains 6.7x. The authors therefore describe the consistent advantage as context efficiency rather than a uniform accuracy win.

Retrieval can fail in two distinct ways, and the paper separates them. MM-KG misses the answer path on only 4 of 1,199 positive items and resolves 753 positives, against 714 for the 32K full-context policy, which retrieves 98.2% of paths but loses 463 positives with the path present. On ADNI, varying path length from one to three hops shows both failure modes: a static relation packet falls below closed-book answering despite usually containing the gold path, and interactive graph querying is accurate at one hop but drops to 0.670 and 0.590 balanced accuracy at two and three hops.

Static graph context gives no gain on outcome prediction

On tasks that do not require combining the two sources, the knowledge graph adds little. For ICU mortality, the zero-shot LLM reaches AUROC 0.733 with the full MM-KG prompt against 0.570 with demographics alone, and the trained GNN reaches 0.804 against 0.778 for a laboratory-only model; laboratory values contribute most for both. Removing PrimeKG leaves mortality AUROC unchanged for both consumers, within run-to-run noise.

The multimodal patient representations still carry information on their own. The GNN outperforms logistic regression on 30-day readmission (0.644 against 0.548), the harmonizer's structured imaging findings exceed the radiologist's verbatim prose by +0.025 AUROC, and in ADNI adding biomarkers and imaging to clinical history improves prognosis balanced accuracy from 0.595 to 0.652.

Conversely, when a question concerns a relation directly, complete PrimeKG edges improve balanced accuracy on 1,920 pharmacological questions across all eight relation families, by +0.063 to +0.420, whereas entity names, length-matched incorrect neighborhoods, and corrupted names do not. The paper notes that the model follows behaviorally corroborated and contradicted edges alike, so traceability to an edge does not establish clinical correctness.

MM-KG leads external graph methods where patient identity fails

On held-out five-candidate ranking, MM-KG outperforms MindMap by +0.131 Hits@1 (95% CI [0.100, 0.163], p < 0.001). An adapted GraphCare-BAT model trained on 19,085 labeled pairs is statistically indistinguishable from MM-KG on the full split, but its score rises from 0.609 to 0.645 when the patient side of its input is removed (p < 0.001), indicating that it ranks candidates by properties of the drug rather than by their relation to the patient.

The identity-hard subset (n=598), where a candidate-only prior ranks the wrong drug and a formulary lookup cannot resolve the item, is where the methods separate: MM-KG reaches 0.582 while both GraphCare variants fall to 0.375 and 0.381. On ADNI, a deterministic PrimeKG lookup reaches AUROC 0.956, so that comparison measures recovery of an explicit patient–relation match rather than differences between methods; the five-candidate design could not be reproduced there because participant medication sets are too small to defeat candidate-identity shortcuts.

The authors summarize this as MM-KG matching the strongest existing method where candidate identity suffices and exceeding every external method where it does not. A strict-matching sensitivity analysis (n=380) preserves the ordering but is not conclusive at that sample size (p=0.060).

The graph helps as a link, not as background

The paper's own conclusion is that knowledge graphs benefit clinical LLMs not as background context but as explicit links between a patient's state and the relation a question requires. The evidence for that claim is the pattern across the factorial design, the disruption tests, and the outcome-prediction ablations: the interaction appears when a question requires both sources, disappears when the correspondence is broken, and does not appear when the graph is added as static context to ordinary prediction.

Two boundaries qualify how far this transfers. The held-out MIMIC results test model and retrieval-policy generalization within a previously designed benchmark, not an independent validation of benchmark construction, because the original 3,200-item join benchmark had no development–test split and shortcut controls were tuned while inspecting the complete item bank. The frozen graph and benchmark also use the bridge-v1 alignment, whereas separately quoted MIMIC mapping-coverage audits use bridge v2, so those coverage percentages should not be treated as properties of the frozen benchmark prompts.

The practical priority this paper sets is therefore not whether to add a knowledge graph to a clinical model, but whether the question requires a patient–relation correspondence. When it does, the retrievable, traceable, and ablatable alignment edge is the object that carries the gain, and query-conditioned retrieval delivers it at roughly one-seventh of the context. When the question is ordinary outcome prediction, the same graph adds nothing measurable, and the patient-side evidence block that contains the condition in question is what determines whether the interaction appears at all.

Sources used in this answer