Can a missingness-aware machine-learning score improve incident dementia risk prediction?

A probability score for all-cause dementia

ITU-DESC estimates the probability that a UK Biobank participant will receive an all-cause dementia diagnosis during available follow-up, using only baseline assessment data. The study followed participants for a median of 16.6 years, and incident dementia occurred in 2.27% of the analyzed cohort, so the outcome is rare and the classes are heavily imbalanced.

That rarity matters for reading the headline numbers. Average precision summarizes how well the model ranks the small positive group against everyone else; ROC AUC summarizes ranking across all thresholds. Neither is a clinical decision rule, and neither tells a reader what share of flagged people would actually develop dementia. The paper reports both because they answer related but distinct ranking questions.

Missing values can carry risk information

In UK Biobank, whether a measurement exists is not random. The paper notes that participants excluded under complete-case analysis differed in age, sex and education from those retained, and those characteristics are themselves dementia risk factors. A machine-learning model can therefore discriminate partly from the presence or absence of a measurement rather than from its value.

The authors treat missing-value handling as a design choice, not preprocessing. They use unconditional random hot-deck imputation, which fills a missing value with a value drawn from that variable's overall distribution. Because the donor is not matched to the participant, the imputed value carries no participant-specific information, and the pattern of which fields were missing is withheld from the model.

Thirty-eight predictors survive ten imputation seeds

The pipeline started from 7,977 candidate features after excluding leakage-prone fields such as hospital inpatient records and diagnosis-year fields, plus variables that mainly reflect data-collection procedures. Feature selection used BorutaSHAP, repeated with ten random-imputation seeds. The authors kept predictors chosen in at least eight of ten seeds, yielding 38 features.

This consensus rule is the paper's stability mechanism. The strict intersection of all ten seeds would have discarded nine features that eight or nine seeds agreed on, so the eight-of-ten cutoff trades a little strictness for a larger retained set. The final model is a CatBoost gradient-boosted decision tree trained on the all-incident, all-cause dementia label without class weighting or resampling.

All scores were tested on the same holdout set

Published performance figures for ANU-ADRI, CAIDE, DRS, UKBDRS and UKB-DRP come from different cohorts, outcome definitions and missing-data procedures, so the authors did not compare against those published numbers. Instead they applied the same random-imputation policy, outcome definitions and isotonic calibration to every comparator and evaluated all models on the same 50,041 holdout participants that no selection or fitting stage had seen.

This design removes cohort and prevalence differences as explanations for the ranking. It also introduces a trade-off the authors state directly: scores developed under different missing-data procedures may be attenuated when evaluated on randomly imputed data, particularly those containing variables with substantial missingness. APOE ε4 status, for example, was missing for 88,961 of 500,410 eligible participants, so a complete-case UKBDRS evaluation would have used a smaller, selected subset.

The margin over UKB-DRP is small

On the holdout set, ITU-DESC reached average precision 0.138 (95% CI 0.123–0.156) and ROC AUC 0.853 (0.844–0.863). UKB-DRP, the strongest comparator, reached 0.118 and 0.841. The confidence intervals for average precision overlap at the boundary, so the improvement is real in the authors' comparison but not large in absolute terms.

The paper also reports that predictions were well calibrated after outcome-specific isotonic recalibration, with calibration slopes near one and calibration-in-the-large near zero. That supports reading the output as an estimated probability rather than only a relative ranking, but the probabilities remain tied to this study's outcome definitions, follow-up and ascertainment framework.

New predictors appear beyond hand-built scores

Several selected predictors are absent from established dementia scores, including leukocyte telomere length and the liver enzymes aspartate aminotransferase and alanine aminotransferase. The authors connect these to existing biological discussion, such as links between a higher AST-to-ALT ratio and cognitive impairment, and the hypothesis that telomere-related aging adds information beyond chronological age.

Age was the most important predictor, and the model shared a core with UKB-DRP: APOE ε4, cognitive and task-performance measures, medication burden, mother's vital status or age at death, and long-standing illness. Sex was not retained by any seed, which the authors suggest means other selected variables already carry that information. Behavioral variables such as television time and mobile-phone use were selected, but the authors caution that associations may be indirect, with mobile-phone duration partly reflecting age or cohort effects.

Where the score does not transfer yet

The evaluation used binary classification rather than time-to-event analysis, and participants who died without a recorded dementia diagnosis were counted as non-cases. UKBDRS was originally developed in a competing-risk framework, so the AP and ROC AUC reported here should not be read as reproducing its published 14-year performance. The authors note that a sensitivity analysis treating death as a competing event supported the binary results at the 5- and 10-year horizons.

Random imputation also has costs. It may add noise and weaken genuine signal, and because donors come from each variable's marginal distribution, it attenuates relationships between variables that a conditional donor pool would keep. The authors frame the question of whether an imputation rule can preserve that structure while still withholding missingness patterns as open. Cost, clinical usability and implementation burden were explicitly outside the study's scope.

What the comparison actually establishes

The paper's specific contribution is not that a machine-learning score replaces clinical judgment, but that a leakage-aware selection pipeline over roughly 8,000 UK Biobank features can produce a 38-predictor score that ranks incident dementia better than established scores when all are forced onto the same missing-data policy and holdout set. The relevant comparison is ITU-DESC versus UKB-DRP at 0.138 versus 0.118 average precision, not versus any published figure.

The practical reading follows from that setup. The gain is a ranking improvement on a rare outcome in a research cohort, achieved with a model whose missingness handling deliberately discards information that other scores may implicitly use. Whether the same pipeline helps in other outcomes with informative missingness is, in the authors' framing, a test of the framework rather than of ITU-DESC itself.

Sources used in this answer