[NeurIPS 2025] SURVHTE-BENCH: Beyond Imputation — Rigorous Benchmarking for Causal Survival Analysis

TL;DR

In the high-stakes world of precision medicine, knowing who benefits from a treatment is as vital as knowing if the treatment works. However, right-censored survival data (where we lose track of patients before an event) breaks most standard machine learning tools. SURVHTE-BENCH is the first large-scale, systematic benchmark designed to stress-test 53 different Heterogeneous Treatment Effect (HTE) estimators. The verdict? While standard "impute and regress" models work in simple trials, Survival Meta-Learners and Causal Survival Forests are essential once censoring becomes heavy or assumptions get messy.

Problem & Motivation: The "Censoring" Gap in Causal Inference

Causal inference usually deals with "what-if" scenarios: What would have happened if Patient A took Drug X instead of Drug Y? In survival analysis, we add two layers of complexity:

1. Censoring: We don't see the event for everyone; some people leave the study early.
2. Assumption Violations: Often, the doctors' choice of treatment is biased (Confounding), or certain patient types never get specific treatments (Positivity violation).

Previously, every paper had its own "bespoke" simulation. If Method A looked better than Method B, was it because Method A was truly superior, or because the simulation happened to use a Cox distribution that favored it? SURVHTE-BENCH ends this "wild west" of evaluation by providing a unified, modular testing ground.

Methodology: The Three Families of Survival HTE

The authors categorize the sprawling landscape of 53 survival HTE methods into three distinct technical families:

1. Outcome Imputation: These methods try to "guess" the missing survival times (using Margin, IPCW-T, or Pseudo-observations) and then run standard CATE models like Double-ML or X-Learners.
2. Direct-Survival CATE Models: Frameworks like Causal Survival Forests (CSF) and SurvITE that are built from the ground up with survival-specific loss functions (e.g., Log-likelihood for censored data).
3. Survival Meta-Learners: Adaptations of S/T/Matching learners that use survival models (DeepSurv, Random Survival Forests) as the "base" to estimate Restricted Mean Survival Time (RMST).

The benchmark is a massive 8x5 grid: 8 Causal Configurations (varying ignorability, positivity, and informative censoring) crossed with 5 Survival Scenarios (Cox, AFT, Poisson distributions at various censoring rates).

Methodology - The Technical Core

The benchmark measures CATE RMSE—the error between the predicted and true treatment effect. The "DeepSurv" versions of survival meta-learners (S-Learner-Survival) leverage neural networks to model non-linear hazards, which the benchmark shows is a critical Inductive Bias when the data-generating process is Poisson or AFT-based.

Experiments & Results: The Death of Simple Imputation

The most striking finding is the Censoring Gradient.

• In Low-Censoring RCTs: Simple models like Double-ML with Margin imputation are highly effective.
• In High-Censoring / Observational Data: As censoring increases (Scenario D & E), direct survival modeling is no longer optional—it’s mandatory.

As shown in the Borda Count rankings below, S-Learner-Survival (using DeepSurv) and Matching-Survival consistently take the top spots. They are significantly more robust to "Informative Censoring"—where the reason a patient drops out is stochastically linked to their health outcome.

Key Insight from Real Data (HIV & Twins): In the HIV clinical trial (ACTG 175), the authors injected artificial censoring to see which models stayed consistent. Causal Survival Forests (green in Figure 4) remained remarkably stable, while Survival Meta-learners (red) showed high sensitivity to the censoring rate, suggesting that while Meta-learners are accurate on average, they might be "jittery" in high-drop-out clinical settings.

Critical Analysis & Conclusion

Takeaway: If you are working with Real-World Evidence (RWE) where censoring is likely informative or heavy (>50%), S-Learner-Survival and Causal Survival Forests are your safest bets. Avoid simple T-learners in imbalanced datasets (RCT-5%), as they fail when the treated subgroup is small.

Limitations: The current benchmark focuses on static treatments. In reality, treatments change over time (Dynamic Treatment Regimes). Furthermore, the benchmark uses RMST as the primary estimand; while robust, it doesn't always capture the nuances of time-varying hazard ratios.

SURVHTE-BENCH provides the first high-resolution map of the "Survival HTE" landscape. It proves that accounting for the physics of survival data—the censoring and the hazards—is far more effective than trying to "fix" the data through imputation.