Why post-hoc explanations raise fidelity concerns
Most explainable AI in medical imaging is post-hoc: an extra model or algorithm is applied to a trained black-box network to explain its predictions. The survey groups common examples into feature attribution methods, such as gradient-based LRP and CAM or perturbation-based LIME and Kernel SHAP, and concept attribution methods such as TCAV and CAR. These tools are often model-agnostic, so they can be attached to many already-trained models.
The paper's stated concern is fidelity: an explanation generated after training may not faithfully reflect how the black-box model actually reached its prediction. S-XAI is presented as the alternative because explainability is incorporated directly into training, so the explanation is generated by the model's own decision process rather than reconstructed around it. This is the paper's framing and rationale, not a measured head-to-head comparison of post-hoc and self-explainable methods.
Three levels where explanation can be built in
The survey's organizing contribution is a three-part taxonomy. Input explainability adds interpretable inputs alongside the raw medical image. Model explainability changes the architecture so that attention, human-defined concepts, or learned prototypes carry the reasoning. Output explainability makes the model generate an explanation together with its prediction, as text or as a visual counterfactual.
These levels answer different questions. Input-level work asks whether the model reasons from medically meaningful evidence; model-level work asks whether intermediate representations are inspectable; output-level work asks whether the model can state or show what would change its decision. Treating them as interchangeable would misread the survey, because each level constrains a different part of the pipeline.
Handcrafted features link predictions to clinical meaning
Input explainability integrates explainable feature engineering or knowledge graphs with deep visual features to jointly inform the prediction. Feature engineering is split into general features, which quantify basic visual attributes with domain-agnostic algorithms such as HOG for shape, color moments and histograms for color, GLCM or LBP for texture, and SIFT or SURF for local keypoints; clinical features, which quantify pathology-aware attributes; and hybrid combinations of the two.
The survey's Sankey summary shows a division of labor: feature engineering is used mainly for recognition-oriented tasks such as classification, segmentation, and detection, while knowledge graphs appear more often in reasoning-intensive tasks such as medical report generation and visual question answering. One cited example illustrates why engineered features matter: Hu et al. combined 1,967 handcrafted radiomic features with VGG16 deep features to predict anti-PD-1 therapy response in NSCLC from CT images, and seven of the nine most important predictors were handcrafted, five of them textural. The authors read this as evidence that tumor complexity and heterogeneity may be critically associated with treatment outcome.
The trade-off is explicit. General features are transparent but capture low-level patterns without clinical semantics, while clinical feature engineering depends on extensive domain expertise, can be subjective, and has limited generalizability. That is the stated motivation for hybrid approaches.
Attention, concepts, and prototypes make the model inspectable
Model explainability covers three design families: attention-based learning, concept-based learning, and prototype-based learning. Concept-based learning is the clearest case for readers new to the area: the model is trained around human-defined concepts, so its reasoning is expressed in units a clinician can name rather than in unnamed latent dimensions. Prototype-based learning instead compares inputs with learned reference examples that stand in for recognizable cases.
The survey also discusses how these designs are combined. When model-level explanation is fused, the paper argues that multi-layer or multi-view attention aggregation, image-concept integration, and prototype-based similarity aggregation are more favorable for explanation quality when the fused signals directly contribute to prediction and stay aligned with clinically relevant structures, concepts, or views. Fusion that merely combines latent features without interpretable structure is less likely to yield faithful, semantically coherent explanations. This is the authors' assessment of design quality, not a benchmark result.
Output explanations appear as text or counterfactuals
Output explainability means the model produces an explanation alongside its prediction. Textual explanations are grouped into three formats: fully structured, such as a list of findings marked present or absent; semi-structured, such as findings organized by anatomical region; and free-structured, such as a fluent radiology-style narrative. Visual counterfactual explanations instead generate an altered image and highlight the region whose change would flip the diagnosis, for example a boxed area where altered features would turn the label from pneumonia to healthy.
These formats serve different readers. Structured text is easier to check item by item, free text is closer to a radiologist's report, and a counterfactual shows a discriminating region rather than a saliency map. The survey treats them as distinct output-level options, so a claim about one format should not be transferred to another.
What the survey organizes but does not settle
The evidence here is a review, not a controlled experiment. The paper outlines desirable characteristics for assessing explanation quality and discusses major challenges and future research directions, but the packet does not provide a quantitative comparison of S-XAI methods against post-hoc baselines, nor a pooled performance estimate across the reviewed studies. The 200-plus paper count describes the scope of the review, not the strength of any single method.
The taxonomy is also a map rather than a ranking. A method placed under input, model, or output explainability is not thereby shown to be more faithful or more clinically useful than one placed elsewhere. Readers should treat the level labels as a way to locate a method's explanatory commitment, and check the individual study for its own data, modality, and task before drawing conclusions.
Use the explanation level to judge a claim
The survey's most practical consequence is a reading rule: identify which level a claimed explanation comes from before judging it. A handcrafted radiomic feature that ranks among the top predictors supports a claim about medically meaningful inputs, as in the NSCLC therapy-response example where seven of nine top predictors were handcrafted. A concept or prototype layer supports a claim about inspectable internal reasoning. A generated report or counterfactual supports a claim about communicable output. None of these automatically establishes that the model's prediction is correct or that a clinician would trust it.
The paper's own priority follows from this split. It argues that future input-level work should move beyond pure performance metrics and prioritize the creation of medically meaningful evidence, and it favors model-level fusion that preserves explicit links between intermediate representations and final decisions. For readers evaluating an S-XAI claim in medical imaging, the first question is therefore not whether an explanation exists, but whether the explanation level matches the claim being made.
Sources used in this answer
Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks
Organizes more than 200 medical imaging papers into input, model, and output explainability, arguing that explanations built into training are better aligned with a model's decision process than post-hoc explanations.