What gaps exist in health technology assessment guidance for generative AI?

The gap is generative AI, not AI in general

Health technology assessment (HTA) bodies and regulators decide whether a technology's evidence is good enough for approval, coverage or pricing. The paper asks whether the guidance those bodies issue can handle generative AI, and its answer is narrow but firm: the guidance has begun to bend toward adaptive AI, yet none of the 56 documents examined contained specific provisions for generative AI technologies.

That distinction matters because the authors treat earlier AI as partly compatible with existing evaluation. Their example is an AI system that sorts mammography images into normal, benign or malignant: three discrete outputs with well-defined correctness boundaries. Generative AI does not share that bounded output space, which is where the evaluation problem starts.

How 56 documents were compared

The authors ran a comparative document analysis across six countries: Canada, France, Germany, South Korea, the United Kingdom and the United States. They searched national policy repositories and PubMed, restricting eligible records to those drafted by government institutions or HTA bodies, published between January 2010 and 1 November 2025, and setting out how clinical or economic evaluations must be performed.

The corpus deliberately mixes body types. Some included institutions are regulatory authorities governing market access, such as the FDA, the MHRA, Health Canada and South Korea's Ministry of Food and Drug Safety. Others are HTA or value assessment bodies informing reimbursement, such as NICE, Haute Autorité de Santé, IQWiG and Germany's Federal Joint Committee, and ICER in the United States. The authors justify this by noting that for AI and digital health the expectations around evidence increasingly blur across the regulatory and HTA boundary, so developers must satisfy both sides. Because the bodies differ in remit, cross-country differences cannot be read purely as disagreements about evidence.

RCTs and QALYs remain the common baseline

Across the 56 documents, the most consistent finding was convergence: a universal preference for randomised controlled trial evidence and for quality-adjusted life-year (QALY)-based economic evaluation. A QALY combines length of life with health-related quality of life into a single unit, which is what makes cost-per-QALY comparisons possible across technologies.

The paper also reports divergence in how jurisdictions accommodate AI's evolving nature. Several have begun incorporating alternative study designs and real-world evidence, and Germany and South Korea have introduced conditional reimbursement pathways intended to incentivise evidence generation after market entry. The authors describe these transitions as promising but incomplete, noting that no evidentiary standards currently exist for the structured integration of real-world evidence into HTA decision-making.

Institutions explain where flexibility appears

The authors argue that several cross-country differences track institutional structure rather than differing views on what counts as evidence. Germany's and South Korea's conditional reimbursement routes sit within systems already built around temporary coverage and post-market data collection, namely the DiGA Fast Track and the Korean conditional pathway. The firmer preference for QALY-based assessment in the UK and France is read as reflecting their single-payer threshold and medico-economic traditions.

The fragmented US picture, in which value assessment is performed by an independent body rather than a statutory HTA agency, is attributed to the absence of a statutory HTA authority linked to universal coverage. The paper's generalisation from this pattern is that flexibility has emerged earliest where reimbursement structures already accommodate staged or revisable evidence. This is the authors' interpretation of the cross-country pattern, not a tested causal claim.

Five mismatches between generative AI and HTA

The paper's analytical core is a set of mismatches between what defines generative AI and what current HTA practice assumes. First, a near-infinite space of possible outputs collides with the assumption of a bounded, enumerable output space. Second, sensitivity to prompting collides with standardised, stable inputs. Third, performance that evolves through adaptive learning collides with a static technology of fixed performance. Fourth, open-ended outputs without predetermined reference standards collide with the availability of predefined correct answers. Fifth, value realised through workflow integration collides with benefit being expressible as a direct clinical outcome.

Each mismatch is presented as producing a concrete evaluation gap. The authors also identify a methodological obstacle: current evaluation frameworks rely on decision-analytic models with fixed transition probabilities, such as Markov models and decision trees, which cannot accommodate the evolving performance of adaptive AI systems. Medical device HTA has struggled with learning curves and product modifications before, and the authors argue that AI's additional opacity substantially complicates that challenge.

Bayesian updating is identified but unused

One candidate adaptation the paper highlights is Bayesian methodology, which it calls a source of untapped potential for HTA of AI technologies. The stated advantage is formal updating: the posterior distributions from one analysis can serve as priors for the next as new performance data arrive. That dynamic updating aligns with both the continuous evolution of AI technologies and the iterative cycle in which foundation models are fine-tuned into application-specific products.

The gap is in application rather than recognition. France and the United States have recognised Bayesian methods in their evidence pathways, but the authors state that their application to AI-specific HTA challenges remains unexplored. This is a claim about the absence of documented use in the reviewed guidance, not a demonstration that Bayesian methods would work.

Equity guidance stays qualitative

Where social and equity evaluations were addressed, the paper reports that guidance was limited to qualitative approaches. The authors acknowledge that qualitative methods are essential for understanding the contextual dimensions of new technologies, but argue they may be insufficient to quantify the trade-offs inherent in adoption decisions about generative AI.

Their proposed direction is quantitative preference designs, with discrete choice experiments named as an example. The paper does not report testing such designs; it identifies them as a methodological gap in the reviewed guidance. This is one place where the reader should keep the authors' recommendation separate from their finding: the finding is that guidance is qualitative, and the recommendation is that quantitative preference work is needed.

What the corpus can and cannot support

The study's scope limits how far its conclusions travel. Eligible records had to be drafted by government institutions or HTA bodies and had to specify how clinical or economic evaluations are performed, so the corpus captures formal guidance rather than actual assessment decisions or developer practice. In the absence of a statutory US HTA body, ICER was included as the closest equivalent, a choice the authors make explicit. The authors also note that machine translation was treated as a reliability safeguard given the risks of relying on it for regulatory and legal texts.

The finding that no document contained generative AI-specific provisions is unsurprising in the authors' own framing, given how recently these technologies were introduced. That makes the paper best read as a map of where existing guidance stops, not as evidence that generative AI has failed assessment in practice.

Where adaptation is most urgent

The paper's practical priority follows from the mismatch list rather than from the convergence finding. A shared preference for RCTs and cost-per-QALY analysis is not the binding constraint, because those methods can in principle be applied to a generative tool. The binding constraints are the assumptions those methods carry: a bounded output space, stable inputs, fixed performance and predefined correct answers.

That ordering tells readers where to look first. Systems that already run conditional reimbursement and post-market data collection, such as Germany's and South Korea's pathways, have the institutional machinery closest to accommodating evolving performance, while the absence of standards for structured real-world evidence integration and the unexplored use of Bayesian updating in AI-specific HTA remain the concrete gaps to close.

Sources used in this answer