How are large language models designed and evaluated in digital mental health?

A scoping review of 84 interactive LLM systems

The paper asks how LLM-based digital mental health systems are actually designed, used, and evaluated, rather than whether LLMs can produce fluent therapeutic-sounding text. The authors ran an HCI-centered scoping review: they searched seven databases (PubMed, Scopus, Web of Science, IEEE Xplore, ACM Digital Library, CINAHL, and Embase) for studies published between January 1, 2019 and May 31, 2026, combining LLM terms with broad mental health terms.

From 6,234 initial records, 84 studies met the inclusion criteria. Inclusion required that LLM-mediated language interaction be a central mechanism of mental health support or clinical assistance; non-interactive prediction or classification, conventional clinical NLP, and systems where a non-text modality was the primary mechanism were excluded, though voice, embodied, and immersive systems were kept when language interaction remained central. Each study was coded along six dimensions: mental health task, target stakeholder, interaction paradigm, foundation model, evaluation method, and evaluation outcomes and measures.

Because the included systems differ in objectives, populations, and designs, the authors synthesized descriptively and thematically rather than pooling effects in a meta-analysis. Counts and percentages describe the literature; they are not treatment effects.

Counseling and self-help absorb most of the field

Counseling and support was the largest task category (40 studies, 47.6%), followed by self-help and well-being (22 studies, 26.2%). Together these two accounted for 73.8% of included studies. Assessment and screening (6 studies, 7.1%) and psychoeducation (4 studies, 4.8%) were far smaller, and clinician support and crisis/risk support remained uncommon.

The authors interpret this concentration as reflecting both the conversational strengths of current LLMs and the lower clinical and regulatory barriers of supportive applications compared with high-risk diagnostic or treatment decisions. That is their explanation, not a measured cause. The distribution matters for readers because it means the published evidence is densest exactly where the consequences of system failure are comparatively bounded.

Growth was rapid: included studies rose from 3 in 2023 to 23 in 2024 and 48 in 2025, with ten more identified in 2026 up to the search date. Geographically, Asia contributed 44.0% (37 studies), Europe 26.2% (22), and North America 23.8% (20), with only small numbers from South America, the Middle East, or multiple regions.

Text chatbots dominate despite broader design exploration

Interaction paradigms stayed largely conventional. Counseling and support applications were predominantly delivered through text-based chatbots, while self-help and well-being applications used a broader range of paradigms. Some studies explored personalization, embodiment, multi-agent interaction, and workflow integration, but most systems still relied primarily on turn-based text interaction.

The authors read this as progress driven more by advances in underlying language models than by innovation in human–AI interaction. For a reader comparing systems, the practical implication is that a new model capability does not by itself change how a person experiences the intervention; the interaction design around it is where most of the field has not moved.

The evidence base also leaned heavily on GPT-family models, with relatively few systematic comparisons across alternative proprietary and open-source foundation models. The authors note this limits understanding of how model choice affects clinical utility, safety, robustness, reproducibility, and user experience.

Response quality is measured far more than symptoms

Evaluation focused on what is easiest to measure. Technical and response performance was assessed in 81.0% of studies, compared with 29.8% assessing mental health symptoms and clinical outcomes and 7.1% including longitudinal follow-up. These are different questions: a system can generate appropriate, fluent responses without any evidence that a user's symptoms improve.

Study designs were correspondingly early-stage. Mixed-methods user studies predominated at 42.9%, and only three studies were randomized controlled trials. Eight studies were classified as longitudinal or field evaluations. Outcome profiles differed across intended tasks, so a percentage for one task category should not be read as the field-wide rate for another.

Outcome domains were coded with a multi-label scheme, meaning a single study could count toward several domains and percentages across domains are not expected to sum to 100%. The denominator for outcome analyses was the number of studies within each task or evaluation-design category, with each study contributing at most once to a given domain.

Matching evaluation to the intended task and risk

Building on the observed patterns and on established evaluation guidance, the authors propose a task-oriented framework organized around the nine intended mental health task categories identified in the review. For each task it specifies priority evaluation domains, additional context-dependent domains, example measures, and suggested assessment timing.

The logic is that the right evidence depends on what the system claims to do. For counseling and support, priority domains include safety and harm, mental health symptoms and clinical outcomes, well-being and functioning, human–AI relational and experiential outcomes, technical and response performance, and usability; example measures include validated condition-relevant symptom measures such as PHQ-9 or GAD-7 where appropriate, therapeutic alliance or perceived-empathy measures, and structured clinician safety review, with follow-up when sustained therapeutic benefit is claimed.

For assessment and screening, priority shifts to technical and response performance, symptoms and clinical outcomes, and safety, with discrimination metrics such as sensitivity, specificity, AUROC, and precision/recall, comparison against an appropriate clinical reference standard, and false-negative and false-positive error analysis. For crisis and risk support, the framework emphasizes sensitivity or recall for high-risk cases, clinically consequential error analysis, harmful-response and failure-mode testing, and escalation or referral outcomes.

The authors describe the framework as an evidence-informed synthesis rather than a formal consensus guideline or prescriptive checklist. Its example measures were informed by the included literature and supplemented with task-relevant approaches from established evaluation guidance.

Lower-risk support is nearer translation than crisis care

The authors' central judgment is a split by intended use, not a single readiness verdict. Bounded, lower-risk supportive applications, particularly self-help and well-being without autonomous clinical decision-making, may be comparatively closer to real-world translation than higher-risk or clinically autonomous uses. That reflects both the larger body of evidence for supportive applications and the lower potential consequences of failure, not evidence that these applications are already ready for deployment.

Counseling and support may likewise be more amenable to translation when its role is clearly bounded, but stronger longitudinal and clinical evidence is needed when therapeutic benefit is claimed. Crisis and risk support and clinician-facing applications that inform consequential clinical decisions require a substantially higher evidentiary threshold, including safety, clinical effectiveness, longitudinal effects, human oversight, and integration into real-world clinical workflows.

The authors also argue for augmentation over replacement: LLMs may assist with lower-risk, time-consuming tasks such as patient education, symptom summarization, documentation, routine follow-up, and communication support, while diagnosis, suicide risk assessment, treatment planning, and crisis intervention should remain under the supervision of qualified professionals.

A practical gap reinforces this: many systems were not integrated with electronic health records, documentation, scheduling, referral pathways, crisis response protocols, or longitudinal follow-up, so even technically capable systems may add workload or disrupt clinical workflows. The review's own limits matter here too: it is a descriptive scoping review of published studies, its counts describe what has been reported rather than what works, and the framework is a synthesis rather than validated guidance. The most useful takeaway is the task-level threshold: judge an LLM mental health system by whether its evaluation matches its claimed use, and treat symptom and longitudinal evidence as the missing requirement wherever clinical benefit is asserted.

Sources used in this answer