Conversational agents in healthcare: a systematic review
Open accessThe DOI matches its Crossref record; 55 of the 60 references it cites were found in a registry or an index; 14 of the 36 sources its claims cite were checked for support.
Abstract
Objective: Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes. Methods: We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were included if they focused on consumers or healthcare professionals; involved a conversational agent using any unconstrained natural language input; and reported evaluation measures resulting from user interaction with the system. Studies were screened by independent reviewers and Cohen's kappa measured inter-coder agreement. Results: The database search retrieved 1513 citations; 17 articles (14 different conversational agents) met the inclusion criteria. Dialogue management strategies were mostly finite-state and frame-based (6 and 7 conversational agents, respectively); agent-based strategies were present in one type of system. Two studies were randomized controlled trials (RCTs), 1 was cross-sectional, and the remaining were quasi-experimental. Half of the conversational agents supported consumers with health tasks such as self-care. The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04). Patient safety was rarely evaluated in the included studies. Conclusions: The use of conversational agents with unconstrained natural language input capabilities for health-related purposes is an emerging field of research, where the few published studies were mainly quasi-experimental, and rarely evaluated efficacy or safety. Future studies would benefit from more robust experimental designs and standardized reporting. Protocol Registration: The protocol for this systematic review is registered at PROSPERO with the number CRD42017065917.
Overview
CoChat Summary
TL;DR
This systematic review found that conversational agents with unconstrained natural language input were still rarely studied in healthcare: 17 articles covering 14 agents, mostly evaluated with quasi-experimental designs. Evidence of efficacy and safety was limited, with only one RCT showing reduced depression symptoms (effect size d = 0.44, p = .04).
Key findings
- The search retrieved 1513 citations; 17 articles evaluating 14 different conversational agents met inclusion criteria.
- Study designs were mainly weak for efficacy claims: 2 RCTs, 1 cross-sectional study, and the rest quasi-experimental. The paper notes that the RCTs had moderate to high risk of bias.
- Dialogue management was mostly finite-state (6 agents / 8 studies) or frame-based (7 agents / 8 studies); agent-based dialogue management appeared in 1 agent.
- The only RCT evaluating efficacy found a significant reduction in depression symptoms with effect size d = 0.44, p = .04; patient safety was rarely evaluated.
Methods
- Systematic review of studies of conversational agents using any unconstrained natural language input for health-related purposes; protocol registered at PROSPERO CRD42017065917 and the review followed PRISMA.
- Searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital Library in April 2017, updated in February 2018; included primary studies of consumers, caregivers, or healthcare professionals that tested the full system with human users and reported evaluation measures.
- Screening was done by independent reviewers with Cohen’s kappa for agreement; because of heterogeneity, the authors conducted a narrative synthesis rather than a meta-analysis.
Limitations
- The paper states it focused only on agents with unconstrained natural language input, excluding constrained-input systems that are also relevant to healthcare.
- The paper states that the small number of included studies, heterogeneity of agents, and predominance of quasi-experimental pilot studies limited its ability to conduct a meta-analysis.
- Evident from the text: reporting was inconsistent, making eligibility assessment and synthesis difficult, and several studies lacked clear funding/conflict-of-interest reporting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
10 Valid · 3 Not valid · 13 Not checked · 25 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“A crossover RCT evaluated the diagnostic performance for depression of an embodied conversational agent compared to a psychiatrist, finding a sensitivity of 49% and specificity of 93%.”
§ Conversational agents in healthcare: a systematic review > Results > Evaluation measures
What the sources say
- Supportedcites [22] Philip, 2017, Virtual human as a new diagnostic tool, a proof of concept study in the field of major depressive disorders, Sci Rep source
The Method (Recorded data in the study; page not given) says patients completed both interviews in randomized order, one with the ECA and one with the psychiatrist. The Results (Global performance of ECA diagnosis of MDD; page not given) report overall sensitivity of 49% and specificity of 93% against the psychiatrist’s diagnosis. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“We found that only one study evaluated the efficacy of a conversational agent on health outcomes using adequate study design methods (RCT) and validated questionnaires.”
§ Conversational agents in healthcare: a systematic review > DISCUSSION > Standards for reporting the evaluation of conversational agents in healthcare
What the sources say
- Supportedcites [19] Fitzpatrick, 2017, Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial, JMIR Ment Health source
The Results report that participants were randomized to Woebot or an information-control group, and the Methods > Measures section identifies the PHQ-9 and GAD-7 as validated questionnaires. This confirms that the cited paper is an RCT evaluating a conversational agent with validated health-outcome measures. “Only one study” describes the citing authors’ review finding, not a claim made by this source. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Over the last two decades, a solid body of evidence has shown the potential benefits of using embodied conversational agents for health-related purposes.”
§ Conversational agents in healthcare: a systematic review > Introduction
What the sources say
- Not checked — no full text of the sourcecites [3] Bickmore, 2013, A randomized controlled trial of an automated exercise coach for older adults, J Am Geriatr Soc source
- Partly supportedcites [4] Bickmore, 2013, Automated interventions for multiple health behaviors using conversational agents, Patient Educ Couns sourceExperimental — accuracy not yet measured
The Results and Discussion report benefits from this conversational-agent intervention, particularly increased fruit and vegetable consumption compared with the control group. The Conclusion also describes the intervention as effective at changing health behavior. The paper does not establish the claim’s broader characterization of a solid body of evidence accumulated over two decades. (LLM, conf=0.86)
- Partly supportedcites [5] Watson, 2012, An internet-based virtual coach to promote physical activity adherence in overweight adults: randomized controlled trial, J Med Internet Res sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) describes prior research on embodied computer agents, and the Discussion conclusion reports that a virtual coach helped maintain activity in overweight participants. This supports potential health benefits, but the paper does not establish a solid body of evidence spanning two decades or benefits across health-related purposes generally. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [6] Edwards, 2013, Use of an interactive computer agent to support breastfeeding, Matern Child Health J source
boostedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04).”
§ Conversational agents in healthcare: a systematic review > Abstract > Results
What the sources say
- Partly supportedcites [19] Fitzpatrick, 2017, Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial, JMIR Ment Health sourceExperimental — accuracy not yet measured
The Results (Preliminary Efficacy) report a significant reduction in depression symptoms for the Woebot group relative to the information control group, with d=0.44 and a Bonferroni-corrected P=.04. The Discussion (Principal Results) calls this, to the authors’ knowledge, the first randomized trial of a nonembodied text-based conversational agent designed for therapeutic use; it does not establish that it was the only RCT evaluating any conversational agent. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Several randomized controlled trials of interventions involving embodied conversational agents have shown significant improvements in physical activity, fruit and vegetable consumption, and accessibility to online health information, among other outcomes.”
§ Conversational agents in healthcare: a systematic review > Introduction
What the sources say
- Not checked — no full text of the sourcecites [3] Bickmore, 2013, A randomized controlled trial of an automated exercise coach for older adults, J Am Geriatr Soc source
- Partly supportedcites [4] Bickmore, 2013, Automated interventions for multiple health behaviors using conversational agents, Patient Educ Couns sourceExperimental — accuracy not yet measured
The Abstract (Methods) describes one randomized trial, not several. The Results (§3.1–3.3) report a significant improvement in fruit and vegetable consumption and faster growth in walking steps for the physical-activity group, but no significant difference in self-reported physical activity. Our copy does not report an outcome concerning accessibility to online health information. (LLM, conf=0.90)
- Partly supportedcites [5] Watson, 2012, An internet-based virtual coach to promote physical activity adherence in overweight adults: randomized controlled trial, J Med Internet Res sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Not checked — no full text of the sourcecites [6] Edwards, 2013, Use of an interactive computer agent to support breastfeeding, Matern Child Health J source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
17 more claims · 2 with their cited sources checked
Create a free account to see all 51 claims and every source verdict.
- Probably not a substantial claim. 12
- It may state two separate claims as one. 20
- Borderline: one of the tests for a substantial claim is unclear. 1
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Previously conducted reviews [5,18,19] did not assess the effectiveness and safety of chatbots in mental health.”
From Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis (2020) · cites this paper as [18]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most studies reviewed in 2018 [14] were limited to task-oriented conversational agents to support patients and clinicians in highly specific scenarios (e.g., information retrieval and predefined clinical interview), restricting user input to predetermined utterances.”
From Conversational Affective Social Robots for Ageing and Dementia Support (2022) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the same note, conversational agents have been proven a valuable tool to deliver digital health interventions [7-9].”
From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These databases were chosen as they cover relevant aspects in the fields of medicine, technology, and interdisciplinary research and have also been used in other systematic reviews covering similar topics [7,8].”
From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing reviews on conversational agents focus on a certain subtype of agents such as virtual coaches [31-33] or embodied conversational agents (ECAs) [34] or on specific functionalities of these agents such as behavior change [35] or mental health applications [36,37].”
From Conversational Agents in Health Care: Scoping Review and Conceptual Analysis (2020) · cites this paper as [32]
Experimental — accuracy not yet measured
62 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 1×
“This systematic review is compliant with the PRISMA statement.(18)”
MethodCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · cited 1×
“49,50 Implications for healthcare, public health, and future research Patient safety was rarely evaluated in the included studies.”
BackgroundEvaluating the quality of medical care
DONABEDIAN, A. (2005) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2005.00397.x · Cited by 6,102 · cited 2×
“…of a conversational system from a health research perspective was considered to involve any health-related results present in the included studies, including process and outcome measures as defined by Donabedian; 16 for example, effectiveness in symptom reduction, diagnostic accuracy, or referrals.”
BackgroundELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“The first well-established chatbot of this kind—ELIZA—was programmed in 1966 to simulate a text-based conversation with a psychotherapist.(2)”
MethodA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“10,56,57 Therefore, the use of conversational agents with unconstrained natural language input capabilities and other artificial intelligence applications in healthcare needs to be carefully monitored.”
BackgroundSTARD 2015: an updated list of essential items for reporting diagnostic accuracy studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · BMJ · doi:10.1136/bmj.h5527 · Cited by 3,660 · cited 1×
“…telehealth (CONSORT-EHEALTH), 46 the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) statement, 47 and the Standards for Reporting Diagnostic accuracy studies (STARD), 48 among others from the Enhancing the Quality and Transparency Of health Research (EQUATOR) network.”
MethodDes Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,738
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 7×
“The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms.(19) Two studies comparing diagnostic performance of conversational agents and clinicians found acceptable sensitivity and specificity.”
Influential
Research works printed but not linked: 2
- Radziwill, 2017, arXiv Prepr, 1704
- Coiera, 2017, First compute no harm, BMJ Opin
Cited by
Items 701–800 of 1,371
Oh, Y. J., Zhang, J., Fang, M.-L. E., & Fukuoka, Y. (2020) · Research Square · doi:10.21203/rs.3.rs-121903/v1 · Cited by 6
On the Generation of Medical Dialogues for COVID19Preprint
Yang, W., Zeng, G., Tan, B., Ju, Z., Chakravorty, S., He, X., Chen, S., Yang, X., Wu, Q., Zhou, Y., et al. (2020) · doi:10.36227/techrxiv.12290345 · Cited by 5
Sak, M. E., Sonlu, S., & Güdükbay, U. (2025) · Computer Animation and Virtual Worlds · doi:10.1002/cav.70033 · Cited by 5
“Conversa-tional systems can input user queries as speech [54] or text [55]; additional information such as the user gaze or facial expressions [56] enables more accurate responses.”
With a little help from my artificial friend: Functional and parasocial value in ChatGPT use
Lasarov, W., Hoffmann, S., & Dwivedi, Y. K. (2026) · Technology in Society · doi:10.1016/j.techsoc.2026.103220 · Cited by 5
Luan, B., & Li, Y. (2026) · BMC Public Health · doi:10.1186/s12889-026-26328-1 · Cited by 5
“…randomized controlled trials specifically targeting older populations remain limited, voice-based interactions and multimodal inputs have lowered the barriers of digital accessibility, suggesting considerable transferability and scalability for promoting mental health in later life [21, 22].”
AI-Powered Physiotherapy: Evaluating LLMs Against Students in Clinical Rehabilitation Scenarios
Michou, I., Fouras, A., Chrysanthakopoulou, D., Theodoritsi, M., Stellatou, S., Mariettou, S., & Koutsojannis, C. M. (2026) · Applied Sciences · doi:10.3390/app16031165 · Cited by 8
Mutluer, T., Fıçıcıoğlu, İ. E. G., Uysal, M., Ekinci, S., Coskun, C. N., Mete, H. K., Çaltıner, Ş. N. İ., Bugay, B. Z., Sever, E., & Başdoğan, Ç. (2026) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-025-02867-7 · Cited by 12
“Clinically, AI tools help address persistent barriers such as stigma, provider shortages, and logistical constraints [23, 68 ].”
Shen, S., Qi, W., Zeng, J., Li, S., Liu, X., Yaping, F., & Cao, S. (2026) · Nurse Education in Practice · doi:10.1016/j.nepr.2026.104799 · Cited by 7
Allan, D. D. (2023) · Behaviour and Information Technology · doi:10.1080/0144929x.2023.2259500 · Cited by 6
“A growing literature suggests messaging-based conversational agents (Henceforth CAs) – software programs that use mobile messaging as an interface as capable of administering the guidance otherwise provided by healthcare professionals (Abd-Alrazaq et al. 2021; He et al. 2022; Laranjo et al. 2018).”
Developing a Generic Focus Modality for Multimodal Interactive Environments
Barros, F., Teixeira, A. J. S., & Silva, S. S. (2023) · ICMI Companion · doi:10.1145/3610661.3617165 · Cited by 3
“Furthermore, the advances in speech technologies have boosted the appearance of conversational agents to assist humans in their lives [7, 24].”
BackgroundViewpoint: Hybrid Intelligence Supports Application Development for Diabetes Lifestyle Management
Dudzik, B., van der Waa, J., Chen, P., Dobbe, R., de Troya, I. M. D. R., Bakker, R. M., de Boer, M., Smit, Q. T. S., Dell’Anna, D., Erdogan, E., et al. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15916 · Cited by 9
Can large language models help solve the cost problem for the right to explanation?
Munch, L. A., & Bjerring, J. C. (2024) · Journal of Medical Ethics · doi:10.1136/jme-2023-109737 · Cited by 4
Abe, K., Quan, C., Cao, S., & Luo, Z. (2025) · Applied Sciences · doi:10.3390/app15073466 · Cited by 7
Vowels, L. M., Sweeney, S. K., & Vowels, M. J. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100141 · Cited by 14
Six, S., Schlesener, E. A., Hill, V., Babu, S. V., & Byrne, K. A. (2025) · JMIR Mental Health · doi:10.2196/67381 · Cited by 17
“The term “virtual agent” refers to a noncontrollable, artificial intelligence (AI)-driven virtual entity, such as chatbots and embodied conversational agents (ECAs) designed to interact with users [16-18].”
Lee, C.-H., Hwang, Y., & Song, H. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2491024 · Cited by 4
“Previous research has predominantly emphasized the effects of communication content and the modality (Laranjo et al., 2018), with less attention given to the form in which that content is delivered.”
Vankit, S. A., Zamanifard, S., Diaz, D., Mousas, C., Richardson, K. C., Duchowski, A. T., & Volonte, M. (2025) · Computer Animation and Virtual Worlds · doi:10.1002/cav.70023 · Cited by 9
Torres, D. R., Wermelinger, E. D., & Ferreira, A. P. (2025) · Saúde em Debate · doi:10.1590/2358-2898202514510070p · Cited by 66
Navigating relationships with GenAI chatbots: User attitudes, acceptability, and potential
Vowels, L. M., Francois‐Walcott, R. R. R., Grandjean, M., Darwiche, J., & Vowels, M. J. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100183 · Cited by 7
Meloni, A., Recupero, D. R., Osborne, F., Salatino, A. A., Motta, E., Vahdati, S., & Lehmann, J. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3757923 · Cited by 14
“…generation [9], academic writing and citation enhancement [13, 15], supporting analytical dashboards [3, 70], automated research hypothesis generation [11, 12, 36], the development of knowledge organization systems [60], and the creation of specialized conversational agents [41, 49], among others.”
Professional representation of conversational agents for health care: a scoping review protocol
MacNeill, A. L., MacNeill, L., Doucet, S. A., & Luke, A. (2021) · JBI Evidence Synthesis · doi:10.11124/jbies-20-00589 · Cited by 6
The Therapeutic Alliance May Yet Prove Effective
O’Hagan, E. T., Leake, H. B., Heathcote, L. C., Stanton, T. R., & Moseley, G. L. (2021) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2021.0203 · Cited by 4
Spagnolli, A., Cenzato, G., & Gamberini, L. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20126182 · Cited by 4
“[23] reviewed 17 studies that assessed the user experience and efficacy of conversational agents with unconstrained natural language.”
BackgroundWahbeh, A., Al-Ramahi, M. A., El-Gayar, O., Elnoshokaty, A., & Nasralah, T. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.392 · Cited by 8
“Despite the fact that conversational agents are widely used in different domains, their use in the domain of healthcare and well-being is relatively low (Laranjo et al., 2018).”
A Hybrid Ayurvedic Drug Recommendation System with Generative AI
Singh, N., Sah, A., Vartika, Mishra, S., Malik, V., Sobti, R., & Brahma, B. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-3588-4_15 · Cited by 5
da Costa, F. L., Matzenbacher, L. S., Maia, I. S., Gheno, V., Brum, M. A. B., de Barros, L. G. B., Blank, L. M., & Telo, G. H. (2024) · Acta Diabetologica · doi:10.1007/s00592-024-02377-z · Cited by 4
AyurChat: Empowering Health through AI-Driven Ayurvedic Guidance
Ahmed, S. A., Gahlawat, G., Imam, F., & Saranya, G. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10724133 · Cited by 5
“In order to better understand conversational agents' potential to improve patient-provider communication and access to care, laranjo et al. [7] undertook a systematic evaluation of conversational agents in healthcare.”
BackgroundAhmed, A. E., Fatani, D. R., Vargas, J. M., AlMutlak, M., Helayel, H. B., Fairaq, R., & Alabdulhadi, H. (2025) · Cureus · doi:10.7759/cureus.78069 · Cited by 10
“Chatbots have the potential to improve patient outcomes and reduce healthcare costs by providing a more efficient and accessible means of healthcare delivery [3].”
Generating Facial Expression Data: Computational and Experimental Evidence
Vaitonyté, J., Blomsma, P. A., Alimardani, M., & Louwerse, M. M. (2019) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3308532.3329443 · Cited by 4
“Interest in applying Embodied Conversational Agents (ECAs) to various real-world scenarios, including domains such as education [10] and healthcare [9, 14], has been growing.”
BackgroundAnalyzing MoLIC's Applicability to Model the interaction of Conversational Agents
Fernandes, U. D. S., Prates, R. O., Chagas, B. A., et al. (2021) · doi:10.1145/3472301.3484367 · Cited by 4
“Por exemplo, os chatbots podem ser usados para personalizar o atendimento junto aos usuários e possuem a vantagem de estarem disponíveis 24 horas por dia, 7 dias por semana e, com isso, podem reduzir o tempo de atendimento e espera dos pacientes que necessitem de atendimentos mais urgentes [6][7].”
BackgroundAlloatti, F., Tasso, R., Bolioli, A., Messina, E., Cantù, D., Guidorzi, E., Agamennone, C., Brunetti, E., & Marinello, R. (2021) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc53001.2021.9631519 · Cited by 4
Neef, C., Mai, V., & Richert, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05412-9_35 · Cited by 6
Ha, J.-Y., & Park, H.-J. (2023) · Journal of Korean Academy of Nursing · doi:10.4040/jkan.22117 · Cited by 5
AI Enabled Chatbot for regulating Mental Health
Tomar, K. J., Khushwaha, A., Giri, A., & Banoth, S. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.51855 · Cited by 4
Leveraging AI in Oncology and Enhancing Patient Satisfaction -- Review Literature
Sajan, S., & John, L. A. (2023) · Acta Scientific Cancer Biology · doi:10.31080/ascb.2023.07.0447 · Cited by 4
Implementation of Anxiety UK’s Ask Anxia Chatbot Service: Lessons Learned
Collins, L. C., Nicholson, N., Lidbetter, N., Smithson, D., & Baker, P. (2024) · JMIR Human Factors · doi:10.2196/53897 · Cited by 6
“…interfaces of varying complexity, from rule-based chatbots that produce prewritten responses based on recognizing programmed terms and phrases to embodied conversational agents manifesting as a computer-generated avatar and smart conversational interfaces such as Apple’s Siri or Amazon’s Alexa [1].”
BackgroundUnderstanding Voice Search Behavior: Review and Synthesis of Research
Xing, Z., Yuan, X., Wu, D., Huang, Y., & Mostafa, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60117-1_23 · Cited by 10
Hocking, J., Oster, C., & Maeder, A. (2020) · JBI Evidence Synthesis · doi:10.11124/jbies-20-00225 · Cited by 4
Artificial Intelligence-based Healthcare Interventions: A Systematic Review
Park, G., Lee, H., & Lee, M. (2021) · Korean Journal of Adult Nursing · doi:10.7475/kjan.2021.33.5.427 · Cited by 5
Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational RepresentationsPreprint
Coope, S., Farghly, T., Gerz, D., Vulić, I., & Henderson, M. (2020) · arXiv (Cornell University) · doi:10.17863/cam.53929 · Cited by 4
The professional representation of conversational agents for health care: A scoping review protocolPreprint
MacNeill, A. L., MacNeill, L., Luke, A., & Doucet, S. A. (2021) · doi:10.17605/osf.io/d8v2q · Cited by 4
Baptista, S., Wadley, G., Bird, D., Oldenburg, B., Speight, J., & The My Diabetes Coach Research Group (2020) · JMIR mHealth and uHealth · doi:10.2196/17038 · Cited by 3
“This is an important finding because people with T2D often experience diabetes-related the consequences of which can include disengagement with or sub-optimal self-care and diabetes-related health outcomes [42].”
BackgroundInfluentialUnveiling Information Through Narrative In Conversational Information Seeking
Javadi, V. S., Trippas, J. R., & Flek, L. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665884 · Cited by 8
“This includes applications in language learning [90], communication skills [21], museum guides [75], and health consultations [47].”
BackgroundPerson-based design and evaluation of MIA, a digital medical interview assistant for radiology
Denecke, K., Reichenpfader, D., Willi, D., Kennel, K., Bonél, H., Nairz, K., Čihorić, N., Papaux, D., & von Tengg‐Kobligk, H. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1431156 · Cited by 5
The Evolution and Potential of Conversational Agents in Healthcare
Lal, M., & Neduncheliyan, S. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-75861-4_18 · Cited by 3
Bianchi, J., & Zheng, M. S. (2025) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12949 · Cited by 7
Jbene, M., Chehri, A., Saadane, R., Tigani, S., & Jeon, G. (2025) · Cognitive Computation · doi:10.1007/s12559-025-10470-w · Cited by 4
“Conversational agents, also known as chatbots or dialogue systems, are computational systems designed to emulate human-like conversation across various communication modalities, including speech, text, facial expressions, and gestures [1].”
A review on recent advancements of ChatGPT and datafication in healthcare applications
Jagatheesaperumal, S. K., Pandiyarajan, A., Prabadevi, B., Deepa, N., Barreto, A. G., & de Albuquerque, V. H. C. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110885 · Cited by 6
Kim, D. D., Zhang, J., Sagae, K., DeVon, H. A., Hoffmann, T. J., Rountree, L., & Fukuoka, Y. (2025) · Journal of Medical Internet Research · doi:10.2196/73184 · Cited by 8
Lim, S., Yang, Q., Meng, J., & Rains, S. A. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2567989 · Cited by 3
“As research has accumulated, numerous systematic reviews have been conducted to summarise the effects of conversational AIs in improving health behaviours and outcomes (e.g. Chattopadhyay et al. 2020; Laranjo et al. 2018).”
Du, J., Jin, T., Niu, R., Zhai, Y., & Fu, X. (2024) · International Symposium of Chinese CHI (Chinese CHI) · doi:10.1145/3758871.3758891 · Cited by 7
“In CAs research with older adults, early reviews have focused on specific healthcare and monitoring scenarios [15, 20].”
Kip, M., Bozan, G. S., & Şahin, G. A. (2025) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.70162 · Cited by 7
Zhang, Q., Wong, A. K. C., & Hui, V. C. C. (2025) · Systematic Reviews · doi:10.1186/s13643-025-02996-9 · Cited by 4
“Notably, as conversational flexibility improves, the likelihood of errors related to natural language understanding or response generation may also rise [55].”
Doğan, U., & Yılmaz, İ. E. (2025) · Informatics for Health and Social Care · doi:10.1080/17538157.2025.2595015 · Cited by 4
“12,15 This study aims to compare the information obtained by nursing students regarding DES from an AI chatbot and a web-based source in terms of knowledge acquisition, accuracy, readability, understandability, and actionability.”
Conversational agents and equity in health care systems: a scoping review
Morias, A., & Renmans, D. (2026) · Journal of Health Equity · doi:10.1080/29944694.2025.2606723 · Cited by 3
Hu, P., Wang, L., Chang, D., Zhu, Z., Chen, Y., Ren, Z., & Dumayi, H. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2605181 · Cited by 5
“…AI systems, such as virtual assistants, chatbots, and automated customer service agents, make interactions more intuitive and accessible, and are widely used in sectors like customer service and healthcare to enhance efficiency and user satisfaction (Dignum, 2017; Laranjo et al., 2018).”
Alzahrani, L., & Weerakkody, V. (2026) · Systems · doi:10.3390/systems14010113 · Cited by 5
“Safety concerns have also been raised, especially in relation to chatbots that provide health-related information or advice [15].”
Artificial intelligence and theory of mind
Matta, D. (. (2026) · Journal of Psychology and AI · doi:10.1080/29974100.2026.2628373 · Cited by 10
Lemberger‐Truelove, M. E. (2026) · The Journal of Humanistic Counseling · doi:10.1002/johc.70040 · Cited by 3
Yang, H., Chang, F., Muroi, F., Liu, Z., Zhang, W., & Cai, J. (2026) · Psychotherapy and Psychosomatics · doi:10.1159/000552072 · Cited by 3
A Conversational Agent for Medical Disclosure of Sexually Transmitted Infections
Moreno, J. C., Sánchez-Anguix, V., Alberola, J. M., Julián, V., & Botti, V. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15471-3_37 · Cited by 4
Establishing of big data clinical dataset in brain vessel aneurysm research
Kivelev, J., Saarenpää, I., & Кривошапкин, А. Л. (2023) · Сибирский научный медицинский журнал · doi:10.18699/ssmj20230311 · Cited by 3
Closing the Loop for Patients with Chronic Diseases - from Problems to a Solution Architecture
Färber, A., de Spindler, A., Moser, A., & Schwabe, G. (2023) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00055 · Cited by 7
“The Education Manager can answer patient questions and provide explanations [44].”
BackgroundAgent-based systems in healthcare
Montagna, S., Mariani, S., Schumacher, M. I., & Manzo, G. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108140 · Cited by 3
Towards Building a Chatbot-Based First Aid Service in Arabic Language
Anwar, J., Nadi, P., & Tawfik, N. S. (2024) · Journal of Advanced Research in Applied Sciences and Engineering Technology · doi:10.37934/araset.45.2.110 · Cited by 7
Speybroeck, E. L., Petrenko, C. L. M., Tapparello, C., Griffin, K., Hargrove, E., Himmelreich, M., Lutke, A., Lutke, C. J., May, M., Zhang, S., et al. (2024) · Digital Health · doi:10.1177/20552076241261458 · Cited by 3
“24,36 The health coach interface will resemble using a typical messaging app, which is familiar to most adults with developmental disabilities.”
BackgroundMoving Healthcare AI Support Systems for Visually Detectable Diseases to Constrained Devices
Watt, T., Chrysoulas, C., Barclay, P. J., Boudani, B. E., & Kalliatakis, G. (2024) · Applied Sciences · doi:10.3390/app142411474 · Cited by 3
Shayaninasab, M., Zahoor, M., & Yalçın, Ö. N. (2024) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3652988.3673968 · Cited by 5
“Although these approaches seem distinct, in healthcare, dialogue systems strive to balance these modes to maintain natural conversation flow and support users empathically [15].”
MacNeill, A. L., Luke, A., & Doucet, S. A. (2025) · Digital Health · doi:10.1177/20552076241311066 · Cited by 4
Andargeery, S. Y., & Elrafey, D. S. (2025) · Scientific Reports · doi:10.1038/s41598-024-80982-2 · Cited by 9
Machine Learning in Government Applications: A Review
Sethi, I. P. S., Gupta, O. P., Dwivedi, R. K., Kapoor, M., Bhaisare, S., Chaudhari, V. K., & Varma, S. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.04.369 · Cited by 6
Exploring Young Adults' Attitudes Toward AI-Driven mHealth Apps: Qualitative Study
Aboueldahab, A., Damaschi, G., D’Αddario, M., & Steca, P. (2025) · JMIR Human Factors · doi:10.2196/76075 · Cited by 6
“This perspective aligns with frameworks proposing hybrid human-AI systems [39], and is supported by findings showing concerns about fully automated health solutions in the absence of clear human intervention mechanisms [40].”
Martinengo, L., Ha, N. H. L., Tay, E. Y. J., Tong, S. C., & Sevdalis, N. (2025) · JMIR Research Protocols · doi:10.2196/72942 · Cited by 4
“Recent systematic reviews offer preliminary evidence that AI-based chatbots promote healthy lifestyle change [22-25]; smoking cessation [26,27]; and self-management of chronic disorders [6,21,28], such as hypertension [29].”
Dolman, A. J., Kaliappan, S., Zhou, Y., Palleti, D., Marquard, J. L., Lee, S. I., Karkar, R., & Jimison, H. B. (2025) · Innovation in Aging · doi:10.1093/geroni/igaf106 · Cited by 5
ZzzMate: Designing an Empathetic Chatbot for Addressing Self-Conscious Emotions for Sleep Adherence
Tang, X., Li, Z., Sun, X., Xu, X., & Zhang, M.-L. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2589503 · Cited by 4
“Effective behavioral interventions have been extensively studied across multiple domains, with research demonstrating various successful approaches to improving health behaviors (Blake et al., 2017; Laranjo et al., 2018; Murawski et al., 2018; Shi Jie, 2025; Wang et al., 2019).”
Supporting the Onboarding of 3D Printers through Conversational Agents
Liu, S., Harun, S. E., Jasche, F., & Ludwig, T. G. (2021) · Mensch und Computer (MuC) · doi:10.1145/3473856.3474010 · Cited by 4
“Conversa-tional agents are already being applied in many scenarios including smart living, healthcare, museums, etc. [3, 7, 8].”
BackgroundDesigning Conversational Assistants to Support Older Adults’ Personal Health Record Access
Karimi, P., Ballard, K., Vazirani, P., Jorigay, R. T. N., & Martin-Hammond, A. (2022) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-99194-4_17 · Cited by 3
Human-Therapeutic Chatbot Interaction Analysis, on the Example of Fido
Karkosz, S., Michałowski, J. M., Sanna, K., Szczepaniak, N., & Konat, B. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-06394-7_53 · Cited by 3
How Can I Help You Today? The Rise of Virtual Assistants in Human Resources Management
Vieru, D., Bourdeau, S., Ringeval, M., & Jung, T. (2022) · Advances in information quality and management · doi:10.4018/978-1-7998-9418-6.ch002 · Cited by 3
Vloga umetne inteligence v izobraževanju in za izobraževanje
Pesek, I., & Krašna, M. (2022) · doi:10.18690/um.ff.11.2022.13 · Cited by 15
Empathic Smart Conversational Agent for Enhanced Recovery from Abdominal Surgery at Home
Díaz, F. D., Pedrosa, I., Quirós, P., & Díaz, P. Á. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-19679-9_51 · Cited by 3
Jingar, M. (2023) · Behaviour and Information Technology · doi:10.1080/0144929x.2023.2196583 · Cited by 3
“A future can be foreseen where these new waves of technologies will be omnipresent in our day-to-day life activities, participating as actors that collaborate with humans during their work and leisure tasks (Laranjo et al. 2018) or as an integrated part of our beings (Farooq and Grudin 2016).”
Lessons Learned from Modeling the Interaction with Conversational Agents
Fernandes, U. D. S., Barbosa, G. A. R., Chagas, B. A., Barbosa, G. D. J., Barbosa, S. D. J., & Prates, R. O. (2023) · Interaction design & architecture(s)/ID&A Interaction design & architecture(s) · doi:10.55612/s-5002-055-007 · Cited by 5
Chen, P., Tielman, M. L., Heylen, D., Jonker, C. M., & van Riemsdijk, M. B. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230077 · Cited by 4
“Conversational agents have already been extensively investigated in the context of healthcare [12], for example, to support users in self-care, retrieving information, or nonP.”
BackgroundInternet of Things in Society 5.0 and the Democratization of Healthcare
Demetriou, D., Mathabe, K., Lolas, G., & Dlamini, Z. (2023) · Springer eBooks · doi:10.1007/978-3-031-36461-7_5 · Cited by 4
Toward Connecting Speech Acts and Search Actions in Conversational Search Tasks
Ghosh, S., Ghosh, S., & Shah, C. S. (2023) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1109/jcdl57899.2023.00027 · Cited by 7
“Although conversational search systems are often defined as artificial systems which can interact, understand, and respond in natural language [50, 72], it overlooks the complexity of a human-human conversation.”
BackgroundMeloni, A., Angioni, S., Salatino, A. A., Osborne, F., Birukou, A., Recupero, D. R., & Motta, E. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47243-5_22 · Cited by 6
Kum, J., Jung, S., & Lee, M. (2024) · Electronics · doi:10.3390/electronics13020430 · Cited by 5
Li, S., Wang, Y., Chen, L., Chen, T., Du, J., Su, H., Jiang, H., Wu, Q., Zhang, L., Bao, J., et al. (2024) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x231219039 · Cited by 7
“25 Digital intervention has already brought many new possibilities in providing mental health services 26 – 28 .”
Carolis, B. N. D., Palestra, G., & Oranger, E. (2024) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/3656650.3656765 · Cited by 6
“Chatbots represent another commonly used interface in this field [1, 2, 10, 11, 13].”
MethodLaymouna, M. A., Ma, Y., Lessard, D., Engler, K., Therrien, R., Schuster, T., Vicente, S., Achiche, S., Haj, M. N. E., Lemire, B., et al. (2024) · Healthcare · doi:10.3390/healthcare12161661 · Cited by 9
Benoit, L., Maradji, V., Madelon, C., Rozette, S., Bentivegna, E., Nguyen-Xuan, H.-T., Azaïs, H., Bats, A., Tsopra, R., & Koual, M. (2025) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.70047 · Cited by 3
“14,15 In a study evaluating the use of an application, Econsult, in breast cancer patients, Carré et al. reported that 98% of patients found that the information transmitted by the digital tool complemented that given by the doctor.”
Jimenez, D., Castano-Villegas, N., Llano, I., Martinez, J., Ortiz, L., Velásquez, L., & Zea, J. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00280 · Cited by 3
The Neural Network Conversation Model enables the Commonly Asked Student Query Agents
Muangnak, N., Thasnas, N., Hengsanunkul, T., & Yotapakdee, J. (2020) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2020.0110421 · Cited by 6
“A survey on the recent literature by Laranjo et al. [14] cited 14 conversational agents without constrained NLP qualification from 17 articles.”
BackgroundAI applications in diagnostic technologies and services
Catania, L. J. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-824477-7.00005-5 · Cited by 3
Theano: A Greek-speaking conversational agent for COVID-19
Ventoura, N., Palios, K., Vasilakis, Y., Paraskevopoulos, G., Katsamanis, N., & Katsouros, V. (2021) · Workshop on NLP for Positive Impact (NLP4PI) · doi:10.18653/v1/2021.nlp4posimpact-1.5 · Cited by 7
“By being explicit in every stage of the dialogue, we also ensure that the user acknowledges the capabilities of the CA and does not expect anything more than what is offered about a sensitive medical topic like COVID-19 (Laranjo et al., 2018).”
BackgroundSymbolic and Statistical Learning Approaches to Speech Summarization: A Scoping Review
Rezazadegan, D., Berkovsky, S., Quiroz, J. C., Kocaballi, A. B., Wang, Y., Laranjo, L., & Coiera, E. W. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101305 · Cited by 19
“In healthcare, speech summarization has the potential to create a new generation of digital scribes (systems which generate clinical records from spoken speech) and conversational agents which can interact with patients (Finley et al., 2018; Coiera et al., 2018; Laranjo et al., 2018).”
BackgroundSelf-citationMarcoux, A., Tessier, M.-H., Grondin, F., Reduron, L. R., & Jackson, P. L. (2021) · Santé mentale au Québec · doi:10.7202/1081509ar · Cited by 6
Spitale, M., Catania, F., Cosentino, G., et al. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3308557.3308684 · Cited by 3
“This kind of systems has been recently investigated a lot as didactic supporting tool and in the study of NeuroDevelopmental Disorders (NDD) such as Autism Spectrum Disorder [8], [4].”
BackgroundData Capture and Analyses from Conversational Devices in the Homes of the Elderly
Purao, S., & Meng, C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-34146-6_14 · Cited by 3
Metadata
- Authors
- Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., & Coiera, E.
- Affiliations
- Macquarie University (AU)
- Corresponding author
- Laranjo, Liliana
- ORCID
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-07-11
- Volume
- 25
- Issue
- 9
- Pages
- 1248-1258
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC6118869
- OpenAlex
- W2843010082
- Cited by
- 1,388
- References
- 60
- Citations by year
- 2026: 291 · 2025: 246 · 2024: 182 · 2023: 182 · 2022: 151 · 2021: 143 · 2020: 124 · 2019: 62 · 2018: 6
- FWCI
- 1.87
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
- Funders
- National Health and Medical Research Council (APP1134919)
- Centre for Research Excellence in Digital Health (APP1054146)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- chatbots, healthcare chatbots, natural language input, dialogue management, self-care support, depression symptom reduction, patient safety, evaluation metrics
- MeSH terms
- Artificial Intelligence, Communication, Delivery of Health Care, Natural Language Processing, Speech Recognition Software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.