Conversational agents in healthcare: a systematic review
Open accessThe DOI matches its Crossref record; 55 of the 60 references it cites were found in a registry or an index; 14 of the 36 sources its claims cite were checked for support.
Abstract
Objective: Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes. Methods: We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were included if they focused on consumers or healthcare professionals; involved a conversational agent using any unconstrained natural language input; and reported evaluation measures resulting from user interaction with the system. Studies were screened by independent reviewers and Cohen's kappa measured inter-coder agreement. Results: The database search retrieved 1513 citations; 17 articles (14 different conversational agents) met the inclusion criteria. Dialogue management strategies were mostly finite-state and frame-based (6 and 7 conversational agents, respectively); agent-based strategies were present in one type of system. Two studies were randomized controlled trials (RCTs), 1 was cross-sectional, and the remaining were quasi-experimental. Half of the conversational agents supported consumers with health tasks such as self-care. The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04). Patient safety was rarely evaluated in the included studies. Conclusions: The use of conversational agents with unconstrained natural language input capabilities for health-related purposes is an emerging field of research, where the few published studies were mainly quasi-experimental, and rarely evaluated efficacy or safety. Future studies would benefit from more robust experimental designs and standardized reporting. Protocol Registration: The protocol for this systematic review is registered at PROSPERO with the number CRD42017065917.
Overview
CoChat Summary
TL;DR
This systematic review found that conversational agents with unconstrained natural language input were still rarely studied in healthcare: 17 articles covering 14 agents, mostly evaluated with quasi-experimental designs. Evidence of efficacy and safety was limited, with only one RCT showing reduced depression symptoms (effect size d = 0.44, p = .04).
Key findings
- The search retrieved 1513 citations; 17 articles evaluating 14 different conversational agents met inclusion criteria.
- Study designs were mainly weak for efficacy claims: 2 RCTs, 1 cross-sectional study, and the rest quasi-experimental. The paper notes that the RCTs had moderate to high risk of bias.
- Dialogue management was mostly finite-state (6 agents / 8 studies) or frame-based (7 agents / 8 studies); agent-based dialogue management appeared in 1 agent.
- The only RCT evaluating efficacy found a significant reduction in depression symptoms with effect size d = 0.44, p = .04; patient safety was rarely evaluated.
Methods
- Systematic review of studies of conversational agents using any unconstrained natural language input for health-related purposes; protocol registered at PROSPERO CRD42017065917 and the review followed PRISMA.
- Searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital Library in April 2017, updated in February 2018; included primary studies of consumers, caregivers, or healthcare professionals that tested the full system with human users and reported evaluation measures.
- Screening was done by independent reviewers with Cohen’s kappa for agreement; because of heterogeneity, the authors conducted a narrative synthesis rather than a meta-analysis.
Limitations
- The paper states it focused only on agents with unconstrained natural language input, excluding constrained-input systems that are also relevant to healthcare.
- The paper states that the small number of included studies, heterogeneity of agents, and predominance of quasi-experimental pilot studies limited its ability to conduct a meta-analysis.
- Evident from the text: reporting was inconsistent, making eligibility assessment and synthesis difficult, and several studies lacked clear funding/conflict-of-interest reporting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
10 Valid · 3 Not valid · 13 Not checked · 25 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“A crossover RCT evaluated the diagnostic performance for depression of an embodied conversational agent compared to a psychiatrist, finding a sensitivity of 49% and specificity of 93%.”
§ Conversational agents in healthcare: a systematic review > Results > Evaluation measures
What the sources say
- Supportedcites [22] Philip, 2017, Virtual human as a new diagnostic tool, a proof of concept study in the field of major depressive disorders, Sci Rep source
The Method (Recorded data in the study; page not given) says patients completed both interviews in randomized order, one with the ECA and one with the psychiatrist. The Results (Global performance of ECA diagnosis of MDD; page not given) report overall sensitivity of 49% and specificity of 93% against the psychiatrist’s diagnosis. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“We found that only one study evaluated the efficacy of a conversational agent on health outcomes using adequate study design methods (RCT) and validated questionnaires.”
§ Conversational agents in healthcare: a systematic review > DISCUSSION > Standards for reporting the evaluation of conversational agents in healthcare
What the sources say
- Supportedcites [19] Fitzpatrick, 2017, Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial, JMIR Ment Health source
The Results report that participants were randomized to Woebot or an information-control group, and the Methods > Measures section identifies the PHQ-9 and GAD-7 as validated questionnaires. This confirms that the cited paper is an RCT evaluating a conversational agent with validated health-outcome measures. “Only one study” describes the citing authors’ review finding, not a claim made by this source. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Over the last two decades, a solid body of evidence has shown the potential benefits of using embodied conversational agents for health-related purposes.”
§ Conversational agents in healthcare: a systematic review > Introduction
What the sources say
- Not checked — no full text of the sourcecites [3] Bickmore, 2013, A randomized controlled trial of an automated exercise coach for older adults, J Am Geriatr Soc source
- Partly supportedcites [4] Bickmore, 2013, Automated interventions for multiple health behaviors using conversational agents, Patient Educ Couns sourceExperimental — accuracy not yet measured
The Results and Discussion report benefits from this conversational-agent intervention, particularly increased fruit and vegetable consumption compared with the control group. The Conclusion also describes the intervention as effective at changing health behavior. The paper does not establish the claim’s broader characterization of a solid body of evidence accumulated over two decades. (LLM, conf=0.86)
- Partly supportedcites [5] Watson, 2012, An internet-based virtual coach to promote physical activity adherence in overweight adults: randomized controlled trial, J Med Internet Res sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) describes prior research on embodied computer agents, and the Discussion conclusion reports that a virtual coach helped maintain activity in overweight participants. This supports potential health benefits, but the paper does not establish a solid body of evidence spanning two decades or benefits across health-related purposes generally. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [6] Edwards, 2013, Use of an interactive computer agent to support breastfeeding, Matern Child Health J source
boostedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04).”
§ Conversational agents in healthcare: a systematic review > Abstract > Results
What the sources say
- Partly supportedcites [19] Fitzpatrick, 2017, Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial, JMIR Ment Health sourceExperimental — accuracy not yet measured
The Results (Preliminary Efficacy) report a significant reduction in depression symptoms for the Woebot group relative to the information control group, with d=0.44 and a Bonferroni-corrected P=.04. The Discussion (Principal Results) calls this, to the authors’ knowledge, the first randomized trial of a nonembodied text-based conversational agent designed for therapeutic use; it does not establish that it was the only RCT evaluating any conversational agent. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Several randomized controlled trials of interventions involving embodied conversational agents have shown significant improvements in physical activity, fruit and vegetable consumption, and accessibility to online health information, among other outcomes.”
§ Conversational agents in healthcare: a systematic review > Introduction
What the sources say
- Not checked — no full text of the sourcecites [3] Bickmore, 2013, A randomized controlled trial of an automated exercise coach for older adults, J Am Geriatr Soc source
- Partly supportedcites [4] Bickmore, 2013, Automated interventions for multiple health behaviors using conversational agents, Patient Educ Couns sourceExperimental — accuracy not yet measured
The Abstract (Methods) describes one randomized trial, not several. The Results (§3.1–3.3) report a significant improvement in fruit and vegetable consumption and faster growth in walking steps for the physical-activity group, but no significant difference in self-reported physical activity. Our copy does not report an outcome concerning accessibility to online health information. (LLM, conf=0.90)
- Partly supportedcites [5] Watson, 2012, An internet-based virtual coach to promote physical activity adherence in overweight adults: randomized controlled trial, J Med Internet Res sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Not checked — no full text of the sourcecites [6] Edwards, 2013, Use of an interactive computer agent to support breastfeeding, Matern Child Health J source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
17 more claims · 2 with their cited sources checked
Create a free account to see all 51 claims and every source verdict.
- Probably not a substantial claim. 12
- It may state two separate claims as one. 20
- Borderline: one of the tests for a substantial claim is unclear. 1
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Previously conducted reviews [5,18,19] did not assess the effectiveness and safety of chatbots in mental health.”
From Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis (2020) · cites this paper as [18]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most studies reviewed in 2018 [14] were limited to task-oriented conversational agents to support patients and clinicians in highly specific scenarios (e.g., information retrieval and predefined clinical interview), restricting user input to predetermined utterances.”
From Conversational Affective Social Robots for Ageing and Dementia Support (2022) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the same note, conversational agents have been proven a valuable tool to deliver digital health interventions [7-9].”
From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These databases were chosen as they cover relevant aspects in the fields of medicine, technology, and interdisciplinary research and have also been used in other systematic reviews covering similar topics [7,8].”
From Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review (2021) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing reviews on conversational agents focus on a certain subtype of agents such as virtual coaches [31-33] or embodied conversational agents (ECAs) [34] or on specific functionalities of these agents such as behavior change [35] or mental health applications [36,37].”
From Conversational Agents in Health Care: Scoping Review and Conceptual Analysis (2020) · cites this paper as [32]
Experimental — accuracy not yet measured
62 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 1×
“This systematic review is compliant with the PRISMA statement.(18)”
MethodCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · cited 1×
“49,50 Implications for healthcare, public health, and future research Patient safety was rarely evaluated in the included studies.”
BackgroundEvaluating the quality of medical care
DONABEDIAN, A. (2005) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2005.00397.x · Cited by 6,102 · cited 2×
“…of a conversational system from a health research perspective was considered to involve any health-related results present in the included studies, including process and outcome measures as defined by Donabedian; 16 for example, effectiveness in symptom reduction, diagnostic accuracy, or referrals.”
BackgroundELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“The first well-established chatbot of this kind—ELIZA—was programmed in 1966 to simulate a text-based conversation with a psychotherapist.(2)”
MethodA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“10,56,57 Therefore, the use of conversational agents with unconstrained natural language input capabilities and other artificial intelligence applications in healthcare needs to be carefully monitored.”
BackgroundSTARD 2015: an updated list of essential items for reporting diagnostic accuracy studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · BMJ · doi:10.1136/bmj.h5527 · Cited by 3,660 · cited 1×
“…telehealth (CONSORT-EHEALTH), 46 the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) statement, 47 and the Standards for Reporting Diagnostic accuracy studies (STARD), 48 among others from the Enhancing the Quality and Transparency Of health Research (EQUATOR) network.”
MethodDes Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817
ELIZA — a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,738
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 7×
“The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms.(19) Two studies comparing diagnostic performance of conversational agents and clinicians found acceptable sensitivity and specificity.”
Influential
Research works printed but not linked: 2
- Radziwill, 2017, arXiv Prepr, 1704
- Coiera, 2017, First compute no harm, BMJ Opin
Cited by
Items 801–900 of 1,371
Vik: A Chatbot to Support Patients with Chronic Diseases
Chaix, B. (2020) · Biomedical Journal of Scientific & Technical Research · doi:10.26717/bjstr.2020.27.004453 · Cited by 3
IMPACT OF AI IN INTERNET OF MEDICAL THINGS FOR HEALTH CARE DELIVERY
Nathani, N., & Hasan, Z. (2021) · International Journal of Engineering Technologies and Management Research · doi:10.29121/ijetmr.v8.i8.2021.1012 · Cited by 3
Agarwal, S., Alhuwail, D., & John, O. (2021) · Springer eBooks · doi:10.1007/978-3-030-45009-0_73 · Cited by 4
Multiuser Privacy and Security Conflicts in the Cloud
Alhelali, E., Ramokapane, K. M., & Such, J. M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581307 · Cited by 6
“Prior eforts in other domains have focused on using conversational agents [12, 17, 33] and explainable and value-based AI methods [41, 42].”
Practical Applications of the Internet of Things in Radiation Oncology
Thaker, N. G., De, B. S., Shah, C., Manda, S., Royce, T. J., & Beriwal, S. (2022) · Applied Radiation Oncology · doi:10.37549/aro1322 · Cited by 4
On the Generation of Medical Dialogues for COVID19Preprint
Yang, W., Zeng, G., Tan, B., Ju, Z., Chakravorty, S., He, X., Chen, S., Yang, X., Wu, Q., Zhou, Y., et al. (2020) · doi:10.36227/techrxiv.12290345.v1 · Cited by 3
He, Y., Yang, L., Zhu, X., Wu, B., Zhang, S., Qian, C., & Tian, T. (2022) · doi:10.2196/preprints.40719 · Cited by 3
Assistive Chatbots for healthcare: a succinct reviewPreprint
Bhattacharya, B. S. (2023) · doi:10.36227/techrxiv.23741469 · Cited by 5
Rose, C. J., Ringsten, M., Bidonde, J., Glanville, J., Berg, R. C., Cooper, C., Muller, A. E., Bergsund, H. B., Meneses-Echavez, J. F., & Potrebny, T. (2023) · Research Square · doi:10.21203/rs.3.rs-3288515/v1 · Cited by 3
From Non-linear Optics to Next-generation Imaging: A Journey through Non-linear Imaging TechniquesPreprint
Rezafirouz, A. (2023) · doi:10.1364/opticaopen.24427420.v1 · Cited by 3
Memory Fabric for Conversational AI Agents: Enabling Shared and Persistent Memory Across Users
Tiwari, A., & Gupta, V. (2025) · TechRxiv · doi:10.36227/techrxiv.176523350.08289935/v1 · Cited by 2
Digital Therapeutics for Healthy Longevity: A Roadmap
Leistner, T., & Kowatsch, T. (2023) · International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC) · doi:10.5220/0011778500003414 · Cited by 2
“Barriers of implementing CAs can be liability issues (Schlieter et al., 2022), for example, patient safety was often overlooked in prior work (Laranjo et al., 2018).”
BackgroundIslam, A., Chaudry, B., & Islam, A. (2024) · mHealth · doi:10.21037/mhealth-23-46 · Cited by 3
Deploying AI-Driven Natural Language Processing Systems for Public Health Crisis Management
Dempere, J. M. (2024) · Springer eBooks · doi:10.1007/978-3-031-57773-4_10 · Cited by 4
Sağlık Hizmetlerinde Hasta Güvenliği ve Yapay Zekâ: Bibliyometrik Bir Analiz
ŞAHİN, D. (2025) · Elektronik Sosyal Bilimler Dergisi · doi:10.17755/esosder.1563082 · Cited by 3
Karimi, P., & Martin-Hammond, A. M. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719773 · Cited by 5
“Building on this mixed-initiative approach, researchers have explored IAs with visual interfaces, incorporating features like voice transcriptions and chatbots [21, 27] that allow users to leverage automation while having control over verifying information sources.”
Reference decisions enhance LLM performance, amplified by source disclosure
Zhang, Y., Liu, Z., Bai, S., Xu, T., & Lau, R. Y. K. (2025) · Digital Health · doi:10.1177/20552076251342078 · Cited by 3
“For LLM1, the ANOVA analysis includes results from experiments 1, 11, and 12.”
Chandler, R. D., Warner, S. C., Guillaume, D., & Wells, J. (2025) · Nursing Outlook · doi:10.1016/j.outlook.2025.102468 · Cited by 3
Job Interview Training with RAG-LLM: An Experimental Study with the Furhat Robot
Buchem, I., Bedemann, R., & Gers, F. A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98417-4_7 · Cited by 2
Collins, C., Fackler, J. C., Sacco, M. J., & Jacobs, M. L. (2025) · Frontiers in Pediatrics · doi:10.3389/fped.2025.1646989 · Cited by 3
Artificial Intelligence-Powered Internet of Medical Things in Radiation Oncology
Thaker, N. G., Penberthy, D. R., Beriwal, S., Royce, T. J., Showalter, T. N., Redjal, N., & Doria, C. (2025) · AI in Precision Oncology · doi:10.1177/2993091x251385557 · Cited by 4
Karimi, P., & Martin-Hammond, A. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757465 · Cited by 5
“…of Intelligent Voice Assistants for Supporting Older Adults’ Health Studies suggest that commercial IVAs such as Alexa and Google Home have the potential to support individuals with health-related information tasks in clinical (e.g., hospital) and non-clinical (e.g., home, community) settings [39].”
Liu, Y., Song, C., Huang, D., Liu, X., Li, Z., & Hu, B. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2598675 · Cited by 2
“As previous systematic reviews of AI and health care have summarized, the application of chatbots predominates in the mental health field (Laranjo et al., 2018).”
Optimizing RAG-based LLMs for healthcare question answering tasks
Bedwa, M., Hooda, N., & Kumar, V. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02638-5 · Cited by 6
“…deep learning architectures such as convolutional neural networks (CNNs) initially processed structured clinical text, while autoencoders helped extract meaningful representations from medical dialogues underscoring neural network’s versatility in conversational healthcare applications [1].”
User experience and safety of generative AI-based mental health chatbots: Scoping review protocol
Olisaeloka, L., Richardson, C., & Vigo, D. (2026) · PLoS ONE · doi:10.1371/journal.pone.0341631 · Cited by 6
“The safety of traditional mental health chatbots previously received limited attention, mainly because the rule-and retrieval-based systems were perceived as low-risk [35].”
Calabrese, C., Xue, H., Zhang, X., & Oh, Y. J. (2026) · Science Communication · doi:10.1177/10755470261418533 · Cited by 3
“They can be incorporated into web or mobile applications, providing round-the-clock communication with users (Laranjo et al., 2018; Zhang et al., 2020).”
Su, J., Lang, Y., White, A., Chen, K., Lara, C., Zhou, Y., & Xiao, Y. (2026) · Patient Education and Counseling · doi:10.1016/j.pec.2026.109547 · Cited by 4
Friend or Therapist? A Systematic Literature Review on Chatbots and Mental Health in Young People
Hashmati, N., Skoog, T., Obaid, M., & Eriksson, T. (2026) · doi:10.1145/3780045.3780052 · Cited by 3
Embedding CAs in Organizational Ecosystems: Accomplishments and Future Research
Liu, D., Yoon, V., & Sen, A. (2026) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2026.3675417 · Cited by 2
“Noticeably,earlyCAswere preprogrammed or rule-based, relying on fixed intent-response mappings and domain-specific scripts that constrained what the agent could say or do [19].”
Public acceptance of LLM-driven healthcare chatbots in China: An empirical study
Qian, Y., Fu, Y., Chen, Y., Ke, L., & Zhao, X. (2026) · Digital Health · doi:10.1177/20552076261437614 · Cited by 3
Are AI Companions Real Companions? A BERT ‐Based Study of Replika Reviews
Gautam, C., & Kumar, P. (2026) · Journal of Consumer Behaviour · doi:10.1002/cb.70161 · Cited by 5
Unlocking AI Chatbot Potential in Healthcare: Trust-Enhanced DeLone & McLean IS Success Model
Sarhan, M. Y., Alarify, M., & Khojah, M. (2026) · Healthcare · doi:10.3390/healthcare14101324 · Cited by 2
“…on users’ behavioural intentions and acceptance studies [35], we nonetheless produce little, if at all, clear understanding about how core information system attributes contribute to user satisfaction, usage, and overall system success when engaging with AI-powered healthcare services [14,48–51].”
Huang, T., Li, S., Wang, Y., & Liu, W. (2026) · Journal of Medical Internet Research · doi:10.2196/88697 · Cited by 4
“Consistent with prior definitions of conversational agents in digital health, they are distinguished from nonconversational digital interventions by their capacity for sustained, multi-turn dialogue, which forms the core therapeutic mechanism [14].”
Enabling Automated, Conversational Health Coaching with Human-Centered Artificial Intelligence
Mitchell, E. G. (2021) · doi:10.7916/d8-drwc-b875 · Cited by 3
Intelligent Conversational Agents in Global Health
Luxton, D. D., & Sirotin, A. P. (2021) · Springer eBooks · doi:10.1007/978-3-030-57296-9_11 · Cited by 3
Sah, Y. J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-85648-5.00013-x · Cited by 6
Hayakawa, M., Watanabe, O., Shiga, K., Fujishita, M., Yamaki, C., Ogo, Y., Takahashi, T., Ikeguchi, Y., & Takayama, T. (2022) · Patient Education and Counseling · doi:10.1016/j.pec.2022.10.004 · Cited by 2
An application of conversational systems to promote healthy lifestyle habits
Griol, D., Callejas, Z., Fernández-Martínez, F., & Esposito, A. A. (2022) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc/picom/cbdcom/cy55231.2022.9927835 · Cited by 4
“healthcare institutions and organizations [12]–[14].”
Derigny, T., Schnitzler, C., Vors, O., Huchez, A., Gerard, T., Mallet, S., Gandrieau, J., & Potdevin, F. (2023) · European Physical Education Review · doi:10.1177/1356336x231160404 · Cited by 3
“These difficulties demonstrated that, just like use of artificial chatbots in customer communications (Gaffney et al., 2019; Laranjo et al., 2018; Zhang et al., 2020), online PM methods mitigated losses in PA levels, but could not replace the value and motivation provided by the face-to-face interactions of school-bas…”
BackgroundVoelskow, V., Meßner, C., Kurth, T., Busam, A., Glatz, T., & Ebert, N. (2023) · Digital Health · doi:10.1177/20552076231180677 · Cited by 2
“In particular, the user perspective and acceptance are yet to be comprehensively investigated, as only a few studies currently include these aspects.(9,10) Against this background, we investigated the use of a hotline assistant (coronavirus voicebot or CovBot) using artificial intelligence (AI)(11) to assist local hea…”
BackgroundPrasad, M. L., Srinivasulu, C., Meenakshi, M. R., & Rohitha, P. (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.55295 · Cited by 2
Intelligent Information Systems in Healthcare Sector: Review Study
Akila, A., Elhoseny, M., & Nour, M. A. (2024) · Engineering cyber-physical systems and critical infrastructures · doi:10.1007/978-3-031-52787-6_11 · Cited by 2
Burns, J. L., Gichoya, J. W., Kohli, M. D., Jones, J., & Purkayastha, S. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000297 · Cited by 3
Depiction of conversational agents as health professionals: a scoping review
MacNeill, A. L., MacNeill, L., Yi, S., Goudreau, A., Luke, A., & Doucet, S. A. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00029 · Cited by 3
Ruh Sağlığı Alanında Yapay Zeka Araştırmaları: Bibliyometrik Bir Değerlendirme
Uslu, E., & Atılgan, E. (2024) · OSMANGAZİ JOURNAL OF MEDICINE · doi:10.20515/otd.1435157 · Cited by 3
Giri, U., Sharma, A., Oza, D. S., Singh, P., Angra, P. K., & Khanna, A. (2024) · International Conference on Emerging Innovations and Advanced Computing (INNOCOMP) · doi:10.1109/innocomp63224.2024.00028 · Cited by 3
Agent-based Mediation on Smartphone Usage among Co-located Couples
Khatra, K., Sin, J., Kuzminykh, A., & Hasan, K. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3676487 · Cited by 3
“Moreover, researchers explored leveraging virtual agents in the medical field, where the agents have been designed to provide advise regarding medical conditions to patients [2, 4, 34, 37, 52].”
Human vs Chatbot: The Role of AI in Healthcare Marketing
Arora, T. (2024) · International Journal of Social Science Research and Review · doi:10.47814/ijssrr.v7i11.2403 · Cited by 4
Reddy, N. S., Nayyar, V., & Khanna, P. (2024) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83608-708-320241015 · Cited by 2
Designing a FAQ Chatbot to Enhance Faculty Support
Thorat, S., Zheng, Y., Varghese, V. J., & Volkova, A. (2024) · Conference on Information Technology Education · doi:10.1145/3686852.3686886 · Cited by 2
“These conversational systems have been successfully developed and applied to multiple areas and applications, such as healthcare [5], social media [13] and recommender systems [4, 14].”
Review on Artificial Intelligence in Medicine and Health
Almaeeni, H. (2024) · IEEE International Conference on Medical Artificial Intelligence (MedAI) · doi:10.1109/medai62885.2024.00015 · Cited by 3
“As noted by Laranjo et al. [31], the underlying system can be based on a variety of foundations, ranging from a set of simple rule-based responses keyword matching to sophisticated NLP techniques.”
AIMT Agent: An Artificial Intelligence-Based Academic Support System
Lytridis, C., & Tsinakos, A. A. (2025) · Information · doi:10.3390/info16040275 · Cited by 6
Sharpe, B. T., King, R., Banerjee, M., Keates, S., Tyndall, I. T., Kooner-Evans, P., Sivyer, L. R., Cotten, E. L., Obine, E. A. C., Tabet, N., et al. (2025) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2025.2511996 · Cited by 5
“More recently, advancements include the development of ai-powered chatbots tailored for dementia care. these chatbots offer companionship, cognitive stimulation, and can even assist in the early detection of cognitive decline [42].”
Corrò, C., & Chittaro, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-94959-3_5 · Cited by 3
Carlo, L., Cruz, E., Arias, A. C., Baca, M., Carlo, M. P. J., Carpio-Arias, T. V., Waters, W. F., Mackintosh, N. J., & Verdezoto, N. X. (2025) · Digital Health · doi:10.1177/20552076251343684 · Cited by 6
“180 However, more research is needed to design chat-bots that are safe for patients 181 by leveraging existing design guidelines 182 (e.g. clarity around capabilities, sustaining conversations, handling dialog failures) and accounting for the socio-technical nuances in the design of human – chatbot…”
Virtual Health Assistants in Preventive Cancer Care Communication: Systematic Review
Raisa, A., Chen, X., Bryan, E. G., Bylund, C. L., Alpert, J. M., Lok, B. C. L., Fisher, C. L., Thomas, L., & Krieger, J. L. R. (2025) · JMIR Cancer · doi:10.2196/73616 · Cited by 3
“Synonyms for these concepts were identified by reviewing highly cited literature, examining search terms from prior relevant reviews, considering database suggestions, and leveraging the authors’ and colleagues’ field expertise [27,33,34].”
Educational Chatbots: A Systematic Mapping Study
Urquiza-Yllescas, J. F., Mendoza, S., Sánchez-Adame, L. M., Rodríguez, J., & Decouchant, D. (2025) · Journal of Intelligent & Fuzzy Systems · doi:10.1177/18758967251356870 · Cited by 7
“Laranjo et al. (2018) focus on healthcare, analyzing conversational agents’ characteristics, applications, and evaluation measures.”
Hu, J., Yao, X., Huang, Z., Liang, D., Yang, D., Liu, D., Li, J., Zhang, Y., & Ma, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3792787 · Cited by 3
“…agents (CAs), also known as chatbots, personal assistants, or digital compan-ions, in diverse application domains to provide interactive services through text-based or speech-based dialogues, including, but not limited to, health care [85, 167], education [156], and customer service [14, 103].”
Lucchini, A., Bozzon, A., & Colombo, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790698 · Cited by 2
“Without such analysis, these systems risk reinforcing biases and marginalizing people impacted by chronic illness [74, 77].”
Gao, B., Zeng, Z., Yu, Y., Werry, I. P., Chan, C. L., Chen, M., Zhang, H., Huang, B., Ji, J., Leung, C., et al. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791422 · Cited by 3
“However, many systems still rely on template or rule-based, struggle to accommodate cultural diversity and individual preferences [28, 88], and lack the flexibility and dynamic personalization needed for sustained engagement [70, 141].”
Information Usefulness as a Strategy for Action Selection in Health Dialogues
Teixeira, M. S., Pereira, C. D. C., & Dragoni, M. (2020) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wiiat50758.2020.00046 · Cited by 2
“Despite significant contributions that conversational agents (CA) can offer to the health domain [7], [12], [15], [21], these agents are still in early stages and lag behind other domains, raising some concerns that still need to be addressed [9].”
BackgroundPark, D., Choo, M., Kim, J.-W., Lee, J., & Shin, Y. (2022) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3528575.3551452 · Cited by 3
“Despite these advantages, few cases have been applying VUIs to children’s healthcare services or medical devices so far [3].”
BackgroundOn the diminishing return of labeling clinical reports
Lamare, J.-B., Olatunji, T., & Yao, L. (2020) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2020.clinicalnlp-1.31 · Cited by 6
“2016), conversational agent (Laranjo et al., 2018), or disease extraction and classification (Chen et al.”
BackgroundTake Back Control: User Privacy and Transparency Concerns in Personalized Conversational Agents
Hendrickx, I., van Waterschoot, J., Khan, A., ten Bosch, L., Catia Cucchiarini, A., & Strik, H. (2021) · University of Twente Research Information · Cited by 5
Effect of Hidden Vector on the Speech of PRVA
Matsui, T., Tani, I., Sasai, K., & Gunji, Y. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.627148 · Cited by 3
“Some review studies surveyed suggested design methods for these kinds of agents (Laranjo et al., 2018; Montenegro et al., 2019; Rheu et al., 2021).”
BackgroundSalman, S., Richards, D., & Caldwell, P. H. Y. (2021) · Italian Journal of Computational Linguistics · doi:10.4000/ijcol.862 · Cited by 5
User-Adapted Semantic Description Generation Using Natural Language Models
Salcedo, J. S., Galván, L. M., Castillo, J. C., Castro‐González, Á., & Salichs, M. A. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-22356-3_13 · Cited by 2
Artificial Intelligence as an Emerging Tool for Cardiologists
Ledziński, Ł., & Grześk, G. (2023) · International Electronic Conference on Biomedicines (IECBM) · doi:10.3390/ecb2023-14339 · Cited by 6
“Another aspect of NLP is the use of speech recognition and question answering to develop chatbots for patients [11].”
BackgroundUser Evaluation of Conversational Agents for Aerospace Domain
Liu, Y., Arnold, A., Dupont, G., Kobus, C., Lancelot, F., Granger, G., Rouillard, Y., Duchevet, A., Imbert, J.-P., & Matton, N. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2239544 · Cited by 2
“…importance of agent performance quality, such as knowledge level and task completion, for building trust in these artificial intelligence devices (Rheu et al., 2021), the efficacy and health outcomes of using these systems in the healthcare domain have rarely been evaluated (Laranjo et al., 2018).”
Bridging the ICT Revolution and Communication Inequality: Lessons for Cancer Survivors
Jung, M. (2023) · Asian Pacific Journal of Cancer Prevention · doi:10.31557/apjcp.2023.24.9.2923 · Cited by 2
“AI chatbots can effectively address the communication needs of cancer survivors by providing reliable health information around the clock (Laranjo et al., 2018).”
BackgroundMartyn, N., Palumbo, O., Zhang, J., Morris, A., & Zaccolo, S. (2023) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct60411.2023.00071 · Cited by 3
“There are several AI parenting solutions ranging from smartphone applications to smartphone applications paired with peripheral devices and full-fledged conversational anthropomorphize mechanical robots [6].”
BackgroundDevelopment of an Intelligent Chatbot for an Intraosseous Access Serious Game (WIP Paper)
Stancati, F., Riitano, A., Kapralos, B., Button, D., Dubrowski, A., & Lamberti, F. (2023) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa59645.2023.10345933 · Cited by 3
“In recent years, chatbots have been introduced into the healthcare sector to address current challenges, including the shortage of healthcare providers, which reduce the availability and accessibility of healthcare services [9]–[11].”
BackgroundMachine Learning and Artificial Intelligence in Evidence Generation and Evidence Synthesis
Malik, V. S., & Singh, M. (2024) · Journal of Medical Evidence · doi:10.4103/jme.jme_95_23 · Cited by 4
Keller, R., Yao, J., Teepe, G. W., Hartmann, S., Lohse, K.-M., von Wangenheim, F., Müller‐Riemenschneider, F., Mair, J. L., & Kowatsch, T. (2021) · International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC) · doi:10.5220/0010412708110816 · Cited by 4
“…finding corroborates recent research findings of literature reviews investigating the state of research on CAs in healthcare which is still considered to be in an early developing stage (Bérubé et al., 2020; Car et al., 2020; Laranjo et al., 2018; Montenegro et al., 2019; Schachner et al., 2020).”
BackgroundResultImplementation of Covid Chatbot
Nadiger, D. K. S., Aravinda, K., Sai, H. P., Deepthi, M., Nagaraj, N., & Priya Das, P. (2022) · International Conference on Computer Communication and Informatics (ICCCI) · doi:10.1109/iccci54379.2022.9740968 · Cited by 2
Alexa Depression and Anxiety Self-tests: A Preliminary Analysis of User Experience and TrustPreprint
Quiroz, J. C., Bongolan, T., & Ijaz, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03892 · Cited by 2
“The proliferation of conversational agents has made them attractive for health applications [11] and mental health [18].”
BackgroundSrivastava, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05561 · Cited by 2
Alzughaibi, S., Luo, J., Nazareth, D. L., Wu, M., & Fink, J. T. (2023) · Research Square · doi:10.21203/rs.3.rs-2769833/v1 · Cited by 5
Assistive Chatbots for healthcare: a succinct reviewPreprint
Bhattacharya, B. S. (2023) · doi:10.36227/techrxiv.23741469.v1 · Cited by 4
Enhancing AI Chatbot Responses in Healthcare: The SMART Prompt Structure in Head and Neck SurgeryPreprint
Vaira, L. A., Lechien, J. R., Abbate, V., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4953716/v1 · Cited by 2
Hang, Y., Wu, W., Feng, Y., Yan, K., Liu, Y., Xiao, X., & Qiao, Z. (2025) · doi:10.2196/preprints.72249 · Cited by 2
Purrfessor: A Fine-tuned Multimodal LLaVA Diet Health Chatbot (Preprint)Preprint
Lu, L., Deng, Y., Tian, C., Yang, S., & Shah, D. V. (2025) · doi:10.2196/preprints.74111 · Cited by 2
Anton, E., Oesterreich, T. D., Fitte, C., & Teuteberg, F. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.481 · Cited by 2
Designing rule-based conversational agents with behavioral programming: a study of human subjects
Rosenfeld, A., & Haimovich, N. (2022) · EuroMed Journal of Business · doi:10.1108/emjb-09-2021-0144 · Cited by 1
Intelligent Agents and Dialog Systems
Bickmore, T., & Wallace, B. (2022) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-09108-7_9 · Cited by 2
Langer, H., & Merkel, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35921-7_13 · Cited by 2
Raisa, A., Fisher, C. L., Silmi, K. P., Alpert, J. M., Bylund, C. L., Lok, B. C. L., & Krieger, J. L. R. (2024) · mHealth · doi:10.21037/mhealth-24-31 · Cited by 2
“VHAs have shown promising benefits in health care over the last two decades, having significantly improved cancer literacy, intention to screen, adoption of preventive behaviors such as physical activity, and fruit and vegetable consumption (4-8).”
Merkel, S., & Schorr, S. (2024) · JMIR AI · doi:10.2196/55673 · Cited by 2
“This finding is in line with a systematic review of CA in health care carried out by Laranjo et al [1].”
InfluentialSemantic Web Framework for Automated Smart Assistants in Public Health
Gangavane, H., Abirami, R., Hassan, M. M., Srinivas, T., Keerthana, S., & Kaliappan, S. (2025) · International Conference on Intelligent Control, Computing and Communications (IC3) · doi:10.1109/ic363308.2025.10956245 · Cited by 1
“The component's as it were client-side nature altogether diminishes overhead, and so, support costs, related with utilizing the savvy right hand [26].”
Enhancing Patient Intake Process in Mental Health Consultations Using RAG-Driven Chatbot
Shayaninasab, M., Zahoor, M., & Yalçın, Ö. N. (2024) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw63320.2024.00053 · Cited by 5
“Their increasingly improved capabilities have resulted in gaining popularity in healthcare use-cases as well, especially with a mental health focus, such as in post-treatment communications, facilitating support groups, offering counseling services, and assisting with administrative tasks [8], [13].”
Leveraging artificial intelligence to strengthen surgical systems in sub-Saharan Africa
Ralph-Okhiria, O. H., & Alonge, I. (2025) · Academia Medicine · doi:10.20935/acadmed7735 · Cited by 3
Chung, J., Ko, J., Choi, Y. K., Lazar, A., Fessele, K. L., & Ameringer, S. (2025) · Research in Gerontological Nursing · doi:10.3928/19404921-20250516-01 · Cited by 2
Qiu, S., Gadiraju, U., & Zheng, X. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729201 · Cited by 1
Uncertainty Detection: A Multi-View Decision Boundary Approach Against Healthcare Unknown Intents
Zhang, Y., & Lau, R. Y. K. (2025) · Applied Sciences · doi:10.3390/app15137114 · Cited by 3
Artificial Intelligence-generated Psychological Counseling for Cancer Patients
Mahendiran, N., Sheik Mohamed, M. I., & Saravanan, S. (2025) · Current Psychiatry Research and Reviews · doi:10.2174/0126660822338819250214064950 · Cited by 1
Igarashi, T., Iijima, K., Nitta, K., & Chen, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3587009 · Cited by 1
Lukas, C., Tuncer, Z., & Breitschwerdt, R. (2025) · i-com · doi:10.1515/icom-2025-0001 · Cited by 3
“24,27 Aguilera highlighted the significant growth in the use of digital technology in health applications, driven in part by the limitations of traditional one-on-one therapy in meeting the mental health needs of the population.”
Hidayat, M. L., Ahmad, I., Assidiqi, M. H., Khanzada, T. J. S., & Meccawy, M. (2025) · International Conference on Innovation in Artificial Intelligence and Internet of Things (AIIT) · doi:10.1109/aiit63112.2025.11082869 · Cited by 2
“The major cited articles in this cluster are [16], [19], [51], [52].”
Xie, X. Y., & Cui, R. (2025) · International Conference on Computer Technologies (ICCTech) · doi:10.1109/icctech66294.2025.00013 · Cited by 1
“Current technologies have proven the efficacy of dialogue analysis in customizing user experiences, especially in healthcare and customer support [8] .”
Minian, N., Mehra, K., Rose, J., Veldhuizen, S., Zawertailo, L. A., Ratto, M., Ting‐A‐Kee, R. A., Melamed, O. C., Tang, V. M., & Selby, P. L. (2025) · Digital Health · doi:10.1177/20552076251380324 · Cited by 1
Metadata
- Authors
- Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., & Coiera, E.
- Affiliations
- Macquarie University (AU)
- Corresponding author
- Laranjo, Liliana
- ORCID
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-07-11
- Volume
- 25
- Issue
- 9
- Pages
- 1248-1258
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC6118869
- OpenAlex
- W2843010082
- Cited by
- 1,388
- References
- 60
- Citations by year
- 2026: 291 · 2025: 246 · 2024: 182 · 2023: 182 · 2022: 151 · 2021: 143 · 2020: 124 · 2019: 62 · 2018: 6
- FWCI
- 1.87
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
- Funders
- National Health and Medical Research Council (APP1134919)
- Centre for Research Excellence in Digital Health (APP1054146)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- chatbots, healthcare chatbots, natural language input, dialogue management, self-care support, depression symptom reduction, patient safety, evaluation metrics
- MeSH terms
- Artificial Intelligence, Communication, Delivery of Health Care, Natural Language Processing, Speech Recognition Software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.