Review of artificial intelligence‐based question‐answering systems in healthcare
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Use of conversational agents, like chatbots, avatars, and robots is increasing worldwide. Yet, their effectiveness in health care is largely unknown. The aim of this advanced review was to assess the use and effectiveness of conversational agents in various fields of health care. A literature search, analysis, and synthesis were conducted in February 2022 in PubMed and CINAHL. The included evidence was analyzed narratively by employing the principles of thematic analysis. We reviewed articles on artificial intelligence‐based question‐answering systems in health care. Most of the identified articles report its effectiveness; less is known about its use. We outlined study findings and explored directions of future research, to provide evidence‐based knowledge about artificial intelligence‐based question‐answering systems. This article is categorized under: Fundamental Concepts of Data and Knowledge > Human Centricity and User Interaction Application Areas > Health Care Technologies > Artificial Intelligence
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1×
“…of students, conversational agents were useful in improving the confidence in communication skills (Borja-Hart et al., 2019), reducing symptoms of depression and anxiety (Fitzpatrick et al., 2017) and in the development of clinical reasoning and history taking skills (Isaza-Restrepo et al., 2018).”
BackgroundConversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063
Nadarzynski, T., Miles, O., Cowie, A., & Ridge, D. (2019) · DIGITAL HEALTH · doi:10.1177/2055207619871808 · Cited by 810 · cited 1×
“Students' willingness to engage with AI-led health chatbots depends on their IT skills, utility, attitude, and perceived trustworthiness (Nadarzynski et al., 2019).”
BackgroundFulmer, R., Joerin, A., Gentile, B., Lakerink, L., & Rauws, M. (2018) · doi:10.2196/preprints.9782 · Cited by 715
The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review
Milne-Ives, M., de Cock, C., Lim, E., Shehadeh, M. H., de Pennington, N., Mole, G., Normando, E., & Meinert, E. (2020) · Journal of Medical Internet Research · doi:10.2196/20346 · Cited by 626 · cited 2×
“health care professionals (Milne-Ives et al., 2020).”
BackgroundConversational Agents in Health Care: Scoping Review and Conceptual Analysis
Tudor Car, L., Dhinagaran, D. A., Kyaw, B. M., Kowatsch, T., Joty, S., Theng, Y.-L., & Atun, R. (2020) · Journal of Medical Internet Research · doi:10.2196/17158 · Cited by 553 · cited 2×
“Most of the literature reports on text-based conversational agents mention the use of AI and smartphone apps (Car et al., 2020).”
BackgroundLy, K. H., Ly, A.-M., & Andersson, G. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.10.002 · Cited by 368
The Personalization of Conversational Agents in Health Care: Systematic Review
Kocaballi, A. B., Berkovsky, S., Quiroz, J. C., Laranjo, L., Tong, H. L., Rezazadegan, D., Briatore, A., & Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/15360 · Cited by 332
Survey of conversational agents in health
Montenegro, J. L. Z., da Costa, C. A., & da Rosa Righi, R. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.054 · Cited by 322
Schachner, T., Keller, R., & v Wangenheim, F. (2020) · Journal of Medical Internet Research · doi:10.2196/20701 · Cited by 267
Chaix, B., Bibault, J.-E., Pienkowski, A., Delamon, G., Guillemassé, A., Nectoux, P., & Brouard, B. (2019) · JMIR Cancer · doi:10.2196/12856 · Cited by 227
Reflection Companion: A Conversational System for Engaging Users in Reflection on Physical Activity
Kocielnik, R., Xiao, L., Avrahami, D., & Hsieh, G. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3214273 · Cited by 221
Hauser-Ulrich, S., Künzli, H., Meier-Peterhans, D., & Kowatsch, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/15806 · Cited by 210
Stein, N., & Brooks, K. (2017) · JMIR Diabetes · doi:10.2196/diabetes.8590 · Cited by 191
Gong, E., Baptista, S., Russell, A., Scuffham, P., Riddell, M., Speight, J., Bird, D., Williams, E., Lotfaliany, M., & Oldenburg, B. (2020) · Journal of Medical Internet Research · doi:10.2196/20322 · Cited by 159
Gardiner, P. M., McCue, K. D., Negash, L. M., Cheng, T., White, L. F., Yinusa-Nyahkoon, L., Jack, B. W., & Bickmore, T. W. (2017) · Patient Education and Counseling · doi:10.1016/j.pec.2017.04.015 · Cited by 139
The virtual patient as a learning tool: a mixed quantitative qualitative study
Isaza-Restrepo, A., Gómez, M. T., Cifuentes, G., & Argüello, A. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1395-8 · Cited by 129 · cited 1×
“…of students, conversational agents were useful in improving the confidence in communication skills (Borja-Hart et al., 2019), reducing symptoms of depression and anxiety (Fitzpatrick et al., 2017) and in the development of clinical reasoning and history taking skills (Isaza-Restrepo et al., 2018).”
BackgroundPhilip, P., Micoulaud-Franchi, J.-A., Sagaspe, P., Sevin, E. D., Olive, J., Bioulac, S., & Sauteraud, A. (2017) · Scientific Reports · doi:10.1038/srep42656 · Cited by 128 · cited 1×
“Conversational agents were useful also for patients (Chaix et al., 2019; Denecke et al., 2020; Inkster et al., 2018; Kowatsch et al., 2017; Philip et al., 2017) and helped them to track treatment effectively (Chaix et al., 2019; Dworkin et al., 2019; Wang et al., 2018).”
BackgroundEMMA: an emotion-aware wellbeing chatbot
Ghandeharioun, A., McDuff, D., Czerwinski, M., & Rowan, K. (2019) · 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2019.8925455 · Cited by 126
Kowatsch, T., Schachner, T., Harperink, S., Barata, F., Dittler, U., Xiao, G., Stanger, C., v Wangenheim, F., Fleisch, E., Oswald, H., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/25060 · Cited by 103
Easton, K., Potter, S., Bec, R., Bennion, M., Christensen, H., Grindell, C., Mirheidari, B., Weich, S., de Witte, L., Wolstenholme, D., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/12996 · Cited by 88
Bian, Y., Xiang, Y., Tong, B., Feng, B., & Weng, X. (2020) · Journal of Medical Internet Research · doi:10.2196/16896 · Cited by 84
Cheng, A., Raghavaraju, V., Kanugo, J., Handrianto, Y. P., & Shang, Y. (2018) · 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC) · doi:10.1109/ccnc.2018.8319283 · Cited by 82
Designing a spoken dialogue interface to an intelligent cognitive assistant for people with dementia
Wolters, M. K., Kelly, F., & Kilgour, J. (2016) · Health Informatics Journal · doi:10.1177/1460458215593329 · Cited by 79 · cited 1×
“Bickmore et al. (2018) reported that only 15% of patients used a conversational assistant regularly while Wolters et al. (2016) found that the intensity and style of the cognitive assistant's voice must be adjusted to the user's wishes.”
BackgroundMartínez-Miranda, J., Martínez, A., Ramos, R., Aguilar, H., Jiménez, L., Arias, H., Rosales, G., & Valencia, E. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1387-1 · Cited by 79
Kowatsch, T., Nißen, M. K., Shih, C.-H. I., Rüegger, D., Volland, D., Filler, A., Künzler, F., Barata, F., Hung, S., Büchter, D., et al. (2017) · ETH Zurich · doi:10.3929/ethz-b-000218776 · Cited by 74 · cited 1×
“Conversational agents were useful also for patients (Chaix et al., 2019; Denecke et al., 2020; Inkster et al., 2018; Kowatsch et al., 2017; Philip et al., 2017) and helped them to track treatment effectively (Chaix et al., 2019; Dworkin et al., 2019; Wang et al., 2018).”
BackgroundA Chatbot System for Mental Healthcare Based on SAT Counseling Method
Kamita, T., Ito, T., Matsumoto, A., Munakata, T., & Inoue, T. (2019) · Mobile Information Systems · doi:10.1155/2019/9517321 · Cited by 69
Bennion, M. R., Hardy, G. E., Moore, R. K., Kellett, S., & Millings, A. (2020) · Journal of Medical Internet Research · doi:10.2196/16794 · Cited by 68
Owens, O. L., Felder, T., Tavakoli, A. S., Revels, A. A., Friedman, D. B., Hughes-Halbert, C., & Hébert, J. R. (2019) · American Journal of Health Promotion · doi:10.1177/0890117118786866 · Cited by 65
Danieli, M., Ciulli, T., Mousavi, S. M., & Riccardi, G. (2021) · JMIR Formative Research · doi:10.2196/30053 · Cited by 64
The potential of conversational agents to provide a rapid HIV counseling and testing services
van Heerden, A., Ntinga, X., & Vilakazi, K. (2017) · 2017 International Conference on the Frontiers and Advances in Data Science (FADS) · doi:10.1109/fads.2017.8253198 · Cited by 57
Social Media–based Conversational Agents for Health Management and Interventions
Wang, H., Zhang, Q., Ip, M., & Fai Lau, J. T. (2018) · Computer · doi:10.1109/mc.2018.3191249 · Cited by 53
Aquabot: A Diagnostic Chatbot for Achluophobia and Autism
Mujeeb, S., Hafeez, M., & Arshad, T. (2017) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2017.080930 · Cited by 44 · cited 1×
“Mujeeb et al. (2017) show that conversational agents can be useful for practitioner psychologists to assist a psychologist as well as save time and resources.”
BackgroundMedical Instructed Real-Time Assistant for Patient with Glaucoma and Diabetic Conditions
Rehman, U. U., Chang, D. J., Jung, Y., Akhtar, U., Razzaq, M. A., & Lee, S. (2020) · Applied Sciences · doi:10.3390/app10072216 · Cited by 37
In bed with Siri and Google Assistant: a comparison of sexual health advice
Wilson, N., MacDonald, E. J., Mansoor, O. D., & Morgan, J. (2017) · BMJ · doi:10.1136/bmj.j5635 · Cited by 32
Borja-Hart, N. L., Spivey, C. A., & George, C. M. (2019) · Currents in Pharmacy Teaching and Learning · doi:10.1016/j.cptl.2019.03.009 · Cited by 31
MP15-03 NOVEL APPLICATION FOR SEXUAL TRANSMITTED INFECTION SCREENING WITH AN AI CHATBOT
Kobori, Y., Osaka, A., Soh, S., & Okada, H. (2018) · Journal of Urology · doi:10.1016/j.juro.2018.02.516 · Cited by 29
CARMIE: A Conversational Medication Assistant for Heart Failure
Lobo, J., Ferreira, L., & Ferreira, A. J. (2017) · International Journal of E-Health and Medical Communications · doi:10.4018/ijehmc.2017100102 · Cited by 15
Amith, M., Lin, R., Cunningham, R., Wu, Q. L., Savas, L. S., Gong, Y., Boom, J. A., Tang, L., & Tao, C. (2020) · arXiv · doi:10.48550/arxiv.2001.02306 · Cited by 14
Greuter, S., Balandin, S., & Watson, J. (2019) · Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts · doi:10.1145/3341215.3356308 · Cited by 10
Cited by
Sohail, S. S., Farhat, F., Himeur, Y., Nadeem, M., Madsen, D. Ø., Singh, Y., Atalla, S., & Mansoor, W. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101675 · Cited by 192
Meyer, A., Riese, J., & Streichert, T. (2024) · JMIR Medical Education · doi:10.2196/50965 · Cited by 110
“5’s training data set includes not only a substantial and reliable portion of psychiatric data, but also its developers might have first fine-tuned ChatGPT specifically in this domain in anticipation of its high demand [51-53].”
Can ChatGPT Help in Electronics Research and Development? A Case Study with Applied Sensors
Tafferner, Z., Balázs, B., Krammer, O., & Géczy, A. (2023) · Sensors · doi:10.3390/s23104879 · Cited by 31
Metadata
- Authors
- Budler, L. C., Gosak, L., & Stiglic, G.
- Venue
- WIREs Data Mining and Knowledge Discovery
- Publisher
- Wiley
- Published
- 2023-01-10
- Volume
- 13
- Issue
- 2
- ISSN
- 1942-4787 (ISSN-L)
- 1942-4795
- OpenAlex
- W4315649305
- Cited by
- 52
- References
- 41
- FWCI
- 1.80
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.