A Chatbot Versus Physicians to Provide Information for Patients With Breast Cancer: Blind, Randomized Controlled Noninferiority Trial
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Background: The data regarding the use of conversational agents in oncology are scarce. Objective: The aim of this study was to verify whether an artificial conversational agent was able to provide answers to patients with breast cancer with a level of satisfaction similar to the answers given by a group of physicians. Methods: This study is a blind, noninferiority randomized controlled trial that compared the information given by the chatbot, Vik, with that given by a multidisciplinary group of physicians to patients with breast cancer. Patients were women with breast cancer in treatment or in remission. The European Organisation for Research and Treatment of Cancer Quality of Life Group information questionnaire (EORTC QLQ-INFO25) was adapted and used to compare the quality of the information provided to patients by the physician or the chatbot. The primary outcome was to show that the answers given by the Vik chatbot to common questions asked by patients with breast cancer about their therapy management are at least as satisfying as answers given by a multidisciplinary medical committee by comparing the success rate in each group (defined by a score above 3). The secondary objective was to compare the average scores obtained by the chatbot and physicians for each INFO25 item. Results: A total of 142 patients were included and randomized into two groups of 71. They were all female with a mean age of 42 years (SD 19). The success rates (as defined by a score >3) was 69% (49/71) in the chatbot group versus 64% (46/71) in the physicians group. The binomial test showed the noninferiority (P<.001) of the chatbot’s answers. Conclusions: This is the first study that assessed an artificial conversational agent used to inform patients with cancer. The EORTC INFO25 scores from the chatbot were found to be noninferior to the scores of the physicians. Artificial conversational agents may save patients with minor health concerns from a visit to the doctor. This could allow clinicians to spend more time to treat patients who need a consultation the most. Trial Registration: Clinicaltrials.gov NCT03556813, https://tinyurl.com/rgtlehq
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“In health care, the first example of a computer program used as a conversational agent was Joseph Weizenbaum’s ELIZA, a program that mimicked a Rogerian psychotherapist and that was able to rephrase the patient’s sentences as questions and provide prerecorded answers [2].”
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388
Miner, A. S., Milstein, A., Schueller, S., Hegde, R., Mangurian, C., & Linos, E. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0400 · Cited by 368
Chaix, B., Bibault, J.-E., Pienkowski, A., Delamon, G., Guillemassé, A., Nectoux, P., & Brouard, B. (2019) · JMIR Cancer · doi:10.2196/12856 · Cited by 227
Self-citation
Cited by
The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review
Milne-Ives, M., de Cock, C., Lim, E., Shehadeh, M. H., de Pennington, N., Mole, G., Normando, E., & Meinert, E. (2020) · Journal of Medical Internet Research · doi:10.2196/20346 · Cited by 626
“10 (91) 1 1 0 1 1 1 1 1 1 1 1 Bibault [46]”
InfluentialXu, L., Sanders, L., Li, K., & Chow, J. C. L. (2021) · JMIR Cancer · doi:10.2196/27850 · Cited by 571
“…and adhering to instructions Infinity [37] Assess health outcomes and impact of phone-based monitoring for cancer patients 65 + years Vik [38, 39] Addresses patient’s daily needs and concerns Support Counselling Vivobot [40] Cognitive and behavioural intervention for positive psychology…”
Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · Journal of Medical Internet Research · doi:10.2196/56930 · Cited by 305
Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review (Preprint)
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · doi:10.2196/preprints.56930 · Cited by 302
Görtz, M., Baumgärtner, K., Schmid, T., Muschko, M., Woessner, P., Gerlach, A., Byczkowski, M., Sültmann, H., Duensing, S., & Hohenfellner, M. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231173304 · Cited by 96
“In this study, the assessment scores from the chatbot were noninferior to the scores of the physicians.(24) In other previous studies, chatbots have been applied to improve access to mental healthcare services, being used for therapy, training, education, counseling, and screening in mental health.”
ResultTawfik, E., Ghallab, E., & Moustafa, A. (2023) · BMC Nursing · doi:10.1186/s12912-023-01243-7 · Cited by 82
“or its non-inferiority in providing high quality information to patients compared to healthcare professionals [43].”
BackgroundArtificial intelligence & clinical nutrition: What the future might have in store
Bond, A., Mccay, K., & Lal, S. (2023) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2023.07.082 · Cited by 79
The Use of Chatbots in Oncological Care: A Narrative Review
Wang, A., Qian, Z., Briggs, L., Cole, A. P., Reis, L. O., & Trinh, Q.-D. (2023) · International Journal of General Medicine · doi:10.2147/ijgm.s408208 · Cited by 73
The Emerging Role of Generative Artificial Intelligence in Medical Education, Research, and Practice
Shoja, M. M., Van de Ridder, J. M. M., & Rajput, V. (2023) · Cureus · doi:10.7759/cureus.40883 · Cited by 69
“The early generation of chatbots has been shown to be non-inferior to clinicians in randomized trials of education, follow-up, and adherence in breast cancer patients [33].”
van Bussel, M. J. P., Odekerken–Schröder, G. J., Ou, C., Swart, R. R., & Jacobs, M. J. G. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08189-7 · Cited by 60
“It improved their adherence rate, provided helpful treatment information and support [15, 16].”
BackgroundChatGPT Is Moderately Accurate in Providing a General Overview of Orthopaedic Conditions
Sparks, C. A., Fasulo, S. M., Windsor, J. T., Bankauskas, V., Contrada, E. V., Kraeutler, M. J., & Scillia, A. J. (2024) · JBJS Open Access · doi:10.2106/jbjs.oa.23.00129 · Cited by 29
Chatbots in cancer applications, advantages and disadvantages: all that glitters is not gold
Goumas, G., Dardavesis, T. I., Syrigos, K., Syrigos, N., & Simou, E. (2024) · Journal of Personalized Medicine · doi:10.3390/jpm14080877 · Cited by 24
“Importantly, a study on women with breast cancer found that the chatbot’s EORTC INFO25 scores were not less than the doctors’ scores in providing the answers the women needed [54].”
BackgroundSanabria, G., Greene, K. Y., Tran, J. T., Gilyard, S., DiGiovanni, L., Emmanuel, P. J., Sanders, L. J., Kosyluk, K., & Galea, J. T. (2023) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-023-00315-4 · Cited by 25
Igelström, H., Hauffman, A., Alfonsson, S., Sjöström, J., Cajander, Å., & Johansson, B. (2020) · Journal of Medical Internet Research · doi:10.2196/16604 · Cited by 16
Hu, Y., Liu, J., & Jiang, W. (2025) · Renal Failure · doi:10.1080/0886022x.2025.2555686 · Cited by 14
“…context. early experiences in other fields suggest that patients often find chatbot explanations articulate and encompassing but not yet perfectly tailored [13]. in nephrology, ongoing pilot studies and patient feedback will be critical to refine LLM communication for this sensitive domain.”
Li, K., Wu, S., Zhang, Y., Zhu, B., Qi, Z., Hou, S., & Wang, W. (2026) · Journal of Clinical Nursing · doi:10.1111/jocn.70020 · Cited by 10
Jiang, X.-H., Yuan, X.-H., Zhao, H., & Peng, J.-S. (2025) · Journal of Medical Internet Research · doi:10.2196/76968 · Cited by 10
Chaix, B., Lobre, G., Mahboub, S., Delamon, G., Bibault, J.-E., & Brouard, B. (2020) · Psycho-Oncologie · doi:10.3166/pson-2020-0113 · Cited by 1
Self-citation
Metadata
- Authors
- Bibault, J.-E., Chaix, B., Guillemassé, A., Cousin, S., Escande, A., Perrin, M., Pienkowski, A., Delamon, G., Nectoux, P., & Brouard, B.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-11-27
- Volume
- 21
- Issue
- 11
- Pages
- e15787
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2990464492
- Cited by
- 202
- References
- 5
- FWCI
- 3.77
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.