The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Patient Perspectives on the Usefulness of an Artificial Intelligence–Assisted Symptom Checker: Cross-Sectional Survey Study
Open accessAbstract
Background: Patients are increasingly seeking Web-based symptom checkers to obtain diagnoses. However, little is known about the characteristics of the patients who use these resources, their rationale for use, and whether they find them accurate and useful. Objective: The study aimed to examine patients’ experiences using an artificial intelligence (AI)–assisted online symptom checker. Methods: An online survey was administered between March 2, 2018, through March 15, 2018, to US users of the Isabel Symptom Checker within 6 months of their use. User characteristics, experiences of symptom checker use, experiences discussing results with physicians, and prior personal history of experiencing a diagnostic error were collected. Results: A total of 329 usable responses was obtained. The mean respondent age was 48.0 (SD 16.7) years; most were women (230/304, 75.7%) and white (271/304, 89.1%). Patients most commonly used the symptom checker to better understand the causes of their symptoms (232/304, 76.3%), followed by for deciding whether to seek care (101/304, 33.2%) or where (eg, primary or urgent care: 63/304, 20.7%), obtaining medical advice without going to a doctor (48/304, 15.8%), and understanding their diagnoses better (39/304, 12.8%). Most patients reported receiving useful information for their health problems (274/304, 90.1%), with half reporting positive health effects (154/302, 51.0%). Most patients perceived it to be useful as a diagnostic tool (253/301, 84.1%), as a tool providing insights leading them closer to correct diagnoses (231/303, 76.2%), and reported they would use it again (278/304, 91.4%). Patients who discussed findings with their physicians (103/213, 48.4%) more often felt physicians were interested (42/103, 40.8%) than not interested in learning about the tool’s results (24/103, 23.3%) and more often felt physicians were open (62/103, 60.2%) than not open (21/103, 20.4%) to discussing the results. Compared with patients who had not previously experienced diagnostic errors (missed or delayed diagnoses: 123/304, 40.5%), patients who had previously experienced diagnostic errors (181/304, 59.5%) were more likely to use the symptom checker to determine where they should seek care (15/123, 12.2% vs 48/181, 26.5%; P=.002), but they less often felt that physicians were interested in discussing the tool’s results (20/34, 59% vs 22/69, 32%; P=.04). Conclusions: Despite ongoing concerns about symptom checker accuracy, a large patient-user group perceived an AI-assisted symptom checker as useful for diagnosis. Formal validation studies evaluating symptom checker accuracy and effectiveness in real-world practice could provide additional useful information about their benefit.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Singh, H., Meyer, A. N. D., & Thomas, E. J. (2014) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002627 · Cited by 677
Self-citationCyberchondria: studies of the escalation of medical concerns in web search
White, R. W., & Horvitz, E. (2009) · ACM Transactions on Information Systems · doi:10.1145/1629096.1629101 · Cited by 546
Evaluation of symptom checkers for self diagnosis and triage: audit study
Semigran, H. L., Linder, J. A., Gidengil, C., & Mehrotra, A. (2015) · BMJ · doi:10.1136/bmj.h3480 · Cited by 489 · cited 3×
“…[Isabel Healthcare] intended for clinicians) and has been shown to have better accuracy than the average symptom checker in a vignette-based study (defined as having the correct diagnosis listed first in 44% of cases compared with an average rate of 34% in the 23 symptom checkers tested) [9].”
BackgroundInfluentialConsumer Mobile Health Apps: Current State, Barriers, and Future Directions
Kao, C., & Liebovitz, D. M. (2017) · PM&R · doi:10.1016/j.pmrj.2017.02.018 · Cited by 350
Beyond Dr. Google: the evidence on consumer-facing digital tools for diagnosis
Millenson, M. L., Baldwin, J. L., Zipperer, L., & Singh, H. (2018) · Diagnosis · doi:10.1515/dx-2018-0009 · Cited by 91 · cited 3×
“In addition to searching the internet for health information, use of digital health care tools includes online, artificial intelligence (AI)–assisted symptom checkers for obtaining diagnoses or self-triage [7-10].”
BackgroundSelf-citationInternet Self-Diagnosis in Hand Surgery
Hageman, M. G. J. S., Anderson, J., Blok, R., Bossen, J. K. J., & Ring, D. (2015) · HAND · doi:10.1007/s11552-014-9707-x · Cited by 26
Cited by
Young, A. T., Amara, D., Bhattacharya, A., & Wei, M. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00132-1 · Cited by 400
Mobile Health (mHealth) in Low- and Middle-Income Countries
McCool, J., Dobson, R., Whittaker, R., & Paton, C. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-052620-093850 · Cited by 269
Data Integration Challenges for Machine Learning in Precision Medicine
Martínez-García, M., & Hernández-Lemus, E. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2021.784455 · Cited by 185
Accuracy of online symptom checkers and the potential impact on service utilisation
Ceney, A., Tolond, S., Glowinski, A., Marks, B., Swift, S., & Palser, T. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254088 · Cited by 125
Miller, S., Gilbert, S., Virani, V., & Wicks, P. (2020) · JMIR Human Factors · doi:10.2196/19713 · Cited by 117
“It remains to be seen how many patients would actually follow advice on where to go next, but in the survey of US Isabel symptom checker users, about half (14/26, 54%) of those advised to go to the emergency department reported that they did so [29].”
BackgroundSalwei, M. E., & Carayon, P. (2022) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434221097357 · Cited by 112
Jones, O. T., Calanzani, N., Saji, S., Duffy, S. W., Emery, J., Hamilton, W., Singh, H., de Wit, N. J., & Walter, F. M. (2021) · Journal of Medical Internet Research · doi:10.2196/23483 · Cited by 65
Self-citationvan Bussel, M. J. P., Odekerken–Schröder, G. J., Ou, C., Swart, R. R., & Jacobs, M. J. G. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08189-7 · Cited by 60
“Another study [23] found that patients used a symptom checker most commonly to understand the cause of their symptoms and saw it as a helpful diagnostic tool receiving useful information.”
BackgroundDeterminants of Laypersons’ Trust in Medical Decision Aids: Randomized Controlled Trial
Kopka, M., Schmieding, M. L., Rieger, T., Roesler, E., Balzer, F., & Feufel, M. A. (2022) · JMIR Human Factors · doi:10.2196/35219 · Cited by 37
Hennemann, S., Kuhn, S., Witthöft, M., & Jungmann, S. M. (2022) · JMIR Mental Health · doi:10.2196/32832 · Cited by 31
“Research indicates that, although symptom checkers seem to be easy to use and well-accepted by most users [30,31], the diagnostic performance varies significantly between different symptom checkers and has been interpreted as low to moderate at best [32,33].”
BackgroundAI and Aphasia in the Digital Age: A Critical Review
Privitera, A. J., Ng, S. H. S., Kong, A. P.-H., & Weekes, B. S. (2024) · Brain Sciences · doi:10.3390/brainsci14040383 · Cited by 26
Peven, K., Wickham, A. P., Wilks, O., Kaplan, Y. C., Marhol, A., Ahmed, S., Bamford, R., Cunningham, A. C., Prentice, C., Meczner, A., et al. (2023) · JMIR mHealth and uHealth · doi:10.2196/46718 · Cited by 9
Metadata
- Authors
- Meyer, A. N. D., Giardina, T. D., Spitzmueller, C., Shahid, U., Scott, T. M. T., & Singh, H.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2020-01-30
- Volume
- 22
- Issue
- 1
- Pages
- e14679
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2982083314
- Cited by
- 161
- References
- 6
- FWCI
- 3.77
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.