The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Accuracy of online symptom checkers and the potential impact on service utilisation
Open accessPublished version of a preprint
Abstract
Objectives: The aims of our study are firstly to investigate the diagnostic and triage performance of symptom checkers, secondly to assess their potential impact on healthcare utilisation and thirdly to investigate for variation in performance between systems. Setting: Publicly available symptom checkers for patient use. Participants: Publicly available symptom-checkers were identified. A standardised set of 50 clinical vignettes were developed and systematically run through each system by a non-clinical researcher. Primary and secondary outcome measures: System accuracy was assessed by measuring the percentage of times the correct diagnosis was a) listed first, b) within the top five diagnoses listed and c) listed at all. The safety of the disposition advice was assessed by comparing it with national guidelines for each vignette. Results: Twelve tools were identified and included. Mean diagnostic accuracy of the systems was poor, with the correct diagnosis being present in the top five diagnoses on 51.0% (Range 22.2 to 84.0%). Safety of disposition advice decreased with condition urgency (being 71.8% for emergency cases vs 87.3% for non-urgent cases). 51.0% of systems suggested additional resource utilisation above that recommended by national guidelines (range 18.0% to 61.2%). Both diagnostic accuracy and appropriate resource recommendation varied substantially between systems. Conclusions: There is wide variation in performance between available symptom checkers and overall performance is significantly below what would be accepted in any other medical field, though some do achieve a good level of accuracy and safety of disposition. External validation and regulation are urgently required to ensure these public facing tools are safe.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Research indicates that, although symptom checkers seem to be easy to use and well-accepted by most users [30,31], the diagnostic performance varies significantly between different symptom checkers and has been interpreted as low to moderate at best [32,33].”
From Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients (2022) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“As previous studies have shown considerable differences between symptom checkers’ diagnostic accuracy [33,35], future studies could compare various symptom checkers for the formal diagnosis of mental disorders.”
From Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients (2022) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Evaluation of symptom checkers for self diagnosis and triage: audit study
Semigran, H. L., Linder, J. A., Gidengil, C., & Mehrotra, A. (2015) · BMJ · doi:10.1136/bmj.h3480 · Cited by 489
Applications of Machine Learning Predictive Models in the Chronic Disease Diagnosis
Battineni, G., Sagaro, G. G., Chintalapudi, N., & Amenta, F. (2020) · Journal of Personalized Medicine · doi:10.3390/jpm10020021 · Cited by 418
Coster, J. E., Turner, J. K., Bradbury, D., & Cantrell, A. (2017) · Academic Emergency Medicine · doi:10.1111/acem.13220 · Cited by 379
Amante, D. J., Hogan, T. P., Pagoto, S., English, T. M., & Lapane, K. L. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4126 · Cited by 350
Chambers, D., Cantrell, A. J., Johnson, M., Preston, L., Baxter, S. K., Booth, A., & Turner, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-027743 · Cited by 218 · cited 1×
“Several previous studies [13, 14] into the effectiveness of algorithmic performance have found deficiencies in the diagnostic capabilities and a cautious approach to triage.”
BackgroundGilbert, S., Mehl, A., Baluch, A., Cawley, C., Challiner, J., Fraser, H., Millen, E., Montazeri, M., Multmeier, J., Pick, F., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040269 · Cited by 183 · cited 3×
“[15]) and one published at the end of last year [16] examined multiple presentations and conditions; the others focused on single condition studies such as those examining system performance for cervical myelopathy, inflammatory arthritis, HIV / Hepatitis C and ophthalmic conditions [17–20].”
BackgroundResultSafety of patient-facing digital symptom checkers
Fraser, H., Coiera, E. W., & Wong, D. (2018) · The Lancet · doi:10.1016/s0140-6736(18)32819-8 · Cited by 182
A Comparison of Artificial Intelligence and Human Doctors for the Purpose of Triage and Diagnosis
Baker, A., Perov, Y., Middleton, K., Baxter, J., Mullarkey, D., Sangar, D., Butt, M., DoRosario, A., & Johri, S. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.543405 · Cited by 167
Meyer, A. N. D., Giardina, T. D., Spitzmueller, C., Shahid, U., Scott, T. M. T., & Singh, H. (2019) · Journal of Medical Internet Research · doi:10.2196/14679 · Cited by 162
O’Cathain, A., Connell, J., Long, J., & Coster, J. E. (2019) · Health Expectations · doi:10.1111/hex.12995 · Cited by 112
Research works printed but not linked: 1
- J Beech, 2019, Closing the Gap: Key Areas for Action on the Health and Care Workforce
Cited by
Sorted by citations
Hirosawa, T., Harada, Y., Yokose, M., Sakamoto, T., Kawamura, R., & Shimizu, T. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043378 · Cited by 351
The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review
Wallace, W. J., Chan, C., Chidambaram, S., Hanna, L., Iqbal, F. M., Acharya, A., Normahani, P., Ashrafian, H., Markar, S. R., Sounderajah, V., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00667-w · Cited by 213
Fraser, H., Crossland, D., Bacher, I., Ranney, M., Madsen, T., & Hilliard, R. (2023) · JMIR mHealth and uHealth · doi:10.2196/49995 · Cited by 128
Advancing AI in healthcare: A comprehensive review of best practices
Polevikov, S. (2023) · Clinica Chimica Acta · doi:10.1016/j.cca.2023.117519 · Cited by 121
Triage Accuracy of Symptom Checker Apps: 5-Year Follow-up Evaluation
Schmieding, M. L., Kopka, M., Schmidt, K. F. R., Schulz-Niethammer, S., Balzer, F., & Feufel, M. A. (2022) · Journal of Medical Internet Research · doi:10.2196/31810 · Cited by 99
ChatGPT for Clinical Vignette Generation, Revision, and Evaluation
Benoit, J. R. A. (2023) · doi:10.1101/2023.02.04.23285478 · Cited by 74
Triage and Diagnostic Accuracy of Online Symptom Checkers: Systematic Review
Sasco, E. R., El-Osta, A. A. R., Alaa, A., Webber, I., Karki, M., Asmar, M. L. E., Purohit, K., Painter, A., & Hayhoe, B. W. J. (2023) · Journal of Medical Internet Research · doi:10.2196/43803 · Cited by 85
Sarbay, İ., Berikol, G. B., & Özturan, İ. U. (2023) · Turkish Journal of Emergency Medicine · doi:10.4103/tjem.tjem_79_23 · Cited by 69
Dialogue System for Early Mental Illness Detection: Toward a Digital Twin Solution
Abilkaiyrkyzy, A., Laamarti, F., Hamdi, M., & Saddik, A. E. (2024) · IEEE Access · doi:10.1109/access.2023.3348783 · Cited by 73
Pairon, A., Philips, H., & Verhoeven, V. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2022.1040926 · Cited by 48
Bushuven, S., Bentele, M., Bentele, S., Gerber, B., Bansbach, J., Ganter, J., Trifunovic-Koenig, M., & Ranisch, R. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-02019-x · Cited by 49
Kopka, M., von Kalckreuth, N., & Feufel, M. A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01566-6 · Cited by 57
Cotte, F., Mueller, T., Gilbert, S., Blümke, B., Multmeier, J., Hirsch, M. C., Wicks, P. J., Wolanski, J., Tutschkow, D., Schade‐Brittinger, C., et al. (2022) · JMIR mhealth and uhealth · doi:10.2196/32340 · Cited by 41
Determinants of Laypersons’ Trust in Medical Decision Aids: Randomized Controlled Trial
Kopka, M., Schmieding, M. L., Rieger, T., Roesler, E., Balzer, F., & Feufel, M. A. (2022) · JMIR Human Factors · doi:10.2196/35219 · Cited by 37
El-Osta, A. A. R., Webber, I., Alaa, A., Bagkeris, E., Mian, S., Sharabiani, M. T. A., & Majeed, A. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-053566 · Cited by 30
Hennemann, S., Kuhn, S., Witthöft, M., & Jungmann, S. M. (2022) · JMIR Mental Health · doi:10.2196/32832 · Cited by 31
“Most recently, a study by Ceney et al [33] yielded comparable average performance rates (51%, range 22.2%-84”
BackgroundResultInfluentialChallenges in evaluating the accuracy of AI-containing digital triage systems: A systematic review
Ilicki, J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0279636 · Cited by 33
Sun, S. H., Huynh, K. N., Cortes, G., Hill, R., Tran, J., Yeh, L., Ngo, A. L., Houshyar, R., Yaghmai, V., & Tran, M. D. (2024) · Radiology · doi:10.1148/radiol.232346 · Cited by 33
Fraser, H., Cohan, G., Koehler, C., Anderson, J. S., Lawrence, A., Patena, J. V., Bacher, I., & Ranney, M. L. (2022) · JMIR mhealth and uhealth · doi:10.2196/38364 · Cited by 32
Radionova, N., Ög, E., Wetzel, A.-J., Rieger, M. A., & Preiser, C. (2023) · Journal of Medical Internet Research · doi:10.2196/39219 · Cited by 35
Online Symptom Checkers: Recommendations for a Vignette-Based Clinical Evaluation Standard
Painter, A., Hayhoe, B. W. J., Sasco, E. R., & El-Osta, A. A. R. (2022) · Journal of Medical Internet Research · doi:10.2196/37408 · Cited by 30
Assessing data gathering of chatbot based symptom checkers - a clinical vignettes study
Ben‐Shabat, N., Sharvit, G., Meimis, B., Joya, D. B., Sloma, A., Kiderman, D., Shabat, A., Tsur, A. M., Watad, ., & Amital, H. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104897 · Cited by 28
Telepsychiatry in an Era of Digital Mental Health Startups
Achtyes, E. D., Glenn, T., Monteith, S., Geddes, J. R., Whybrow, P. C., Martini, J., & Bauer, M. (2023) · Current Psychiatry Reports · doi:10.1007/s11920-023-01425-9 · Cited by 32
Benchmarking the symptom-checking capabilities of ChatGPT for a broad range of diseases
Chen, A., Chen, D. O., & Tian, L. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad245 · Cited by 25
Prevalence and course of pregnancy symptoms using self-reported pregnancy app symptom tracker data
Nissen, M., Campo, N. B., Flaucher, M., Jaeger, K. M., Titzmann, A., Blunck, D., Fasching, P. A., Engelhardt, V., Eskofier, B. M., & Leutheuser, H. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00935-3 · Cited by 28
Knitza, J., Taşçılar, K., Fuchs, F., Mohn, J., Kühn, S., Bohr, D., Muehlensiepen, F., Bergmann, C., Labinsky, H., Morf, H., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/55542 · Cited by 24
Faqar-Uz-Zaman, S. F., Anantharajah, L., Baumartz, P., Sobotta, P., Filmann, N., Zmuc, D., Wagner, M., Detemble, C., Sliwinski, S., Marschall, U., et al. (2022) · Annals of Surgery · doi:10.1097/sla.0000000000005614 · Cited by 25
Extracting symptoms from free-text responses using ChatGPT among COVID-19 cases in Hong Kong
Wei, W. I., Leung, C. L. K., Tang, A., McNeil, E. B., Wong, S. Y. S., & Kwok, K. O. (2023) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2023.11.002 · Cited by 24
Kopka, M., Feufel, M. A., Berner, E. S., & Schmieding, M. L. (2023) · Digital Health · doi:10.1177/20552076231194929 · Cited by 23
The Triage Capability of Laypersons: Retrospective Exploratory Analysis
Kopka, M., Feufel, M. A., Balzer, F., & Schmieding, M. L. (2022) · JMIR Formative Research · doi:10.2196/38977 · Cited by 22
Triage Errors in Primary and Pre–Primary Care
Nguyen, H., Meczner, A., Burslam-Dawe, K., & Hayhoe, B. W. J. (2022) · Journal of Medical Internet Research · doi:10.2196/37209 · Cited by 21
Latent classes associated with the intention to use a symptom checker for self-triage
Aboueid, S., Meyer, S. B., Wallace, J. R., & Chaurasia, A. K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0259547 · Cited by 16
Gilbert, S., Fenech, M. E., Upadhyay, S., Wicks, P. J., & Novorol, C. (2021) · Australian Journal of Primary Health · doi:10.1071/py21032 · Cited by 17
Wetzel, A.-J., Koch, R., Preiser, C., Mueller, R., Klemmt, M., Ranisch, R., Ehni, H.-J., Wiesing, U., Rieger, M. A., Henking, T., et al. (2022) · JMIR Research Protocols · doi:10.2196/34026 · Cited by 15
Guideline for software life cycle in health informatics
Hauschild, A.-C., Martin, R., Holst, S. C., Wienbeck, J., & Heider, D. (2022) · iScience · doi:10.1016/j.isci.2022.105534 · Cited by 22
Zaman, N. I. D., Hau, Y. W., Leong, M. C., & Al‐Ashwal, R. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10528-x · Cited by 16
Evaluating the Diagnostic Performance of Symptom Checkers: Clinical Vignette Study
Hammoud, M., Douglas, S., Darmach, M., Alawneh, S., Sanyal, S., & Kanbour, Y. (2024) · JMIR AI · doi:10.2196/46875 · Cited by 21
Schmieding, M. L., Kopka, M., Bolanaki, M., Napierala, H., Altendorf, M. B., Kuschick, D., Piper, S. K., Scatturin, L., Schmidt, K. F. R., Schorr, C., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/64028 · Cited by 14
Ferhi, L. A., Amar, M. B., Choubani, F., & Bouallègue, R. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1397388 · Cited by 14
Naved, B. A., Ravishankar, S., Colbert, G. E., Johnston, A., Slott, Q. M., & Luo, Y. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01779-9 · Cited by 13
Kopka, M., & Feufel, M. A. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00096-7 · Cited by 13
Leaders’ support for using online symptom checkers in Finnish primary care: Survey study
Laukka, E., Kujala, S., Gluschkoff, K., Kanste, O. I., Hörhammer, I., & Heponiemi, T. (2021) · Health Informatics Journal · doi:10.1177/14604582211052259 · Cited by 12
Diseasomics: Actionable machine interpretable disease knowledge at the point-of-care
Talukder, A. K., Schriml, L. M., Ghosh, A., Biswas, R., Chakrabarti, P., & Haas, R. E. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000128 · Cited by 12
Wetzel, A.-J., Koch, R., Koch, N. N., Klemmt, M., Müller, R., Preiser, C., Rieger, M. A., Rösel, I., Ranisch, R., Ehni, H., et al. (2024) · Digital Health · doi:10.1177/20552076241231555 · Cited by 11
Kopka, M., Napierala, H., Privoznik, M., Sapunova, D., Zhang, S., & Feufel, M. A. (2024) · Scientific Reports · doi:10.1038/s41598-024-83844-z · Cited by 14
Chatbots in Pharmacy: A Boon or a Bane for Patient Care and Pharmacy Practice?
Ramadhani, C. (2023) · Sciences of Pharmacy · doi:10.58920/sciphar02030001 · Cited by 14
Symptoms are known by their companies: towards association guided disease diagnosis assistant
Tiwari, A., Saha, T., Saha, S., Bhattacharyya, P., Begum, S., Dhar, M., & Tiwari, S. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-05032-y · Cited by 11
Faqar-Uz-Zaman, S. F., Filmann, N., Mahkovic, D., Wagner, M., Detemble, C., Kippke, U., Marschall, U., Anantharajah, L., Baumartz, P., Sobotta, P., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-041396 · Cited by 10
Technology-supported self-triage decision making
Kopka, M., Wang, S. M., Kunz, S., Schmid, C., & Feufel, M. A. (2025) · npj Health Systems · doi:10.1038/s44401-024-00008-x · Cited by 11
Vuillaume, L. A., Turpinier, J., Cipolat, L., Arnaud-Dépil-Duval, Dumontier, T., Peschanski, N., Kieffer, Y., Barbat, B., Riquier, T., Dinot, V., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0277568 · Cited by 9
Schmude, M., Salim, N., Azadzoy, H., Bane, M., Millen, E., O’Donnell, L., Bode, P., Türk, E., Vaidya, R., & Gilbert, S. (2022) · JMIR Research Protocols · doi:10.2196/34298 · Cited by 9
de Arriba-Pérez, F., & García-Méndez, S. (2025) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-024-09508-2 · Cited by 8
Reinders, P., Augustin, M., & Otten, M. (2024) · JDDG Journal der Deutschen Dermatologischen Gesellschaft · doi:10.1111/ddg.15454 · Cited by 7
Liu, V., Kaila, M., & Koskela, T. (2024) · JMIR Human Factors · doi:10.2196/55099 · Cited by 7
Empowering Medical Diagnosis: A Machine Learning Approach for Symptom-Based Health Checker
Ferhi, L. A., Amar, M. B., Choubani, F., & Bouallègue, R. (2024) · Mobile Networks and Applications · doi:10.1007/s11036-024-02369-x · Cited by 7
Toward Symptom Assessment Guided Symptom Investigation and Disease Diagnosis
Tiwari, A., Raj, R., Saha, S., Bhattacharyya, P., Tiwari, S., & Dhar, M. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3236897 · Cited by 7
Validation of an Artificial Intelligence-Powered Virtual Assistant for Emergency Triage in Neurology
Alessandro, L., Crema, S., Castiglione, J. I., Dossi, D., Eberbach, F., Köhler, A., Laffue, A., Marone, A., Nagel, V., Rueda, J. M. P., et al. (2025) · The Neurologist · doi:10.1097/nrl.0000000000000594 · Cited by 6
Gilbert, A., Diep, A. N., Boufraioua, M., Pétré, B., Donneau, A., & Ghuysen, A. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08571-5 · Cited by 6
International dimensions of clinical decision support systems
Magrabi, F., Cresswell, K. M., & Fraser, H. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-91200-6.00013-9 · Cited by 6
Liu, V., Koskela, T., & Kaila, M. (2023) · JMIR Research Protocols · doi:10.2196/41423 · Cited by 6
Benoit, J. R. A., Hartling, L., & Scott, S. D. (2023) · PEC Innovation · doi:10.1016/j.pecinn.2023.100152 · Cited by 5
Koch, R., Steffen, M.-T., Wetzel, A.-J., Preiser, C., Klemmt, M., Ehni, H., Mueller, R., & Joos, S. (2025) · JMIR Formative Research · doi:10.2196/60647 · Cited by 5
Avey: An Accurate AI Algorithm for Self-DiagnosisPreprint
Hammoud, M., Douglas, S., Darmach, M., Alawneh, S., Sanyal, S., & Kanbour, Y. (2022) · medRxiv · doi:10.1101/2022.03.08.22272076 · Cited by 5
Evaluating digital triage symptom checker with historical triage-related adverse events
Ilicki, J., Edman, S., Stalfors, J., & Molin, C. J. (2025) · Scandinavian Journal of Primary Health Care · doi:10.1080/02813432.2025.2563517 · Cited by 4
Wallace, W. J., Chan, C., Chidambaram, S., Hanna, L., Acharya, A., Daniels, E., Normahani, P., Matin, R., Markar, S. R., Sounderajah, V., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000558 · Cited by 5
Machine Learning in Clinical Text Classification: Specialty Identification and COVID-19 Risk
Tan, C. Y.-X., Talreja, K., Shivam, A., Nag, A., Miranda, M., Raghav, S., Rajaram, A., Lutz, M., Kumar, S., & Kumar, A. (2022) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc55196.2022.9952512 · Cited by 4
Sendtner, G. W., Muecke, M., Grigull, L., Bender, T. T. A., Behning, C., & Schäfer, V. S. (2025) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-025-04112-5 · Cited by 3
A Comprehensive Examination of User Experience in AI-Based Symptom Checker Chatbots
Ferreira, M. C., Veloso, M., & Tavares, J. M. R. S. (2024) · Lecture notes in business information processing · doi:10.1007/978-3-031-59376-5_8 · Cited by 3
Analysing weighted disease symptoms to recommend an expert through an integrated healthcare model
Mohapatra, S., & Roy, A. (2024) · Discover Public Health · doi:10.1186/s12982-024-00261-3 · Cited by 4
Pankow, A., Meißner-Bendzko, N., Kaufeld, J. K., Fouquette, L., Cotte, F., Gilbert, S., Türk, E., Das, A. M., Terkamp, C., Burmester, G.-R., et al. (2025) · JMIR AI · doi:10.2196/55001 · Cited by 3
Sellin, J., Pantel, J. T., Börsch, N., Conrad, R., & Mücke, M. (2024) · Der Schmerz · doi:10.1007/s00482-023-00777-8 · Cited by 3
Kopka, M., Napierala, H., Privoznik, M., Sapunova, D., Zhang, S., & Feufel, M. A. (2024) · medRxiv · doi:10.1101/2024.04.02.24305193 · Cited by 3
Kopka, M., von Kalckreuth, N., & Feufel, M. A. (2024) · medRxiv · doi:10.1101/2024.09.13.24313657 · Cited by 3
Gehlen, T., Joost, T., Solbrig, P., Stahnke, K., Zahn, R., Jahn, M., Amini, D. A., & Back, D. A. (2025) · Diagnostics · doi:10.3390/diagnostics15020221 · Cited by 2
Pecina, J. L., Curry, E. N., Miller, N. E., Thompson, M. C., & North, F. (2025) · Health Services Insights · doi:10.1177/11786329251385022 · Cited by 2
Kopka, M., & Feufel, M. A. (2026) · JMIR Biomedical Engineering · doi:10.2196/88053 · Cited by 2
Patient-targeted mobile applications in healthcare
Karia, J., Mohamed, R., & Petrushkin, H. (2023) · British Journal of Hospital Medicine · doi:10.12968/hmed.2023.0158 · Cited by 2
Avnat, E., Samin, M., Joya, D. B., Schneider, E., Yanko, E., Eshel, D., Ovadia, S., Lev, Y., & Souroujon, D. (2024) · Journal of Medical Internet Research · doi:10.2196/49570 · Cited by 2
Congruency of Symptom Choice Between Patients Performing Online Self-Triage and Nurse Phone Triage
Curry, E. N., Miller, N. E., North, F., Thompson, M. C., & Pecina, J. L. (2025) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2024.0589 · Cited by 2
Health Sync: An Integrated Healthcare Management Platform
Sathya, A., Tarunes, K., Sriram, G. J., Naresh, K. N., Narendran, M. R., & S, K. V. V. (2025) · International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV) · doi:10.1109/icicv64824.2025.11085743 · Cited by 1
Evaluating the use of red flags by online symptom checkers
Sutaria, S., Devakumar, D., Mallinson, P. A. C., Kinra, S., Malak, T., & Meczner, A. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13353-w · Cited by 1
Kopka, M., & Feufel, M. A. (2026) · JMIR Human Factors · doi:10.2196/76168 · Cited by 1
Xu, Y., Prentice, C., Rueda, S. T., Meczner, A., Multmeier, J., Wickham, A. P., Kelly, L., Klepchukova, A., Stsefanovich, H., & Zhaunova, L. (2026) · npj Digital Medicine · doi:10.1038/s41746-025-02332-4 · Cited by 3
AI applications in veterinary digital health: a systematic survey
Polapragada, S. K. S. S. (2026) · Frontiers in Veterinary Science · doi:10.3389/fvets.2026.1853395 · Cited by 2
Vuillaume, L. A., Carval, G., Grajoszex, M., Ouamara, N., & Salvestrini, J. (2022) · Annales françaises de médecine d’urgence · doi:10.3166/afmu-2022-0382 · Cited by 1
Eden, D. J., Hermann, A. D., Sombrotto, L. B., Wilner, P. J., & Chen, J. A. (2024) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000401 · Cited by 1
Accuracy of Dental Symptom Checker Web Application in the Singapore Military Population
Lim, S. N., Woon, X. R., Goh, C. E., Ng, J. C., Ang, S. Y., Lim, T. J., & Allen, P. (2024) · International Dental Journal · doi:10.1016/j.identj.2024.07.006 · Cited by 1
North, F., Buss, R. J., Nelson, E. M., Thompson, M. C., Pecina, J. L., & Crum, B. A. (2024) · Health Services Research and Managerial Epidemiology · doi:10.1177/23333928241271933 · Cited by 1
Comprehensive Mobile Solution for Personalized Healthcare Services using Enhanced AI
Kareem, M. S., Manikandan, B., Selvaraj, K., Anju, A., Narendran, M. R., & S, K. V. V. (2025) · International Conference on Data Science and Business Systems (ICDSBS) · doi:10.1109/icdsbs63635.2025.11031900 · Cited by 1
Wesinger, A., Riedl, E., Kittler, H., & Tschandl, P. (2022) · Dermatology Practical & Conceptual · doi:10.5826/dpc.1203a117 · Cited by 1
Study of the User Behaviour Caused by Automatic Recommendation Systems Call to Action
Kopanitsa, G., & Kovalchuk, S. V. (2022) · Studies in health technology and informatics · doi:10.3233/shti220966 · Cited by 1
Painter, A., Hayhoe, B. W. J., Sasco, E. R., & El-Osta, A. A. R. (2022) · doi:10.2196/preprints.42481 · Cited by 1
Said, N., Schumacher, L., & Huff, M. (2023) · doi:10.31234/osf.io/m5q3p · Cited by 1
Painter, A., & Hayhoe, B. W. J. (2024) · doi:10.2196/preprints.63423 · Cited by 1
El-Osta, A. A. R., Kopka, M., Altalib, S., Neves, A. L., Faisal, A. A., POTTS, H. W. W., & Spinelli, G. (2026) · Research Square · doi:10.21203/rs.3.rs-9004371/v1 · Cited by 1
Painter, A., Hayhoe, B. W. J., & El-Osta, A. A. R. (2022) · doi:10.2196/preprints.37408
Fraser, H., Cohan, G., Koehler, C., Anderson, J. S., Lawrence, A., Patena, J. V., Bacher, I., & Ranney, M. L. (2022) · doi:10.2196/preprints.38364
Vuillaume, L. A., Turpinier, J., Cipolat, L., Depil-Duval, A., Dumontier, T., Peschanski, N., Kieffer, Y., Barbat, B., Riquier, T., Dinot, V., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-1580704/v1
Assessing Data Gathering Quality of Chatbot Based Symptom Checkers - a Clinical Vignettes StudyPreprint
Ben‐Shabat, N., Sharvit, G., Meimis, B., Joya, D. B., Sloma, A., Kiderman, D., Shabat, A., Tsur, A. M., Watad, ., & Amital, H. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4128799
Sampietro-Colom, L., Fernández-Barceló, C., Abbas, I., Valdasquin, B., Rabasseda, N., García‐Lorenzo, B., Sánchez, M., Sans-Corrales, M., García, N., & Granados, A. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192113873
Metadata
- Authors
- Ceney, A., Tolond, S., Glowinski, A., Marks, B., Swift, S., & Palser, T.
- Affiliations
- University of Exeter (GB)
- University of Leicester (GB)
- Corresponding author
- Ceney, Adam
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2021-07-15
- Volume
- 16
- Issue
- 7
- Pages
- e0254088
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC8282353
- OpenAlex
- W3182482959
- Cited by
- 125
- References
- 18
- Citations by year
- 2026: 14 · 2025: 19 · 2024: 25 · 2023: 29 · 2022: 33 · 2021: 5
- FWCI
- 6.70
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- medRxiv (acceptedVersion)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute for Health and Care Research (AI_AWARD02290)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- online symptom checkers, diagnostic accuracy, triage, health care utilization, clinical vignettes, guideline adherence, external validation, emergency care
- MeSH terms
- Humans, Patient Acceptance of Health Care, Quality of Health Care, Self Care, Triage, Symptom Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
