Computer Decision Support as a Source of Interpretation Error: The Case of Electrocardiograms
Open accessAbstract
OBJECTIVE: The aim of this study was to determine the effect that the computer interpretation (CI) of electrocardiograms (EKGs) has on the accuracy of resident (noncardiologist) physicians reading EKGs. DESIGN: A randomized, controlled trial was conducted in a laboratory setting from February through June 2001, using a two-period crossover design with matched pairs of subjects randomly assigned to sequencing groups. MEASUREMENTS: Subjects' interpretive accuracy of discrete, cardiologist-determined EKG findings were measured as judged by a board-certified internist. RESULTS: Without the CI, subjects interpreted 48.9% (95% confidence interval, 45.0% to 52.8%) of the findings correctly. With the CI, subjects interpreted 55.4% (51.9% to 58.9%) correctly (p < 0.0001). When the CIs that agreed with the gold standard (Correct CIs) were not included, 53.1% (47.7% to 58.5%) of the findings were interpreted correctly. When the correct CI was included, accuracy increased to 68.1% (63.2% to 72.7%; p < 0.0001). When computer advice that did not agree with the gold standard (Incorrect CI) was not provided to the subjects, 56.7% (48.5% to 64.5%) of findings were interpreted correctly. Accuracy dropped to 48.3% (40.4% to 56.4%) when the incorrect computer advice was provided (p = 0.131). Subjects erroneously agreed with the incorrect CI more often when it was presented with the EKG 67.7% (57.2% to 76.7%) than when it was not 34.6% (23.8% to 47.3%; p < 0.0001). CONCLUSIONS: Computer decision support systems can generally improve the interpretive accuracy of internal medicine residents in reading EKGs. However, subjects were influenced significantly by incorrect advice, which tempers the overall usefulness of computer-generated advice in this and perhaps other areas.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Another explanation for increased anxiety might be that participants worry about physicians' automation bias.
“Another explanation might be that participants worry about physicians’ automation bias, 48 that is, physicians might blindly rely on the AI-enabled system's suggestions without critically reviewing them resulting in reduced accuracy 49 and increased medical errors. 50”
From Patient perspectives on informed consent for medical AI: A web-based experiment (2024) · cites this paper as [49]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650 · see citing works
(2005) · Springer eBooks · doi:10.1007/0-387-27255-0_9 · Cited by 20,333 · see citing works
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550 · see citing works
Institute of Medicine (2000) · National Academies Press eBooks · doi:10.17226/9728 · Cited by 14,174 · see citing works
Judgment under Uncertainty: Heuristics and Biases
(1982) · doi:10.1017/cbo9780511809477 · Cited by 8,240 · see citing works
Wall, A. (2000) · British Journal of Healthcare Management · doi:10.12968/bjhc.2000.6.9.19311 · Cited by 5,909 · see citing works
The Design and Analysis of Clinical Experiments
Fleiss, J. L. (1999) · Wiley eBooks · doi:10.1002/9781118032923 · Cited by 4,819 · see citing works
Applied Linear Regression Models
(1997) · Technometrics · doi:10.1080/00401706.1997.10485142 · Cited by 4,638 · see citing works
Judgment under uncertainty: List of contributors
Kahneman, D. J., Slovic, P., & Tversky, A. (1982) · Cited by 4,630 · see citing works
Judgment under uncertainty: Causality and attribution
Kahneman, D. J., Slovic, P., & Tversky, A. (1982) · Cited by 4,630 · see citing works
Judgement under Uncertainty: Heuristics and Biases
Hirschman, E. C., Kahneman, D. J., Slovic, P., & Tversky, A. (1983) · Journal of Marketing Research · doi:10.2307/3151689 · Cited by 3,813 · see citing works
To Err is Human: Building a Safer Health System in 1999.
Stefl, M. E. (2001) · PubMed · Cited by 1,738 · see citing works
Judgement under Uncertainty: Heuristics and Biases
Kahneman, D. J., Slovic, P., & Tversky, A. (1985) · The British Journal for the Philosophy of Science · Cited by 1,362 · see citing works
Statistics with Confidence: Confidence Intervals and Statistical Guidelines.
JOLLIFFE, F. R., Gardner, M. J., & Altman, D. G. (1990) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982808 · Cited by 1,319 · see citing works
Information Systems: A Management Perspective
Alter, S. (1991) · Cited by 540 · see citing works
The Diagnostic Performance of Computer Programs for the Interpretation of Electrocardiograms
Willems, J. L., Abreu‐Lima, C., Arnaud, P., van Bemmel, J. H., Brohet, C., Degani, R., Denis, B., Gehring, J., Graham, I. M., van Herpen, G., et al. (1991) · New England Journal of Medicine · doi:10.1056/nejm199112193252503 · Cited by 538 · see citing works
Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation
Friedman, C. P., Elstein, A. S., Wolf, F. M., Murphy, G. C., Franz, T. M., Heckerling, P. S., Fine, P. L., Miller, T. M., & Abraham, V. (1999) · JAMA · doi:10.1001/jama.282.19.1851 · Cited by 219 · see citing works
Impact of a clinical scenario on accuracy of electrocardiogram interpretation
Hatala, R. M., Norman, G. R., & Brooks, L. R. (1999) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1999.00298.x · Cited by 168 · see citing works
The impact of Computer-assisted Test Interpretation on Physician Decision Making
Brailer, D. J., Kroch, E. A., & Pauly, M. V. (1997) · Medical Decision Making · doi:10.1177/0272989x9701700109 · Cited by 49 · see citing works
Hillson, S. D., Connelly, D. P., & Liu, Y. (1995) · Medical Decision Making · doi:10.1177/0272989x9501500202 · Cited by 47 · see citing works
Goodacre, S. W., Webster, A. P., & Morris, F. (2001) · Postgraduate Medical Journal · doi:10.1136/pmj.77.909.455 · Cited by 45 · see citing works
Electrocardiographic Training in Primary Care Residency Programs
Pinkerton, R. E. (1981) · JAMA · doi:10.1001/jama.1981.03320020040021 · Cited by 34 · see citing works
The effect of clinical history on physiciansʼ ECG interpretation skills
Hatala, R. A., Norman, G. R., & Brooks, L. R. (1996) · Academic Medicine · doi:10.1097/00001888-199610000-00047 · Cited by 32 · see citing works
Computerized Electrocardiography — An Adjunct to the Physician
Laks, M. M., & Selvester, R. H. S. (1991) · New England Journal of Medicine · doi:10.1056/nejm199112193252509 · Cited by 23 · see citing works
A Brief History of Computer-Assisted Electrocardiography
Macfarlane, P. W. (1990) · Methods of Information in Medicine · doi:10.1055/s-0038-1634791 · Cited by 22 · see citing works
[Computer-based interpretation of ECG--guiding or misleading?].
Morisbak, B., & Gjesdal, K. (1999) · PubMed · Cited by 4 · see citing works
Ww, P., & Dw, B. (1996) · PubMed · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Ash, J. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1471 · Cited by 1,789 · see citing works
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · see citing works
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · see citing works
Unintended consequences of machine learning in medicine
Cabitza, F., Rasoini, R., & Gensini, G. F. (2017) · JAMA · doi:10.1001/jama.2017.7797 · Cited by 1,066 · see citing works
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
Overconfidence as a Cause of Diagnostic Error in Medicine
Berner, E. S., & Graber, M. L. (2008) · The American Journal of Medicine · doi:10.1016/j.amjmed.2008.01.001 · Cited by 1,034 · see citing works
Automation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451 · see citing works
Kaminski, M. F., Hassan, C., Bisschops, R., Pohl, J., Pellisé, M., Dekker, E., Ignjatovic-Wilson, A., Hoffman, A. W., Longcroft‐Wheaton, G., Heresbach, D., et al. (2014) · Endoscopy · doi:10.1055/s-0034-1365348 · Cited by 430 · see citing works
Jussupow, E., Spohrer, K., Heinzl, A., & Gawlitza, J. (2021) · Information Systems Research · doi:10.1287/isre.2020.0980 · Cited by 422 · see citing works
(2013) · Europe PMC (PubMed Central) · Cited by 396 · see citing works
Computer-Interpreted Electrocardiograms
Schläpfer, J., & Wellens, H. J. J. (2017) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2017.07.723 · Cited by 379 · see citing works
Bisschops, R., East, J. E., Hassan, C., Hazewinkel, Y., Kaminski, M. F., Neumann, H., Pellisé, M., Antonelli, G., Bustamante, M., Coron, E., et al. (2019) · Endoscopy · doi:10.1055/a-1031-7657 · Cited by 344 · see citing works
The Research We Still Are Not Doing: An Agenda for the Study of Computer-Based Learning
Cook, D. A. (2005) · Academic Medicine · doi:10.1097/00001888-200506000-00005 · Cited by 289 · see citing works
Promberger, M., & Baron, J. (2006) · Journal of Behavioral Decision Making · doi:10.1002/bdm.542 · Cited by 242 · see citing works
Algorithmic Management: Bright and Dark Sides, Practical Implications, and Research Opportunities
Benlian, A., Wiener, M., Cram, W. A., Krasnova, H., Maedche, A., Möhlmann, M., Recker, J., & Remus, U. (2022) · Business & Information Systems Engineering · doi:10.1007/s12599-022-00764-w · Cited by 180 · see citing works
Overview of Clinical Decision Support Systems
Berner, E. S., & Lande, T. J. L. (2016) · Health informatics · doi:10.1007/978-3-319-31913-1_1 · Cited by 173 · see citing works
General Practitioners’ Attitudes Toward Artificial Intelligence–Enabled Systems: Interview Study
Buck, C., Doctor, E., Hennrich, J., Jöhnk, J., & Eymann, T. (2022) · Journal of Medical Internet Research · doi:10.2196/28916 · Cited by 147 · see citing works
The four dimensions of contestable AI diagnostics - A patient-centric approach to explainable AI
Ploug, T., & Holm, S. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101901 · Cited by 152 · see citing works
Ethics of Artificial Intelligence in Medicine and Ophthalmology
Abdullah, Y. I., Schuman, J. S., Shabsigh, R., Caplan, A. L., & Al‐Aswad, L. A. (2021) · Asia-Pacific Journal of Ophthalmology · doi:10.1097/apo.0000000000000397 · Cited by 159 · see citing works
Automation bias: Empirical results assessing influencing factors
Goddard, K., Roudsari, A., & Wyatt, J. C. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.01.001 · Cited by 163 · see citing works
Human Factors Challenges for the Safe Use of Artificial Intelligence in Patient Care
Sujan, M., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I., & Reynolds, N. (2019) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100081 · Cited by 144 · see citing works
Quality assurance of computer-aided detection and diagnosis in colonoscopy
Vinsard, D. G., Mori, Y., Misawa, M., Kudo, S., Rastogi, A. N., Bağcı, U., Rex, D. K., & Wallace, M. B. (2019) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2019.03.019 · Cited by 141 · see citing works
Abdelwanis, M., Alarafati, H. K., Tammam, M. M. S., & Simsekler, M. C. E. (2024) · Journal of Safety Science and Resilience · doi:10.1016/j.jnlssr.2024.06.001 · Cited by 137 · see citing works
Natali, C., Marconi, L., Dias Duran, L. D., & Cabitza, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11352-1 · Cited by 143 · see citing works
Part 9: Acute Coronary Syndromes
O’Connor, R. E., Ali, A. S., Brady, W. J., Ghaemmaghami, C. A., Menon, V., Welsford, M., & Shuster, M. (2015) · Circulation · doi:10.1161/cir.0000000000000263 · Cited by 126 · see citing works
Common errors in computer electrocardiogram interpretation
Guglin, M. E., & Thatai, D. (2005) · International Journal of Cardiology · doi:10.1016/j.ijcard.2005.02.007 · Cited by 122 · see citing works
Chugunova, M., & Sele, D. (2022) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2022.101897 · Cited by 118 · see citing works
Part 9: Acute Coronary Syndromes
O’Connor, R. E., Bossaert, L. L., Arntz, H.-R., Brooks, S. C., Diercks, D., Feitosa-Filho, G. S., Nolan, J. P., Hoek, T. L. V., Walters, D., Wong, A., et al. (2010) · Circulation · doi:10.1161/circulationaha.110.985549 · Cited by 116 · see citing works
McKibbon, K. A., & Fridsma, D. B. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2087 · Cited by 112 · see citing works
Self-citationResponsibility beyond design: Physicians’ requirements for ethical medical AI
Sand, M., Durán, J. M., & Jongsma, K. R. (2021) · Bioethics · doi:10.1111/bioe.12887 · Cited by 109 · see citing works
Machine learning in medicine: what clinicians should know
Sim, J. Z. T., Fong, Q. W., Huang, W., & Tan, C. H. (2021) · Singapore Medical Journal · doi:10.11622/smedj.2021054 · Cited by 99 · see citing works
Patient perspectives on informed consent for medical AI: A web-based experiment
Park, H. J. (2024) · DIGITAL HEALTH · doi:10.1177/20552076241247938 · Cited by 95 · see citing works
Harrington, L., Kennerly, D., & Johnson, C. (2011) · Journal of Healthcare Management · doi:10.1097/00115514-201101000-00006 · Cited by 93 · see citing works
Strategies to reduce diagnostic errors: a systematic review
Abimanyi-Ochom, J., Mudiyanselage, S. B., Catchpool, M., Firipis, M., Dona, S. W. A., & Watts, J. J. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0901-1 · Cited by 92 · see citing works
Automated prediction of mastitis infection patterns in dairy herds using machine learning
Hyde, R. M., Down, P. M., Bradley, A. J., Breen, J. E., Hudson, C. D., Leach, K. A., & Green, M. J. (2020) · Scientific Reports · doi:10.1038/s41598-020-61126-8 · Cited by 90 · see citing works
Armstrong, G. W., & Lorch, A. (2019) · International Ophthalmology Clinics · doi:10.1097/iio.0000000000000298 · Cited by 85 · see citing works
MINA: Multilevel Knowledge-Guided Attention for Modeling Electrocardiography Signals
Hong, S., Xiao, C., Ma, T., Li, H., & Sun, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/816 · Cited by 76 · see citing works
Electronic Medical Records and Their Impact on Resident and Medical Student Education
Keenan, C. R., Nguyen, H., & Srinivasan, M. (2006) · Academic Psychiatry · doi:10.1176/appi.ap.30.6.522 · Cited by 69 · see citing works
Rubinstein, J., Dhoble, A., & Ferenchick, G. S. (2009) · BMC Medical Education · doi:10.1186/1472-6920-9-4 · Cited by 68 · see citing works
Educational Impact of the Electronic Medical Record
Schenarts, P. J., & Schenarts, K. D. (2011) · Journal of surgical education · doi:10.1016/j.jsurg.2011.10.008 · Cited by 65 · see citing works
Identifying QT prolongation from ECG impressions using a general-purpose Natural Language Processor
Denny, J. C., Miller, R. A., Waitman, L. R., Arrieta, M. A., & Peterson, J. F. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.09.001 · Cited by 63 · see citing works
Felmingham, C. M., Adler, N. R., Ge, Z., Morton, R. L., Janda, M., & Mar, V. J. (2021) · American Journal of Clinical Dermatology · doi:10.1007/s40257-020-00574-4 · Cited by 57 · see citing works
Expectations for Artificial Intelligence (AI) in Psychiatry
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., Achtyes, E. D., & Bauer, M. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01378-5 · Cited by 56 · see citing works
Machine Learning na Medicina: Revisão e Aplicabilidade
Paixao, G. M., Santos, B. C., de Araujo, R. M., Ribeiro, M. H., de Moraes, J. L., & Ribeiro, A. L. P. (2022) · Arquivos Brasileiros de Cardiologia · doi:10.36660/abc.20200596 · Cited by 66 · see citing works
Computer Analysis of the Electrocardiogram
Kors, J. A., & van Herpen, G. (2010) · Springer eBooks · doi:10.1007/978-1-84882-046-3_37 · Cited by 53 · see citing works
Prolonged QT Interval Diagnosis Suppression by a Widely Used Computerized ECG Analysis System
Garg, A., & Lehmann, M. H. (2012) · Circulation Arrhythmia and Electrophysiology · doi:10.1161/circep.112.976803 · Cited by 51 · see citing works
Bae, M. H., Lee, J. H., Yang, D. H., Park, H. S., Cho, Y., Chae, S. C., & Jun, J. (2012) · Clinical Cardiology · doi:10.1002/clc.22000 · Cited by 49 · see citing works
Golden Standard or Obsolete Method? Review of ECG Applications in Clinical and Experimental Context
Stračina, T., Ronzhina, M., Ředina, R., & Novàkovâ, M. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.867033 · Cited by 48 · see citing works
Diagnostic Reasoning and Decision Making in the Context of Health Information Technology
Patel, V. L., Kaufman, D. R., & Kannampallil, T. (2013) · Reviews of Human Factors and Ergonomics · doi:10.1177/1557234x13492978 · Cited by 45 · see citing works
Clinical decision support systems: Potential with pitfalls
Eberhardt, J. S., Bilchik, A. J., & Stojadinovic, A. (2012) · Journal of Surgical Oncology · doi:10.1002/jso.23053 · Cited by 43 · see citing works
Part 9: Acute coronary syndromes
Bossaert, L. L., O’Connor, R. E., Arntz, H.-R., Brooks, S. C., Diercks, D., Feitosa-Filho, G. S., Nolan, J. P., Hoek, T. L. V., Walters, D., Wong, A., et al. (2010) · Resuscitation · doi:10.1016/j.resuscitation.2010.09.001 · Cited by 42 · see citing works
Li, L. T., Haley, L. C., Boyd, A. K., & Bernstam, E. V. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104531 · Cited by 45 · see citing works
Koshy, R. (2005) · Transfusion · doi:10.1111/j.1537-2995.2005.00619.x · Cited by 39 · see citing works
Unintended consequences of machine learning in medicine?
McDonald, L., Ramagopalan, S. V., Cox, A. P., & Oguz, M. (2017) · F1000Research · doi:10.12688/f1000research.12693.1 · Cited by 38 · see citing works
Diagnostic Decision Support Systems: How to Determine the Gold Standard?
Berner, E. S. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1416 · Cited by 34 · see citing works
Erroneous data: The Achilles' heel of AI and personalized medicine
Kristiansen, T. B., Kristensen, K., Uffelmann, J., & Brandslund, I. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.862095 · Cited by 33 · see citing works
Identifying UMLS concepts from ECG Impressions using KnowledgeMap.
Denny, J. C., Spickard, A., Miller, R. A., Schildcrout, J. S., Darbar, D., Rosenbloom, S. T., & Peterson, J. F. (2005) · Europe PMC (PubMed Central) · Cited by 33 · see citing works
Ameen, S., Wong, M. C., Yee, K., & Turner, P. A. (2022) · Applied Sciences · doi:10.3390/app12073341 · Cited by 30 · see citing works
Cairns, A. W., Bond, R., Finlay, D., Breen, C. J., Guldenring, D., Gaffney, R., Gallagher, A. G., Peace, A. J., & Henn, P. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.09.016 · Cited by 30 · see citing works
Diagnostic Decision Support Systems
Miller, R. A. (2016) · Health informatics · doi:10.1007/978-3-319-31913-1_11 · Cited by 30 · see citing works
We and It: An Interdisciplinary Review of the Experimental Evidence on Human-Machine InteractionPreprint
Chugunova, M., & Sele, D. (2020) · SSRN Electronic Journal · doi:10.3929/ethz-b-000442053 · Cited by 30 · see citing works
Lindow, T., Kron, J., Thulesius, H. O., Ljungström, E., & Pahlm, O. (2019) · Scandinavian Journal of Primary Health Care · doi:10.1080/02813432.2019.1684429 · Cited by 28 · see citing works
To err is human: Exploratory multilevel analysis of supply chain delivery delays
Awaysheh, A., Frohlich, M. T., Flynn, B. B., & Flynn, P. J. (2021) · Journal of Operations Management · doi:10.1002/joom.1154 · Cited by 28 · see citing works
The Effect of Erroneous Computer Interpretation of ECGs on Resident Decision Making
Southern, W. N., & Arnsten, J. H. (2009) · Medical Decision Making · doi:10.1177/0272989x09333125 · Cited by 28 · see citing works
Biases Affecting Human Decision Making in AI-Supported Second Opinion Settings
Cabitza, F. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26773-5_25 · Cited by 26 · see citing works
Novotný, T., Bond, R., Andršová, I., Koc, L., Šišáková, M., Finlay, D., Guldenring, D., Špinar, J., & Malík, M. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.02.007 · Cited by 25 · see citing works
Biesen, W. V., Cauwenberge, D. V., Decruyenaere, J. M. A., Leune, T., & Sterckx, S. (2022) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-022-01933-3 · Cited by 24 · see citing works
Overreliance on computerized algorithms to interpret electrocardiograms
Hongo, R. H., & Goldschlager, N. (2004) · The American Journal of Medicine · doi:10.1016/j.amjmed.2004.08.006 · Cited by 24 · see citing works
Evaluating the Impact of Uncertainty Visualization on Model Reliance
Zhao, J. L., Wang, Y., Mancenido, M. V., Chiou, E. K., & Maciejewski, R. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3251950 · Cited by 26 · see citing works
Computer-based rhythm diagnosis and its possible influence on nonexpert electrocardiogram readers
Hakacova, N., Trägårdh, E., Wagner, G. S., Maynard, C. C., & Pahlm, O. (2012) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2011.05.007 · Cited by 23 · see citing works
Use of Diagnostic Decision Support Systems in Medical Education
McGowan, J. J., & Berner, E. S. (2010) · Methods of Information in Medicine · doi:10.3414/me9309 · Cited by 22 · see citing works
Artificial Intelligence in Ophthalmology – Status Quo and Future Perspectives
Matos, P. A. W., Reimer, R. P., Rokohl, A. C., Caldeira, L., Heindl, L. M., & Hokamp, N. G. (2022) · Seminars in Ophthalmology · doi:10.1080/08820538.2022.2139625 · Cited by 24 · see citing works
Ethical and Legal Issues in the Use of Health Information Technology to Improve Patient Safety
Berner, E. S. (2008) · HEC Forum · doi:10.1007/s10730-008-9074-5 · Cited by 22 · see citing works
Ploug, T., & Holm, S. (2023) · Bioethics · doi:10.1111/bioe.13124 · Cited by 20 · see citing works
Differential diagnosis checklists reduce diagnostic error differentially: A randomised experiment
Kämmer, J. E., Schauber, S. K., Hautz, S. C., Stroben, F., & Hautz, W. E. (2021) · Medical Education · doi:10.1111/medu.14596 · Cited by 20 · see citing works
Computer-aided diagnosis system outperforms scoring analysis in myocardial perfusion imaging
Johansson, L., Edenbrandt, L., Nakajima, K., Lomsky, M., Svensson, S.-E., & Trägårdh, E. (2014) · Journal of Nuclear Cardiology · doi:10.1007/s12350-013-9815-y · Cited by 18 · see citing works
Cairns, A. W., Bond, R., Finlay, D., Guldenring, D., Badilini, F., Libretti, G., Peace, A. J., & Leslie, S. J. (2017) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2017.08.007 · Cited by 18 · see citing works
Computer Analysis of the Electrocardiogram
Kors, J. A., & van Herpen, G. (2011) · Springer eBooks · doi:10.1007/978-0-85729-880-5_5 · Cited by 18 · see citing works
The most common errors in automatic ECG interpretation
Kraik, K., Dykiert, I. A., Niewiadomska, J., Ziemer, M., Mikołajczak, K., Kreń, M., Kukiełka, P., Martuszewski, A., Harych, T., Poręba, R., et al. (2025) · Frontiers in Physiology · doi:10.3389/fphys.2025.1590170 · Cited by 17 · see citing works
The automation of doctors and machines: A classification for AI in medicine (ADAM framework)
Kazzazi, F. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2020-0189 · Cited by 17 · see citing works
Wilson, R. E., Kado, H. S., PERCY, R. F., Butterfield, R. C., Sabato, J., Strom, J. A., & Box, L. C. (2013) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2013.04.013 · Cited by 17 · see citing works
Prognostic Implications of Q Waves and T-Wave Inversion Associated With Early Repolarization
Uberoi, A., Sallam, K., Perez, M. V., Jain, N., Ashley, E. A., & Froelicher, V. F. (2012) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2012.04.009 · Cited by 16 · see citing works
Overcoming barriers to implementation of artificial intelligence in gastroenterology
Sutton, R. A., & Sharma, P. (2021) · Best Practice & Research Clinical Gastroenterology · doi:10.1016/j.bpg.2021.101732 · Cited by 15 · see citing works
The Physician and the Laboratory
Graber, M. L. (2006) · Pathology Patterns Reviews · doi:10.1309/54xr770u8wtegg1h · Cited by 16 · see citing works
Benefits and Risks of Machine Learning Decision Support Systems
Licitra, L. F., Trama, A., & Hosni, H. (2017) · JAMA · doi:10.1001/jama.2017.16627 · Cited by 15 · see citing works
Knoery, C., Bond, R., Iftikhar, A., Rjoob, K. W. Y., McGilligan, V., Peace, A. J., Heaton, J., & Leslie, S. J. (2019) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2019.08.006 · Cited by 14 · see citing works
Willcox, M. E., Compton, S. J., & Bardy, G. H. (2021) · Heart Rhythm O2 · doi:10.1016/j.hroo.2021.09.008 · Cited by 14 · see citing works
Electrocardiograms with pacemakers: accuracy of computer reading
Guglin, M. E., & Datwani, N. (2006) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2006.07.001 · Cited by 14 · see citing works
Diagnostic Decision Support Systems
Miller, R. A., & Geissbühler, A. (2007) · Springer eBooks · doi:10.1007/978-0-387-38319-4_5 · Cited by 14 · see citing works
An Objective Study of the Impact of the Electronic Medical Record on Outcomes in Trauma Patients
Schenarts, P. J., Goettler, C. E., White, M. A., & Waibel, B. H. (2012) · The American Surgeon · doi:10.1177/000313481207801134 · Cited by 13 · see citing works
Bentzen, S. M. (2025) · Clinical Oncology · doi:10.1016/j.clon.2025.103798 · Cited by 12 · see citing works
The New ISO/IEC Standard for Automated ECG Interpretation
Young, B., & Schmid, J.-J. (2021) · Hearts · doi:10.3390/hearts2030032 · Cited by 12 · see citing works
Impact of Computer-Interpreted ECGs on the Accuracy of Healthcare Professionals
Kashou, A. H., Noseworthy, P. A., Beckman, T. J., Anavekar, N. S., Cullen, M. W., Angstman, K. B., Sandefur, B. J., Shapiro, B. P., Wiley, B. M., Kates, A. M., et al. (2023) · Current Problems in Cardiology · doi:10.1016/j.cpcardiol.2023.101989 · Cited by 12 · see citing works
Pilbery, R., Teare, M. D., Goodacre, S. W., & Morris, F. A. (2016) · Emergency Medicine Journal · doi:10.1136/emermed-2015-204988 · Cited by 12 · see citing works
The influence of context on diagnostic reasoning: A narrative synthesis of experimental findings
Schmidt, H. G., Norman, G. R., Mamede, S., & Magzoub, M. E. (2024) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.14023 · Cited by 12 · see citing works
Diagnostic Reasoning and Expertise in Health Care
Patel, V. L., Kaufman, D. R., & Kannampallil, T. (2019) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198795872.013.27 · Cited by 11 · see citing works
Dunnmon, J. (2021) · Radiologic Clinics of North America · doi:10.1016/j.rcl.2021.07.006 · Cited by 10 · see citing works
Natali, C., Marconi, L., Duran, L. D. D., Miglioretti, M., & Cabitza, F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5166364 · Cited by 10 · see citing works
Kim, R. Y. (2024) · Cancer Biomarkers · doi:10.3233/cbm-230360 · Cited by 9 · see citing works
Roser, D., Meinikheim, M., Muzalyova, A., Mendel, R., Palm, C., Probst, A., Nagl, S., Scheppach, M. W., Römmele, C., Schnoy, E., et al. (2025) · DEN Open · doi:10.1002/deo2.70150 · Cited by 9 · see citing works
- Authors
- Tsai, T. L., Fridsma, D. B., & Gatti, G.
- Affiliations
- University of Pittsburgh (US)
- Corresponding author
- Tsai, Theodore L.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2003-09
- Volume
- 10
- Issue
- 5
- Pages
- 478-483
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC212785
- OpenAlex
- W1869541995
- Cited by
- 161
- References
- 27
- Citations by year
- 2026: 13 · 2025: 8 · 2024: 10 · 2023: 4 · 2022: 20 · 2021: 17 · 2020: 7 · 2019: 12 · 2018: 1 · 2017: 9 · 2016: 7 · 2015: 3 · 2014: 3 · 2013: 5 · 2012: 8
- FWCI
- 4.79
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. National Library of Medicine (5 T15 LM/DE07059)
- Topics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- computer decision support, ECG interpretation, diagnostic accuracy, internal medicine residents, clinical judgment, electrocardiographic findings
- MeSH terms
- Diagnosis, Computer-Assisted, Diagnostic Errors, Electrocardiography, Humans, Internship and Residency, Observer Variation, Cross-Over Studies, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
