The Clinician and Dataset Shift in Artificial Intelligence
Open accessAbstract
Dataset Shift in Clinical Trials This letter outlines how to identify, and potentially mitigate, common sources of “dataset shift” in machine-learning systems. This occurs when the model “training ...
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
2 of 7 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured
- Premisecitation
A major driver of AI system malfunction is dataset shift.
“A major driver of AI system malfunction is known as “dataset shift.”4,5”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [4] Subbaswamy A, Saria S. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics 2020; 21: 345– 52. 31742354 source
- cites [5] Saria S, Subbaswamy A. Tutorial: safe and reliable machine learning. In: Proceedings of the conference on fairness, accountability, and transparency, January 29–31, 2019. New York: Association for Computing Machinery, 2019. DOI: 10.1056/NEJMc2104626
Not checked in the free run.
- Rests on an assumption: Most clinical AI systems today use machine learning.
- Interpretationreasoning
In the University of Michigan sepsis-alerting model case, dataset shift fundamentally altered the relationship between fevers and bacterial sepsis.
“This was a case in which dataset shift fundamentally altered the relationship between fevers and bacterial sepsis, leading the hospital’s clinical AI governing committee (which one of the authors of this letter chairs) to decommission its use.”
Quote in the paperNot a substantial claim- Rests on an assumption: In April 2020, the University of Michigan Hospital deactivated the Epic sepsis-alerting model because of spurious alerting owing to changes in patients’ demographic characteristics associated with the COVID-19 pandemic.
- Premisecitation
Artificial intelligence systems are now regularly being used in medical settings.
“Artificial intelligence (AI) systems are now regularly being used in medical settings,1”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med 2019; 380: 1347– 58. 30943338 source
- Classified as probably not a substantial claim. 3
- May combine two claims. 2
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
InterpretationcitationSuccessful recognition and mitigation of dataset shift require both vigilant clinicians and sound technical oversight through AI governance teams.
“Successful recognition and mitigation of dataset shift require both vigilant clinicians and sound technical oversight through AI governance teams.4,5”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [4] Subbaswamy A, Saria S. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics 2020; 21: 345– 52. 31742354 source
Section 4, “Open Challenges and Looking Forward” (page not supplied), calls for monitoring to detect shifts, maintenance to address them, and collaboration with domain experts when constructing models. It does not say that vigilant clinicians and AI governance teams are both required for this work. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [5] Saria S, Subbaswamy A. Tutorial: safe and reliable machine learning. In: Proceedings of the conference on fairness, accountability, and transparency, January 29–31, 2019. New York: Association for Computing Machinery, 2019. DOI: 10.1056/NEJMc2104626
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenCommon causes of dataset shift can be grouped into changes in technology, changes in population and setting, and changes in behavior.
“we group into changes in technology (e.g., software vendors), changes in population and setting (e.g., new demographics), and changes in behavior (e.g., new reimbursement incentives)”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationRegulatory oversight of AI in medical settings is inconsistent and undeveloped.
“although regulatory oversight is inconsistent and undeveloped.2,3”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [2] Ross C Hospitals are using AI to predict the decline of Covid-19 patients — before knowing it works. Stat 4 24, 2020 (https://www.statnews.com/2020/04/24/coronavirus-hospitals-use-ai-to-predict-patient-decline-before-knowing-it-works/).
Not checked in the free run.
- cites [3] Artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD) action plan Silver Spring, MD: Food and Drug Administration, January 2021 (https://www.fdagov/media/145022/download).
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningSafe deployment of clinical AI requires informed clinician-users.
“Safe deployment of clinical AI requires informed clinician-users”
Quote in the paperNot a substantial claim- Rests on an assumption: Clinicians are generally responsible for identifying and reporting emerging problems with clinical AI.
The paper relies on this without support.
Clinicians are generally responsible for identifying and reporting emerging problems with clinical AI.
“who are generally responsible for identifying and reporting emerging problems.”
The paper relies on this without support.
Most clinical AI systems today use machine learning.
“Most clinical AI systems today use machine learning, algorithms that leverage statistical methods to learn key patterns from clinical data.”
The paper relies on this without support.
In April 2020, the University of Michigan Hospital deactivated the Epic sepsis-alerting model because of spurious alerting owing to changes in patients’ demographic characteristics associated with the COVID-19 pandemic.
“the University of Michigan Hospital implemented the widely used sepsis-alerting model developed by Epic Systems; in April 2020, the model had to be deactivated because of spurious alerting owing to changes in patients’ demographic characteristics associated with the coronavirus disease 2019 pandemic.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Variations in patient demographics, clinical practices, and data collection procedures can lead to inaccurate predictions when predictive models are deployed across different hospitals or healthcare systems.
“This quality becomes crucial when predictive models are deployed across different hospitals or healthcare systems, where variations in patient demographics, clinical practices, and data collection procedures can lead to inaccurate predictions [17, 18].”
From Toward generalizable and interpretable machine learning models in healthcare: Insights from ICU outcome predictions (2026) · cites this paper as [18]
In “To the Editor” (page unavailable), the paper says dataset shift can make a clinical AI system underperform when deployment data differ from development data. It identifies changing patient demographics and clinical behavior or practice as potential causes, and describes a hospital sepsis model producing spurious alerts. Our copy does not specifically identify differences in data-collection procedures as a cause. (LLM, conf=0.84)
Experimental — accuracy not yet measuredSmall changes in data distribution between training and clinical evaluation populations (dataset shift) can lead to substantial variation in clinical performance and expose patients to potential unexpected harm.
“Small changes in the distribution of the underlying data between the algorithm training and clinical evaluation populations (i.e., dataset shift) can lead to substantial variation in clinical performance and expose patients to potential unexpected harm [21, 22].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [21]
The 'TO THE EDITOR' section defines dataset shift as a development-to-deployment data mismatch that causes underperformance, and describes a sepsis-alert model deactivated after demographic changes caused spurious alerts. It also says some causes of shift are subtle and calls for AI safety oversight. It does not establish that small distribution changes cause substantial performance variation or describe patient harm, though such harm is framed as a potential risk in the claim. (LLM, conf=0.84)
Experimental — accuracy not yet measuredDataset shift in medicine is the rule rather than the exception, driven by changes in care delivery, clinical incentives, and documentation rather than by random variation alone.
“Finlayson et al. argue that dataset shift in medicine is “the rule rather than the exception,” driven by changes in care delivery, clinical incentives, and documentation rather than by random variation alone [8].”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [8]
In the 'TO THE EDITOR' section (page not available), the paper describes dataset shift as a major driver of clinical AI malfunction and lists common causes, including changes in behavior such as reimbursement incentives. It does not say that dataset shift is 'the rule rather than the exception' or specifically identify documentation changes as a cause. (LLM, conf=0.86)
Experimental — accuracy not yet measuredBetween 3 and 4 weeks after its first COVID-19 hospitalization, the University of Michigan paused ESM-generated alerts in April 2020 after nursing reports of overalerting.
“Between 3 and 4 weeks after its first COVID-19 hospitalization, the University of Michigan paused ESM-generated alerts in April 2020 after nursing reports of overalerting.5”
From Quantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic (2021) · cites this paper as [5]
The “TO THE EDITOR” section says the University of Michigan Hospital deactivated Epic’s sepsis-alerting model in April 2020 because of spurious alerts during the COVID-19 pandemic. It does not give the interval since the hospital’s first COVID-19 hospitalization or say that nursing reports prompted the decision. (LLM, conf=0.88)
Experimental — accuracy not yet measuredClinicians who are well-prepared to recognize when AI tools are misbehaving are essential partners in safe deployment, and institutions need governance structures, monitoring protocols, and clear pathways for frontline staff to escalate concerns.
“Finlayson et al 55 have argued that clinicians who are well-prepared to recognize when AI tools are misbehaving are essential partners in safe deployment, and that institutions need governance structures, monitoring protocols, and clear pathways for frontline staff to escalate concerns.”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [55]
Understanding persistence and degradation is particularly critical for ML models, where dataset shift should be expected to cause declines in performance when the data used to train a model begins to look different from the data in a new context.
“Understanding persistence and degradation is perhaps particularly critical for ML models, where dataset shift should be expected to cause declines in performance when the data used to train a model begins to look different from the data in a new context [8].”
From Managing a 'responsibility vacuum' in AI monitoring and governance in healthcare: a qualitative study (2025) · cites this paper as [8]
Degradation may be linked to how data are produced within evolving workflows and information systems, not only to who the patients are.
“This supports a workflow-aware interpretation of post-deployment fragility: degradation may be linked to how data are produced within evolving workflows and information systems, not only to who the patients are [8,18,33,35].”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [8]
A wide variety of governance best practices in the form of frameworks, governance structures, checklists, and guidelines have been proposed.
“As AI solutions have emerged, a wide variety of governance best practices in the form of frameworks, governance structures, checklists, and guidelines have been proposed7–23.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [15]
Predictive AI models may not generalize across settings or over time due to dataset shift.
“It is well-known that such models may not generalize across settings or over time due to changes in patient populations, practice patterns, and data processes — a phenomenon known as dataset shift.1”
From Challenges in the postmarket surveillance of clinical prediction models. (2025) · cites this paper as [1]
There is no guarantee that an AI algorithm with high performance will maintain its high performance in the future because the healthcare landscape is constantly changing.
“The healthcare landscape is constantly changing; therefore, there is no guarantee that an AI algorithm with high performance will maintain its high performance in the future [89].”
From Fairness of artificial intelligence in healthcare: review and recommendations (2024) · cites this paper as [89]
Biases in AI models may emerge over time.
“The models should be monitored regularly to address biases that may emerge over time [89].”
From Fairness of artificial intelligence in healthcare: review and recommendations (2024) · cites this paper as [89]
Distribution shifts in patient characteristics necessitate periodic retraining and re-evaluation of AI systems to maintain their relevance and efficacy.
“Consequently, these changes necessitate the periodic retraining and re-evaluation of AI systems to maintain their relevance and efficacy [69, 70].”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [70]
Dynamic changes in clinical practice, comorbidity prevalence, and socio-environmental factors can shift the distribution of patient characteristics.
“The dynamic nature of clinical practice, fluctuations in the prevalence of comorbidities, and various socio-environmental factors can precipitate shifts in the distribution of patient characteristics.”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [70]
Laboratory performance does not always translate directly to clinical practice.
“Laboratory performance does not always translate directly to clinical practice [142,143,144,145,146].”
From Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery (2025) · cites this paper as [144]
Cites
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579 · see citing works
Self-citationFrom development to deployment: dataset shift, causality, and shift-stable models in health AI
Subbaswamy, A., & Saria, S. (2019) · Biostatistics · doi:10.1093/biostatistics/kxz041 · Cited by 276 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Saria S Subbaswamy A. Tutorial: safe and reliable machine learning. In: Proceedings of the conference on fairness accountability and transparency January 29–31 2019. New York: Association for Computing Machinery 2019.
Cited by
Sorted by citations
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028 · see citing works
“Existing medical AI models struggle with distribution shifts, in which distributions of data shift owing to changes in technologies, procedures, settings or populations 37,38 .”
BackgroundVasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01772-9 · Cited by 1,039 · see citing works
Self-citationVasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070904 · Cited by 665 · see citing works
Self-citationFairness of artificial intelligence in healthcare: review and recommendations
Ueda, D., Kakinuma, T., Fujita, S., Kamagata, K., Fushimi, Y., Ito, R., Matsui, Y., Nozaki, T., Nakaura, T., Fujima, N., et al. (2024) · Japanese Journal of Radiology · doi:10.1007/s11604-023-01474-3 · Cited by 607 · see citing works
Feng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00611-y · Cited by 423 · see citing works
Competencies for the Use of Artificial Intelligence–Based Tools by Health Care Professionals
Russell, R. G., Lovett Novak, L., Patel, M., Garvey, K. V., Craig, K. J. T., Jackson, G. P., Moore, D., & Miller, B. M. (2023) · Academic Medicine · doi:10.1097/acm.0000000000004963 · Cited by 320 · see citing works
Artificial Intelligence in Medicine
Beam, A. L., Drazen, J. M., Kohane, I. S., Leong, T.-Y., Manrai, A. K., & Rubin, E. J. (2023) · New England Journal of Medicine · doi:10.1056/nejme2206291 · Cited by 303 · see citing works
Self-citationWILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · see citing works
Chakraborty, C., Bhattacharya, M., Pal, S., & Lee, S. (2023) · Current Research in Biotechnology · doi:10.1016/j.crbiot.2023.100164 · Cited by 295 · see citing works
Data drift in medical machine learning: implications and potential remedies
Sahiner, B., Chen, W., Samala, R. K., & Petrick, N. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20220878 · Cited by 279 · see citing works
A Guide to Cross-Validation for Artificial Intelligence in Medical Imaging
Bradshaw, T., Huemann, Z., Hu, J., & Rahmim, A. (2023) · Radiology Artificial Intelligence · doi:10.1148/ryai.220232 · Cited by 284 · see citing works
Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging
Azizi, S., Culp, L., Freyberg, J., Mustafa, B., Baur, S., Kornblith, S., Chen, T., Tomasev, N., Mitrović, J., Strachan, P., et al. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01049-7 · Cited by 250 · see citing works
Artificial intelligence in positive mental health: a narrative review
Thakkar, A., Gupta, A., & Sousa, A. D. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1280235 · Cited by 299 · see citing works
Machine learning in point-of-care testing: innovations, challenges, and opportunities
Han, G., Goncharov, A., Eryılmaz, M., Ye, S., Palanisamy, B., Ghosh, R., Lisi, F., Rogers, E., Guzmán, D., Yigci, D., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-58527-6 · Cited by 260 · see citing works
Harnessing the Power of Clinical Decision Support Systems: Challenges and Opportunities
Chen, Z., Liang, N., Zhang, H., Li, H., Yang, Y., Zong, X., Chen, Y., Wang, Y., & Shi, N. (2023) · Open Heart · doi:10.1136/openhrt-2023-002432 · Cited by 244 · see citing works
Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review
Plana, D., Shung, D. L., Grimshaw, A. A., Saraf, A., Sung, J. J. Y., & Kann, B. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.33946 · Cited by 236 · see citing works
Artificial intelligence, machine learning, and deep learning in liver transplantation
Bhat, M., Rabindranath, M., Chara, B. S., & Simonetto, D. A. (2023) · Journal of Hepatology · doi:10.1016/j.jhep.2023.01.006 · Cited by 235 · see citing works
Precision medicine in stroke: towards personalized outcome predictions using artificial intelligence
Bonkhoff, A. K., & Grefkes, C. (2021) · Brain · doi:10.1093/brain/awab439 · Cited by 195 · see citing works
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168 · see citing works
“10,48,140 By identifying and addressing biases, these systems can improve healthcare outcomes and equity.”
Arora, A., & Arora, A. (2022) · Future Healthcare Journal · doi:10.7861/fhj.2022-0013 · Cited by 161 · see citing works
Use of artificial intelligence in critical care: opportunities and obstacles
Pinsky, M. R., Bedoya, A., Bihorac, A., Celi, L., Churpek, M., Economou-Zavlanos, N. J., Elbers, P., Saria, S., Liu, V., Lyons, P. G., et al. (2024) · Critical Care · doi:10.1186/s13054-024-04860-z · Cited by 160 · see citing works
Self-citationLum, Z. C. (2023) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000002704 · Cited by 152 · see citing works
Medical Artificial Intelligence and Human Values
Yu, K.-H., Healey, E., Leong, T.-Y., Kohane, I. S., & Manrai, A. K. (2024) · New England Journal of Medicine · doi:10.1056/nejmra2214183 · Cited by 149 · see citing works
Self-citationREFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J. R., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 191 · see citing works
Understanding Liability Risk from Using Health Care Artificial Intelligence Tools
Mello, M. M., & Guha, N. (2024) · New England Journal of Medicine · doi:10.1056/nejmhle2308901 · Cited by 155 · see citing works
APPRAISE-AI Tool for Quantitative Evaluation of AI Studies for Clinical Decision Support
Kwong, J. C. C., Khondker, A., Lajkosz, K., McDermott, M. B. A., Borrat, X., McCradden, M. D., Mamdani, M., Kulkarni, G. S., & Johnson, A. E. W. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.35377 · Cited by 151 · see citing works
Petch, J., Kemppainen, J., Pettengell, C., Aviv, S., Butler, B., Pond, G. R., Saha, A., Bogach, J., Allard-Coutu, A., Sztur, P., et al. (2023) · JCO Clinical Cancer Informatics · doi:10.1200/cci.22.00182 · Cited by 140 · see citing works
Predictive models for clinical decision making: Deep dives in practical machine learning
Eloranta, S., & Boman, M. (2022) · Journal of Internal Medicine · doi:10.1111/joim.13483 · Cited by 133 · see citing works
Henry, K. E., Adams, R., Parent, C., Soleimani, H., Sridharan, A., Johnson, L., Hager, D. N., Cosgrove, S. E., Markowski, A., Klein, E. Y., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01895-z · Cited by 127 · see citing works
Self-citationDemographic bias in misdiagnosis by computational pathology models
Vaidya, A. J., Chen, R. J., Williamson, D. F. K., Song, A. H., Jaume, G., Yang, Y., Hartvigsen, T., Dyer, E., Lu, M. Y., Lipková, J., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02885-z · Cited by 124 · see citing works
Empirical data drift detection experiments on real-world medical imaging data
Kore, A., Abbasi Bavil, E., Subasri, V., Abdalla, M., Fine, B., Dolatabadi, E., & Abdalla, M. (2024) · Nature Communications · doi:10.1038/s41467-024-46142-w · Cited by 124 · see citing works
“…is to detect clinically important changes which might have been caused by changes in the underlying population of patients, disease prevalence, equipment changes, practice/referral pattern among many other factors, and facilitate intervention (retain, warn users, retire model) promptly 6,28,35 .”
A Research Ethics Framework for the Clinical Translation of Healthcare Machine Learning
McCradden, M. D., Anderson, J. A., Stephenson, E. A., Drysdale, E., Erdman, L., Goldenberg, A., & Shaul, R. Z. (2022) · The American Journal of Bioethics · doi:10.1080/15265161.2021.2013977 · Cited by 117 · see citing works
A Nationwide Network of Health AI Assurance Laboratories
Shah, N. H., Halamka, J. D., Saria, S., Pencina, M., Tazbaz, T., Tripathi, M., Callahan, A., Hildahl, H., & Anderson, B. (2024) · JAMA · doi:10.1001/jama.2023.26930 · Cited by 114 · see citing works
Self-citationJha, A. K., Bradshaw, T., Buvat, I., Hatt, M., Prabhat, K. C., Liu, C., Obuchowski, N., Saboury, B., Slomka, P. J., Sunderland, J. J., et al. (2022) · Journal of Nuclear Medicine · doi:10.2967/jnumed.121.263239 · Cited by 113 · see citing works
Stevens, A., & Stetson, P. D. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104550 · Cited by 117 · see citing works
Javaid, A., Zghyer, F., Kim, C., Spaulding, E. M., Isakadze, N., Ding, J., Kargilis, D., Gao, Y., Rahman, F., Brown, D. E., et al. (2022) · American Journal of Preventive Cardiology · doi:10.1016/j.ajpc.2022.100379 · Cited by 109 · see citing works
Self-citationLu, S.-C., Swisher, C. L., Chung, C. D., Jaffray, D. A., & Sidey-Gibbons, C. J. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1129380 · Cited by 117 · see citing works
Silva-Rodríguez, J., Chakor, H., Kobbi, R., Dolz, J., & Ayed, I. B. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103357 · Cited by 121 · see citing works
Beyond generalization: a theory of robustness in machine learning
Freiesleben, T., & Grote, T. (2023) · Synthese · doi:10.1007/s11229-023-04334-9 · Cited by 106 · see citing works
Patient Perspectives on the Use of Artificial Intelligence in Health Care: A Scoping Review
Moy, S., Irannejad, M., Manning, S. J., Farahani, M., Ahmed, Y., Gao, E., Prabhune, R., Lorenz, S., Mirza, R. M., & Klinger, C. A. (2024) · Journal of patient-centered research and reviews · doi:10.17294/2330-0698.2029 · Cited by 113 · see citing works
Generalizability of FDA-Approved AI-Enabled Medical Devices for Clinical Use
Windecker, D., Baj, G., Shiri, I., Kazaj, P. M., Kaesmacher, J., Gräni, C., & Siontis, G. C. M. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.8052 · Cited by 123 · see citing works
Applications of artificial intelligence and machine learning in heart failure
Averbuch, T., Sullivan, K., Sauer, A. J., Mamas, M. A., Voors, A. A., Gale, C. P., Metra, M., Ravindra, N. G., & Spall, H. G. C. V. (2022) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztac025 · Cited by 104 · see citing works
D’Amico, S., Dall’Olio, D., Sala, C., Dall’Olio, L., Sauta, E., Zampini, M., Asti, G., Lanino, L., Maggioni, G., Campagna, A., et al. (2023) · JCO Clinical Cancer Informatics · doi:10.1200/cci.23.00021 · Cited by 97 · see citing works
Ghosh, S., Zhao, X., Alim, M., Brudno, M., & Bhat, M. (2024) · Gut · doi:10.1136/gutjnl-2023-331740 · Cited by 101 · see citing works
Liao, F., Adelaine, S., Afshar, M., & Patterson, B. W. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.931439 · Cited by 94 · see citing works
Koçak, B., Klontzas, M. E., Stanzione, A., Meddeb, A., Demircioğlu, A., Blüthgen, C., Bressem, K. K., Ugga, L., Mercaldo, N. D., Díaz, O., et al. (2025) · European Journal of Radiology Artificial Intelligence · doi:10.1016/j.ejrai.2025.100030 · Cited by 89 · see citing works
Clift, A. K., Dodwell, D., Lord, S., Petrou, S., Brady, M., Collins, G. S., & Hippisley-Cox, J. (2023) · BMJ · doi:10.1136/bmj-2022-073800 · Cited by 97 · see citing works
The impact of commercial health datasets on medical research and health-care algorithms
Alberto, I. R. I., Alberto, N. R. I., Ghosh, A. K., Jain, B., Jayakumar, S., Martinez-Martin, N., McCague, N., Moukheiber, D., Moukheiber, L., Moukheiber, M., et al. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00025-0 · Cited by 86 · see citing works
Artificial intelligence applications in histopathology
Bahadir, C. D., Omar, M., Rosenthal, J., Marchionni, L., Liechty, B., Pisapia, D. J., & Sabuncu, M. R. (2024) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-023-00012-7 · Cited by 84 · see citing works
Homeyer, A., Geißler, C., Schwen, L. O., Zakrzewski, F., Evans, T., Strohmenger, K., Westphal, M., Bülow, R. D., Kargl, M., Karjauv, A., et al. (2022) · Modern Pathology · doi:10.1038/s41379-022-01147-y · Cited by 83 · see citing works
Risk of Bias in Chest Radiography Deep Learning Foundation Models
Glocker, B., Jones, C., Roschewitz, M., & Winzeck, S. (2023) · Radiology Artificial Intelligence · doi:10.1148/ryai.230060 · Cited by 90 · see citing works
Li, F., Wang, S., Gao, Z., Qing, M., Pan, S. L., Liu, Y., & Hu, C. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2024.1510792 · Cited by 87 · see citing works
Analyzing Patient Trajectories With Artificial Intelligence
Allam, A., Feuerriegel, S., Rebhan, M., & Krauthammer, M. (2021) · Journal of Medical Internet Research · doi:10.2196/29812 · Cited by 77 · see citing works
Wang, R., Chaudhari, P., & Davatzikos, C. A. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2211613120 · Cited by 84 · see citing works
London, A. J. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100622 · Cited by 76 · see citing works
Detecting and Remediating Harmful Data Shifts for the Responsible Deployment of Clinical AI Models
Subasri, V., Krishnan, A., Kore, A., Dhalla, A., Pandya, D., Wang, B., Malkin, D., Razak, F., Verma, A. A., Goldenberg, A., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.13685 · Cited by 87 · see citing works
Kann, B. H., Likitlersuang, J., Bontempi, D., Ye, Z., Aneja, S., Bakst, R. L., Kelly, H. R., Juliano, A. F. T., Payabvash, S., Guenette, J. P., et al. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00046-8 · Cited by 74 · see citing works
Gomez-Cabello, C. A., Borna, S., Pressman, S., Haider, S. A., Haider, C. R., & Forte, A. J. (2024) · European Journal of Investigation in Health, Psychology and Education · doi:10.3390/ejihpe14030045 · Cited by 74 · see citing works
Deep Learning for Cardiovascular Imaging
Wehbe, R. M., Katsaggelos, A. K., Hammond, K. J., Hong, H., Ahmad, F. S., Ouyang, D., Shah, S. J., McCarthy, P. M., & Thomas, J. D. (2023) · JAMA Cardiology · doi:10.1001/jamacardio.2023.3142 · Cited by 71 · see citing works
AI in radiology: From promise to practice − A guide to effective integration
Katal, S., York, B. R., & Gholamrezanezhad, A. (2024) · European Journal of Radiology · doi:10.1016/j.ejrad.2024.111798 · Cited by 71 · see citing works
Chen, S., Yu, J., Chamouni, S., Wang, Y., & Li, Y. (2024) · BMC Medicine · doi:10.1186/s12916-024-03566-x · Cited by 73 · see citing works
“Consequently, these changes necessitate the periodic retraining and re-evaluation of AI systems to maintain their relevance and efficacy [69, 70].”
BackgroundThe silent trial - the bridge between bench-to-bedside clinical AI applications
Kwong, J. C. C., Erdman, L., Khondker, A., Skreta, M., Goldenberg, A., McCradden, M. D., Lorenzo, A. J., & Rickard, M. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.929508 · Cited by 73 · see citing works
“…trained on retrospective data alone cannot reliably function in real-world clinical settings as they are prone to dataset drift, which may include variations in how data is de fi ned and collected, or potential changes in the standard of care if training cohorts span long periods of time (13).”
BackgroundSingh, H., Mhasawade, V., & Chunara, R. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000023 · Cited by 67 · see citing works
Ferrario, A., Facchini, A., & Termine, A. (2024) · Minds and Machines · doi:10.1007/s11023-024-09681-1 · Cited by 64 · see citing works
“What we can design is a controlled comparison of the performance of a human expert and the one of a system that executes a (precisely defined) predictive 17 Others use the term ‘dataset shift’ (Finlayson et al., 2021) instead.”
Liu, X., Hu, P., Yeung, W. L. K., Zhang, Z., Ho, V., Liu, C., DuMontier, C., Thoral, P. J., Mao, Z., Cao, D., et al. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00128-0 · Cited by 61 · see citing works
Stogiannos, N., Malik, R., Kumar, A., Barnes, A., Pogose, M., Harvey, H., McEntee, M. F., & Malamateniou, C. (2023) · British Journal of Radiology · doi:10.1259/bjr.20221157 · Cited by 62 · see citing works
Cheng, F., Wang, F., Tang, J., Zhou, Y., Fu, Z., Zhang, P., Haines, J. L., Leverenz, J. B., Gan, L., Hu, J., et al. (2024) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101379 · Cited by 63 · see citing works
Lyons, P. G., Hofford, M. R., Yu, S. C., Michelson, A. P., Payne, P. R. O., Hough, C. L., & Singh, K. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2022.7182 · Cited by 62 · see citing works
Self-citationChest X-ray in Emergency Radiology: What Artificial Intelligence Applications Are Available?
Irmici, G., Cè, M., Caloro, E., Khenkina, N., Pepa, G. D., Ascenti, V., Martinenghi, C. M. A., Papa, S., Oliva, G., & Cellina, M. I. (2023) · Diagnostics · doi:10.3390/diagnostics13020216 · Cited by 59 · see citing works
Gulati, G., Upshaw, J., Wessler, B. S., Brazil, R. J., Nelson, J., Klaveren, D. V., Lundquist, C. M., Park, J. G., McGinnes, H., Steyerberg, E. W., et al. (2022) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.121.008487 · Cited by 60 · see citing works
Artificial intelligence and machine learning for clinical pharmacology
Ryan, D. K., Maclean, R., Balston, A. J., Scourfield, A., Shah, A. D., & Ross, J. (2023) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15930 · Cited by 63 · see citing works
Innovative applications of artificial intelligence in zoonotic disease management
Guo, W., Lv, C., Guo, M., Zhao, Q. C., Yin, X., & Zhang, L. (2023) · Science in One Health · doi:10.1016/j.soh.2023.100045 · Cited by 64 · see citing works
Quantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic
Wong, A., Cao, J., Lyons, P. G., Dutta, S., Major, V. J., Ötleş, E., & Singh, K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.35286 · Cited by 57 · see citing works
Self-citationArtificial Intelligence In Health And Health Care: Priorities For Action
Matheny, M. E., Goldsack, J. C., Saria, S., Shah, N. H., Gerhart, J. L., Cohen, I. G., Price, W. N., Patel, B., Payne, P., Embí, P. J., et al. (2025) · Health Affairs · doi:10.1377/hlthaff.2024.01003 · Cited by 69 · see citing works
Self-citationGe, J., Kim, W. R., Lai, J. C., & Kwong, A. J. (2022) · Journal of Hepatology · doi:10.1016/j.jhep.2022.03.003 · Cited by 57 · see citing works
Expectations for Artificial Intelligence (AI) in Psychiatry
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., Achtyes, E. D., & Bauer, M. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01378-5 · Cited by 56 · see citing works
Tree-based local explanations of machine learning model predictions, AraucanaXAI
Parimbelli, E., Buonocore, T. M., Nicora, G., Michalowski, W., Wilk, S., & Bellazzi, R. (2021) · Artif. Intell. Medicine · doi:10.1016/j.artmed.2022.102471 · Cited by 46 · see citing works
EHR foundation models improve robustness in the presence of temporal distribution shift
Guo, L., Steinberg, E., Fleming, S. L., Posada, J., Lemmon, J., Pfohl, S. R., Shah, N. H., Fries, J., & Sung, L. (2023) · Scientific Reports · doi:10.1038/s41598-023-30820-8 · Cited by 59 · see citing works
Sobhy, M., Abdelkarim, E. A., Hussein, M. A., Aziz, T. A., Al‐Asmari, F., Alabbosh, K. F., Cui, H., & Lin, L. (2025) · Food Bioscience · doi:10.1016/j.fbio.2025.105937 · Cited by 56 · see citing works
Abbas, S. R., Seol, H., Abbas, Z., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13141642 · Cited by 55 · see citing works
Clinical AI tools must convey predictive uncertainty for each individual patient
Banerji, C. R. S., Chakraborti, T., Harbron, C. G., & MacArthur, B. D. (2023) · Nature Medicine · doi:10.1038/s41591-023-02562-7 · Cited by 55 · see citing works
Integrating artificial intelligence into healthcare systems: more than just the algorithm
Kwong, J. C. C., Nickel, G., Wang, S., & Kvedar, J. C. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01066-z · Cited by 54 · see citing works
Koch, L. M., Baumgartner, C. F., & Berens, P. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01085-w · Cited by 53 · see citing works
Singer, M., Angus, D., Annane, D., Bauer, M. W., Kalil, A. C., Klompas, M., Machado, F. R., Martin, G. S., Randolph, A. G., Shankar‐Hari, M., et al. (2026) · The Lancet · doi:10.1016/s0140-6736(25)02422-5 · Cited by 55 · see citing works
Coiera, E. W., & Liu, S. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100860 · Cited by 52 · see citing works
Munroe, E. S., Spicer, A. B., Castellví-Font, A., Zalucky, A. A., Dianti, J., Linck, E. J. G., Talisa, V. B., Urner, M., Angus, D. C., Kassis, E. B., et al. (2025) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(25)00054-2 · Cited by 52 · see citing works
Roschewitz, M., Jones, C., & Glocker, B. (2022) · Nature Medicine · doi:10.1038/s41591-022-01846-8 · Cited by 50 · see citing works
A normative framework for artificial intelligence as a sociotechnical system in healthcare
McCradden, M. D., Joshi, S. D., Anderson, J. A., & London, A. J. (2023) · Patterns · doi:10.1016/j.patter.2023.100864 · Cited by 49 · see citing works
Artificial Intelligence in Cardiovascular Clinical Trials
Cunningham, J. W., Abraham, W. T., Bhatt, A. S., Dunn, J., Felker, G. M., Jain, S. S., Lindsell, C. J., Mace, M., Martyn, T., Shah, R. U., et al. (2024) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2024.08.069 · Cited by 52 · see citing works
Sustainable deployment of clinical prediction tools—a 360° approach to model maintenance
Davis, S. E., Embí, P. J., & Matheny, M. E. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae036 · Cited by 46 · see citing works
Davis, S. E., Dorn, C., Park, D. J., & Matheny, M. E. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf039 · Cited by 43 · see citing works
Clinical use of artificial intelligence requires AI-capable organizations
Novak, L. L., Russell, R. G., Garvey, K., Patel, M., Thomas Craig, K. J., Snowdon, J., & Miller, B. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad028 · Cited by 43 · see citing works
Ética e inteligencia artificial
Galiana, L. I., Corral-Gudino, L., & Miramontes‐González, J. P. (2024) · Revista Clínica Española · doi:10.1016/j.rce.2024.01.007 · Cited by 51 · see citing works
Coronary Risk Estimation Based on Clinical Data in Electronic Health Records
Petrazzini, B. O., Chaudhary, K., Márquez‐Luna, C., Forrest, I. S., Rocheleau, G., Cho, J. H., Narula, J., Nadkarni, G. N., & Do, R. (2022) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2022.01.021 · Cited by 46 · see citing works
Implications of the Use of Artificial Intelligence Predictive Models in Health Care Settings
Vaid, A., Sawant, A. S., Suárez‐Fariñas, M., Lee, J., Kaul, S. K., Kovatch, P., Freeman, R., Jiang, J. J., Jayaraman, P., Fayad, Z. A., et al. (2023) · Annals of Internal Medicine · doi:10.7326/m23-0949 · Cited by 46 · see citing works
Self-citationThe application of artificial intelligence in the management of sepsis
Yang, J., Hao, S., Huang, J., Chen, T., Liu, R., Zhang, P., Feng, M., He, Y., Xiao, W., Hong, Y., et al. (2023) · Medical Review · doi:10.1515/mr-2023-0039 · Cited by 44 · see citing works
The AI Future of Emergency Medicine
Petrella, R. J. (2024) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2024.01.031 · Cited by 43 · see citing works
A proposal for developing a platform that evaluates algorithmic equity and accuracy
Cerrato, P. L., Halamka, J. D., & Pencina, M. J. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100423 · Cited by 43 · see citing works
Advancing healthcare AI governance through a comprehensive maturity model based on systematic review
Hussein, R., Zink, A., Ramadan, B., Howard, F. M., Hightower, M., Shah, S., & Beaulieu-Jones, B. K. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02418-7 · Cited by 42 · see citing works
Artificial intelligence for the detection, prediction, and management of atrial fibrillation
Isaksen, J. L., Baumert, M., Hermans, A. N. L., Maleckar, M., & Linz, D. (2022) · Herzschrittmachertherapie + Elektrophysiologie · doi:10.1007/s00399-022-00839-x · Cited by 42 · see citing works
- Authors
- Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., & Saria, S.
- Affiliations
- Harvard University (US)
- Johns Hopkins University (US)
- University of Michigan (US)
- Yale University (US)
- Boston University (US)
- Bayesian Health, New York, NY
- ORCID
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2021-07-15
- Volume
- 385
- Issue
- 3
- Pages
- 283-286
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- PMC
- PMC8665481
- OpenAlex
- W3177617320
- Cited by
- 855
- References
- 2
- Citations by year
- 2026: 336 · 2025: 141 · 2024: 141 · 2023: 135 · 2022: 83 · 2021: 18 · 2020: 1
- FWCI
- 27.15
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Science Foundation (1840088)
- Alfred P. Sloan Foundation (FG-2018-10877)
- National Institute of General Medical Sciences (T32GM007753)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- distribution shift, clinical trials, machine learning, clinician bias, model generalization, bias mitigation, clinical decision support
- MeSH terms
- Machine Learning, Information Technology, Artificial Intelligence, Humans, Medical Informatics, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-11-02.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-11-02.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
