The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index; 2 of the 7 sources its claims cite were checked for support.
The Clinician and Dataset Shift in Artificial Intelligence
Open accessAbstract
Dataset Shift in Clinical Trials This letter outlines how to identify, and potentially mitigate, common sources of “dataset shift” in machine-learning systems. This occurs when the model “training ...
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 of 7 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“A major driver of AI system malfunction is known as “dataset shift.”4,5”
§ The Clinician and Dataset Shift in Artificial Intelligence > TO THE EDITOR:
What the sources say
- Supportedcites [4] Subbaswamy A, Saria S. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics 2020; 21: 345– 52. 31742354 source
The paper’s opening discussion says shifts between training and deployment are common and can cause poor performance or dangerous decisions. It explicitly calls this problem of shifting conditions that prevent generalization “dataset shift,” supporting the citing sentence’s paraphrase. (LLM, conf=0.92)
- Not checked yetcites [5] Saria S, Subbaswamy A. Tutorial: safe and reliable machine learning. In: Proceedings of the conference on fairness, accountability, and transparency, January 29–31, 2019. New York: Association for Computing Machinery, 2019. DOI: 10.1056/NEJMc2104626
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: Most clinical AI systems today use machine learning.
- Supported
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was a case in which dataset shift fundamentally altered the relationship between fevers and bacterial sepsis, leading the hospital’s clinical AI governing committee (which one of the authors of this letter chairs) to decommission its use.”
§ The Clinician and Dataset Shift in Artificial Intelligence > TO THE EDITOR:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Rests on an assumption: In April 2020, the University of Michigan Hospital deactivated the Epic sepsis-alerting model because of spurious alerting owing to changes in patients’ demographic characteristics associated with the COVID-19 pandemic.
In the paper
“Artificial intelligence (AI) systems are now regularly being used in medical settings,1”
§ The Clinician and Dataset Shift in Artificial Intelligence > TO THE EDITOR:
What the sources say
- Not checked — no full text of the sourcecites [1] Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med 2019; 380: 1347– 58. 30943338 source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Probably not a substantial claim. 3
- It may state two separate claims as one. 2
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Successful recognition and mitigation of dataset shift require both vigilant clinicians and sound technical oversight through AI governance teams.4,5”
§ The Clinician and Dataset Shift in Artificial Intelligence > TO THE EDITOR:
What the sources say
- Partly supportedcites [4] Subbaswamy A, Saria S. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics 2020; 21: 345– 52. 31742354 sourceExperimental — accuracy not yet measured
Section 4, “Open Challenges and Looking Forward” (page not supplied), calls for monitoring to detect shifts, maintenance to address them, and collaboration with domain experts when constructing models. It does not say that vigilant clinicians and AI governance teams are both required for this work. (LLM, conf=0.86)
- Not checked yetcites [5] Saria S, Subbaswamy A. Tutorial: safe and reliable machine learning. In: Proceedings of the conference on fairness, accountability, and transparency, January 29–31, 2019. New York: Association for Computing Machinery, 2019. DOI: 10.1056/NEJMc2104626
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
- Partly supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“we group into changes in technology (e.g., software vendors), changes in population and setting (e.g., new demographics), and changes in behavior (e.g., new reimbursement incentives)”
§ The Clinician and Dataset Shift in Artificial Intelligence > TO THE EDITOR:
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“although regulatory oversight is inconsistent and undeveloped.2,3”
§ The Clinician and Dataset Shift in Artificial Intelligence > TO THE EDITOR:
What the sources say
- Not checked yetcites [2] Ross C Hospitals are using AI to predict the decline of Covid-19 patients — before knowing it works. Stat 4 24, 2020 (https://www.statnews.com/2020/04/24/coronavirus-hospitals-use-ai-to-predict-patient-decline-before-knowing-it-works/).
- Not checked yetcites [3] Artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD) action plan Silver Spring, MD: Food and Drug Administration, January 2021 (https://www.fdagov/media/145022/download).
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Safe deployment of clinical AI requires informed clinician-users”
§ The Clinician and Dataset Shift in Artificial Intelligence > TO THE EDITOR:
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
- Rests on an assumption: Clinicians are generally responsible for identifying and reporting emerging problems with clinical AI.
The paper relies on this without support.
Clinicians are generally responsible for identifying and reporting emerging problems with clinical AI.
“who are generally responsible for identifying and reporting emerging problems.”
The paper relies on this without support.
Most clinical AI systems today use machine learning.
“Most clinical AI systems today use machine learning, algorithms that leverage statistical methods to learn key patterns from clinical data.”
The paper relies on this without support.
In April 2020, the University of Michigan Hospital deactivated the Epic sepsis-alerting model because of spurious alerting owing to changes in patients’ demographic characteristics associated with the COVID-19 pandemic.
“the University of Michigan Hospital implemented the widely used sepsis-alerting model developed by Epic Systems; in April 2020, the model had to be deactivated because of spurious alerting owing to changes in patients’ demographic characteristics associated with the coronavirus disease 2019 pandemic.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This quality becomes crucial when predictive models are deployed across different hospitals or healthcare systems, where variations in patient demographics, clinical practices, and data collection procedures can lead to inaccurate predictions [17, 18].”
From Toward generalizable and interpretable machine learning models in healthcare: Insights from ICU outcome predictions (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Small changes in the distribution of the underlying data between the algorithm training and clinical evaluation populations (i.e., dataset shift) can lead to substantial variation in clinical performance and expose patients to potential unexpected harm [21, 22].”
From Use of artificial intelligence in critical care: opportunities and obstacles (2024) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finlayson et al. argue that dataset shift in medicine is “the rule rather than the exception,” driven by changes in care delivery, clinical incentives, and documentation rather than by random variation alone [8].”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [8]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Between 3 and 4 weeks after its first COVID-19 hospitalization, the University of Michigan paused ESM-generated alerts in April 2020 after nursing reports of overalerting.5”
From Quantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic (2021) · cites this paper as [5]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Finlayson et al 55 have argued that clinicians who are well-prepared to recognize when AI tools are misbehaving are essential partners in safe deployment, and that institutions need governance structures, monitoring protocols, and clear pathways for frontline staff to escalate concerns.”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [55]
The cited source says what the claim attributes to it.
In the citing paper
“Understanding persistence and degradation is perhaps particularly critical for ML models, where dataset shift should be expected to cause declines in performance when the data used to train a model begins to look different from the data in a new context [8].”
From Managing a 'responsibility vacuum' in AI monitoring and governance in healthcare: a qualitative study (2025) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“This supports a workflow-aware interpretation of post-deployment fragility: degradation may be linked to how data are produced within evolving workflows and information systems, not only to who the patients are [8,18,33,35].”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“As AI solutions have emerged, a wide variety of governance best practices in the form of frameworks, governance structures, checklists, and guidelines have been proposed7–23.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“It is well-known that such models may not generalize across settings or over time due to changes in patient populations, practice patterns, and data processes — a phenomenon known as dataset shift.1”
From Challenges in the postmarket surveillance of clinical prediction models. (2025) · cites this paper as [1]
In the citing paper
“The healthcare landscape is constantly changing; therefore, there is no guarantee that an AI algorithm with high performance will maintain its high performance in the future [89].”
From Fairness of artificial intelligence in healthcare: review and recommendations (2024) · cites this paper as [89]
In the citing paper
“The models should be monitored regularly to address biases that may emerge over time [89].”
From Fairness of artificial intelligence in healthcare: review and recommendations (2024) · cites this paper as [89]
In the citing paper
“Consequently, these changes necessitate the periodic retraining and re-evaluation of AI systems to maintain their relevance and efficacy [69, 70].”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [70]
In the citing paper
“The dynamic nature of clinical practice, fluctuations in the prevalence of comorbidities, and various socio-environmental factors can precipitate shifts in the distribution of patient characteristics.”
From Integrating machine learning and artificial intelligence in life-course epidemiology: pathways to innovative public health solutions (2024) · cites this paper as [70]
In the citing paper
“Laboratory performance does not always translate directly to clinical practice [142,143,144,145,146].”
From Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery (2025) · cites this paper as [144]
Citations
Cites
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579
Self-citationFrom development to deployment: dataset shift, causality, and shift-stable models in health AI
Subbaswamy, A., & Saria, S. (2019) · Biostatistics · doi:10.1093/biostatistics/kxz041 · Cited by 276
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Saria S Subbaswamy A. Tutorial: safe and reliable machine learning. In: Proceedings of the conference on fairness accountability and transparency January 29–31 2019. New York: Association for Computing Machinery 2019.
Cited by
Sorted by citations
Items 301–400 of 848
Time-series forecasting through recurrent topology
Chomiak, T., & Hu, B. (2024) · Communications Engineering · doi:10.1038/s44172-023-00142-8 · Cited by 9
Yao, M. W. M., Jenkins, J. M., Nguyen, E. T., Swanson, T., & Menabrito, M. (2024) · Seminars in Reproductive Medicine · doi:10.1055/s-0044-1791536 · Cited by 9
Li, D., Hu, L., Peng, X., Xiao, N., Zhao, H., Liu, G., Liu, H., Li, K., Ai, B., Xia, H., et al. (2022) · iScience · doi:10.1016/j.isci.2022.103961 · Cited by 10
Bertin, D., Bongrand, P., & Bardin, N. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25063270 · Cited by 10
The role of artificial intelligence in macular hole management: A scoping review
Mikhail, D., Milad, D., Antaki, F., Hammamji, K., Qian, C. X., Rezende, F., & Duval, R. (2024) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2024.09.003 · Cited by 8
Unmasking the chameleons: A benchmark for out-of-distribution detection in medical tabular data
Azizmalayeri, M., Abu–Hanna, A., & Cinà, G. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105762 · Cited by 16
Ungureanu, A., Matei, S.-C., & Malita, D. C. (2025) · Diagnostics · doi:10.3390/diagnostics15020230 · Cited by 10
Jong, M. R., Kusters, C. H. J., van Bokhorst, Q. N. E., Jukema, J. B., van Heslinga, R. A. H. V. E., Fockens, K. N., Houwen, B. B. S. L., Jaspers, T. J. M., Boers, T. G. W., Vlugt, M. V., et al. (2025) · Endoscopy · doi:10.1055/a-2530-1845 · Cited by 8
Jones, W. S., & Farrow, D. J. (2025) · Scientific Reports · doi:10.1038/s41598-025-94427-x · Cited by 8
Artificial intelligence and physician burnout: A productivity paradox
Goodson, D. A., Garcia, B., Hogarth, M. A., & Tu, S.-P. (2025) · Learning Health Systems · doi:10.1002/lrh2.70013 · Cited by 11
Cai, J., Guo, L., Zhu, L., Li, X., Qian, L., Lure, Y. M. F., & Yin, X.-P. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1140635 · Cited by 8
Zhu, B., Li, Y., Wu, M., Wu, Q., Liu, K., Li, Y., Guo, W., & Zhao, Z. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1276149 · Cited by 8
Ben-Haim, G., Yosef, M., Eyade, R., Ben-Yosef, J., Berman, A., Sina, S., Halabi, N., Grossbard, E., Marziano, Y., & Segal, G. (2024) · Digital Health · doi:10.1177/20552076241277673 · Cited by 8
Advancing Healthcare AI Governance: A Comprehensive Maturity Model Based on Systematic ReviewPreprint
Hussein, R., Zink, A., Ramadan, B., Howard, F. M., Hightower, M., Shah, S. D., & Beaulieu‐Jones, B. K. (2024) · medRxiv · doi:10.1101/2024.12.30.24319785 · Cited by 8
Developing a suicide risk model for use in the Indian Health Service
Adams, R., Haroz, E. E., Rebman, P., Suttle, R., Grosvenor, L., Bajaj, M. A., Dayal, R. R., Maggio, D., Kettering, C. L., & Goklish, N. A. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00088-5 · Cited by 7
Foundation models in healthcare require rethinking reliability
Grote, T., Freiesleben, T., & Berens, P. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00924-5 · Cited by 7
Jong, M. R., van Heslinga, R. A. H. V. E., Kusters, C. H. J., Jaspers, T. J. M., Boers, T. G. W., Duits, L. C., Pouw, R. E., Weusten, B. L. A. M., Alkhalaf, A., van der Sommen, F., et al. (2025) · Endoscopy · doi:10.1055/a-2642-7584 · Cited by 9
Kolla, L., Linn, K. A., Navathe, A. S., Kreisler, C., Roberts, C. B., Park, S.-H., Singh, H., Feng, J., Chen, J., & Parikh, R. B. (2025) · JAMA Health Forum · doi:10.1001/jamahealthforum.2025.2717 · Cited by 8
Mathes, S., Ferber, D., Dreyer, T. F., Borm, K. J., Modersohn, L., Willem, T., Dirven, R. J., Vibert, J., Kreutzfeldt, S., Pérez-López, R., et al. (2025) · npj Precision Oncology · doi:10.1038/s41698-025-01180-5 · Cited by 7
Chang, L., Li, H., Wu, W., Liu, X., Yan, J., Chen, Z., Wu, H., & Song, S. (2025) · Journal of Translational Medicine · doi:10.1186/s12967-025-07591-z · Cited by 10
Herrmann, M., Wabro, A., & Winkler, E. C. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09764-8 · Cited by 7
The Impact of Domain Shift on Left and Right Ventricle Segmentation in Short Axis Cardiac MR Images
Uğurlu, D., Anton, E. P., Ruijsink, B., Young, A. A., Machado, I. P., Hammernik, K., King, A. P., & Schnabel, J. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-93722-5_7 · Cited by 7
Marwaha, J. S., Beaulieu-Jones, B. R., Kennedy, C. J., Nathanson, L. A., Robinson, K. A., Tandon, M., & Brat, G. A. (2022) · NEJM Catalyst · doi:10.1056/cat.21.0477 · Cited by 7
Monitoring Approaches for a Pediatric Chronic Kidney Disease Machine Learning Model
Morse, K., Brown, C., Fleming, S. L., Todd, I., Powell, A., Russell, W. A., Scheinker, D., Sutherland, S. M., Lu, J., Watkins, B., et al. (2022) · Applied Clinical Informatics · doi:10.1055/s-0042-1746168 · Cited by 8
Ogero, M. O., Ndiritu, J., Sarguta, R. J., Tuti, T., Aluvaala, J., & Akech, S. (2023) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12948 · Cited by 7
Prabhudesai, S., Hauth, J. M., Guo, D., Rao, A. U. K., Banović, N., & Huan, X. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1071174 · Cited by 9
Xue, H., Hu, G., Hong, N., Dunnick, N. R., & Jin, Z. (2023) · 中国科学通报：英文版 · doi:10.1016/j.scib.2023.03.031 · Cited by 7
Robustness Stress Testing in Medical Image Classification
Islam, M., Li, Z., & Glocker, B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44336-7_17 · Cited by 8
Fitzke, M., Stack, C., Dourson, A., Santana, R. M. B., Wilson, D., Ziemer, L. S., Soin, A., Lungren, M. P., Fisher, P. E., & Parkinson, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08165 · Cited by 12
Bhavani, S. V. (2022) · Critical Care Medicine · doi:10.1097/ccm.0000000000005568 · Cited by 7
HuBERT-ECG as a self-supervised foundation model for broad and scalable cardiac applicationsPreprint
Coppola, E., Savardi, M., Massussi, M., Adamo, M., Metra, M., & Signoroni, A. (2024) · medRxiv · doi:10.1101/2024.11.14.24317328 · Cited by 18
Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery
Kuno, M., Osumi, H., Udagawa, S., Yoshikawa, K., Ooki, A., Shinozaki, E., Ishikawa, T., Oba, J., Yamaguchi, K., & Sakurada, K. (2025) · Current Oncology · doi:10.3390/curroncol32110588 · Cited by 7
Preserving the Fairness Guarantees of Classifiers in Changing Environments: A Survey
Barrainkua, A., Gordaliza, P., Lozano, J. A., & Quadrianto, N. (2023) · ACM Computing Surveys · doi:10.1145/3637438 · Cited by 14
Ahmad, F. S., Ling-Hu, T., ADLER, E. D., Petito, L. C., Wehbe, R. M., Wilcox, J. E., Mutharasan, R. K., Nardone, B., Tadel, M., Greenberg, B., et al. (2024) · Clinical Research in Cardiology · doi:10.1007/s00392-024-02433-2 · Cited by 6
Zhang, Q., Cai, H., Long, Y., & Yu, S. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.107483 · Cited by 6
Hatherley, J. J. (2025) · Ethics and Information Technology · doi:10.1007/s10676-025-09829-2 · Cited by 9
Using routinely collected data for research purposes: challenges and mitigation strategies
Hoffmann, S., Morris, T. P., Herrmann, M., Heinze, G., Wynants, L., CALSTER, B. V., Bischl, B., Schmid, M., Shaw, P. A., Mathes, T., et al. (2026) · BMJ · doi:10.1136/bmj-2025-087812 · Cited by 8
Advances in the diagnosis and detection of chronic kidney disease
Lees, J. S., Zhang, L., Anandh, U., Kronbichler, A., Fisher, L.-A., Borg, M., Wang, A. Y.-M., & Levin, A. (2026) · The Lancet · doi:10.1016/s0140-6736(26)00702-6 · Cited by 6
Somwanshi, S. K., Tharayil, A. S., Budhewar, A. S., Velu, G., Thamızhanban, D., K.C.Gayathri, Dasarwar, A. N., Patil, P. G., Somwanshi, S. K., Patil, B. K., et al. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7862-6.ch007 · Cited by 6
Lee, S. H., Geng, H., & Xiao, Y. (2021) · Clinical Oncology · doi:10.1016/j.clon.2021.11.020 · Cited by 6
Wang, Y., Cui, H., Li, L., Cao, Y., Qu, H., Ailina, H., Dou, Z., Tang, C., Qin, W., Wang, C., et al. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2023.1146941 · Cited by 6
Applications of Artificial Intelligence in Helicopter Emergency Medical Services: A Scoping Review
Hsueh, J., Fritz, C. L., Thomas, C. E., Reimer, A. P., Reisner, A. T., Schoenfeld, D. W., Haimovich, A. D., & Thomas, S. H. (2023) · Air Medical Journal · doi:10.1016/j.amj.2023.11.012 · Cited by 6
Clinical Integration of Artificial Intelligence for Breast Imaging
Wilkinson, L., Dunbar, J. K., & Lip, G. (2024) · Radiologic Clinics of North America · doi:10.1016/j.rcl.2023.12.006 · Cited by 6
Keim‐Malpass, J., Moorman, L. P., Moorman, J. R., Hamil, S., Yousefvand, G., Monfredi, O., Ratcliffe, S. J., Krahn, K. N., Jones, M., Clark, M., et al. (2024) · Physiological Measurement · doi:10.1088/1361-6579/ad4e90 · Cited by 5
Prediction of stroke severity: systematic evaluation of lesion representations
Bonkhoff, A. K., Cohen, A. L., Drew, W., Ferguson, M. A., Hussain, A., Lin, C., Schaper, F. L. W. V. J., Bourached, A., Giese, A., de Oliveira, L. C., et al. (2024) · Annals of Clinical and Translational Neurology · doi:10.1002/acn3.52215 · Cited by 6
Hu, H., Li, J., Ge, H., Wu, B., Feng, T., Wu, X., & Wu, X. (2025) · Nursing in Critical Care · doi:10.1111/nicc.13306 · Cited by 7
Trinkley, K. E., Simon, S. T., & Rosenberg, M. A. (2025) · Journal of Medical Internet Research · doi:10.2196/68256 · Cited by 6
Multicenter Validation of a Machine Learning Model for Surgical Transfusion Risk at 45 US Hospitals
Lou, S. S., Kumar, S., Goss, C. W., Avidan, M. S., Kheterpal, S., Kannampallil, T., Freundlich, R. E., Mathis, M. R., O’Reilly-Shah, V. N., & Karamchandani, K. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.17760 · Cited by 7
Emani, S. S., Swaminathan, A., Grobman, B., Duvall, J. B., López, I., Arnaout, O. M., & Huang, K. T. (2023) · Neurosurgical FOCUS · doi:10.3171/2023.3.focus2352 · Cited by 7
Trash in/trash out? Using routinely collected clinical data for data science in the ICU: Con
Pörteners, B., Jung, C., & Meyfroidt, G. (2024) · Intensive Care Medicine · doi:10.1007/s00134-024-07739-3 · Cited by 6
Machine learning in cancer-associated thrombosis: hype or hope in untangling the clot
Patell, R. D., Zwicker, J. I., Singh, R. K., & Mantha, S. H. (2024) · Bleeding Thrombosis and Vascular Biology · doi:10.4081/btvb.2024.123 · Cited by 5
Watson, M., Chambers, P., Steventon, L., King, J. H., Ercia, A., Shaw, H. M., & Moubayed, N. A. (2024) · BMJ Oncology · doi:10.1136/bmjonc-2024-000430 · Cited by 6
3D Printing Automation Control System Based on Artificial Intelligence
Liu, S., & Huang, J. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.10.057 · Cited by 5
Deep learning-based automatic image quality assessment in ultra-widefield fundus photographs
Oh, R., Park, U. C., Park, K. H., Park, S. J., & Yoon, C. K. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-100058 · Cited by 5
Ruan, Q., Shi, J., Dai, Y., Yang, P., Zhu, N., & Wang, S. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02247-3 · Cited by 7
Shah, C., Ghodasara, S., Chen, D., & Chen, P. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.09.032 · Cited by 5
Emerging role of artificial intelligence in gastroenterology and hepatology
Shrestha, U. K. (2025) · World Journal of Gastroenterology · doi:10.3748/wjg.v31.i39.111495 · Cited by 8
Integrating AI into healthcare systems: A multivocal literature review
Mallardi, G., Calefato, F., Quaranta, L., & Lanubile, F. (2025) · Smart Health · doi:10.1016/j.smhl.2025.100631 · Cited by 6
Jiang, Z., Chen, H., Wu, Y., Qin, Y., Pei, C., Zeng, D., Sheng, B., & Wong, T. Y. (2026) · Intelligent Medicine · doi:10.1016/j.imed.2026.01.001 · Cited by 6
Ong, A. Y., Struyven, R., Denniston, A. K., Merle, D. A., Engelmann, J., Kim, H., Zhou, Y., Keane, P. A., & Laíns, I. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02463-2 · Cited by 5
Artificial intelligence in emergency nursing: A scoping review of applications and implications
Cha, C., Yi, N., & Baek, G. (2026) · International Emergency Nursing · doi:10.1016/j.ienj.2026.101809 · Cited by 5
Explainability in context: calibrating appropriate trust and reliance in artificial intelligence
Davis, S. E., & Salwei, M. E. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag082 · Cited by 8
Koh, A., Dhanya, B. A. B. Y., Martis, W. R., & Daniel Capurro, N. (2025) · Minerva Anestesiologica · doi:10.23736/s0375-9393.25.19197-9 · Cited by 5
Pediatric Emergency Department Sepsis Screening Tool Accuracy During the COVID-19 Pandemic
Yan, A. P., Zipursky, A. R., Capraro, A., Harper, M. B., & Eisenberg, M. (2022) · PEDIATRICS · doi:10.1542/peds.2022-057492 · Cited by 5
Fairness as adequacy: a sociotechnical view on model evaluation in machine learning
Grote, T. (2023) · AI and Ethics · doi:10.1007/s43681-023-00280-x · Cited by 6
Bertin, D., Bongrand, P., & Bardin, N. (2023) · Journal of Autonomous Intelligence · doi:10.32629/jai.v6i2.664 · Cited by 5
Normalizing flow based uncertainty estimation for deep regression analysis
Zhang, B., Sui, W., Huang, Z., Li, M., & Qi, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127645 · Cited by 6
Chen, C., Han, P., Chen, S., Shardell, M. D., & Qin, J. (2024) · Biometrics · doi:10.1093/biomtc/ujae090 · Cited by 5
Practical foundations of machine learning for addiction research. Part II. Workflow and use cases
Morgado, P. C., Carusso, M., Alemany, L. A., & Ación, L. (2022) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2021.1966435 · Cited by 6
Kamineni, M., Ötleş, E., Oh, J., Rao, K. K., Young, V. B., Li, B. Y., West, L. R., Hooper, D. C., Shenoy, E. S., Guttag, J. V., et al. (2022) · Infection Control and Hospital Epidemiology · doi:10.1017/ice.2022.218 · Cited by 5
James, N., Gerrish, L., Rokotyan, N., & Gladding, P. A. (2021) · medRxiv · doi:10.1101/2021.07.26.21261115 · Cited by 5
Reimagining cancer treatments in the era of generative AI
Derbal, Y. (2025) · DIGITAL HEALTH · doi:10.1177/20552076251394631 · Cited by 4
Establishing and implementing a responsible artificial intelligence framework: a 1-year review
Tsoi, A. H., Gartner, G., Cotten, S. W., Kim, J., Nazarian, J., Thomas, J., McSwain, S. D., Ahmadi-Moosavi, R., & Rimal, R. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf147 · Cited by 4
Amer, M., Gittins, R., Millana, A. M., Scheibein, F., Ferri, M., Tofighi, B., Sullivan, F., Handley, M., Ghosh, M., Baldacchino, A., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/58723 · Cited by 5
Geng, J., Sui, X., Du, R., Feng, J., Wang, R., Wang, M., Yao, K., Chen, Q., Bai, L., Wang, S., et al. (2024) · Radiation Oncology · doi:10.1186/s13014-024-02463-0 · Cited by 4
Park, S.-H. (2025) · Journal of Neurosonology and Neuroimaging · doi:10.31728/jnn.2025.00171 · Cited by 4
Cardiovascular risk assessment enhanced by automated machine learning in a multi-phase study
Bibi, I., Schaffert, D., Blanke, P., Illian, L., Lenzing, F., Martin, N., Leipe, J., März, W., Stach, K., & Olsavszky, V. (2025) · Scientific Reports · doi:10.1038/s41598-025-24189-z · Cited by 4
The role of artificial intelligence in hypertension management
Dherani, M., Wagner, S. K., & Shantsila, E. (2026) · Current Opinion in Nephrology & Hypertension · doi:10.1097/mnh.0000000000001157 · Cited by 4
Ramadan, R. Α., Madi, N. K. M., Fageeri, S. O., Yousif, K. A., Al-Daweri, M. S., & Rajhi, W. K. H. A. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.132919 · Cited by 5
Dullabh, P., Zott, C., Gauthreaux, N. A., Peterson, C., Aronoff, A., Monkhouse, K., & Sittig, D. F. (2026) · Journal of Medical Internet Research · doi:10.2196/81628 · Cited by 7
A cancer-type-aware framework for robust multimodal survival prediction under missing modalities
Song, Y., Zhan, Z., Xie, F., Wang, N., Peng, Y., Zhang, R., & Lin, M. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag124 · Cited by 4
Sevryugina, Y. V., & Vargas, D. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c01620 · Cited by 5
Klunder, L. A., Guericke, D., Bemthuis, R., Koenderink, M., & Machado, M. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01138-1 · Cited by 5
A hybrid adaptive approach for instance transfer learning with dynamic and imbalanced data
Zhang, X., Liu, K., Yuan, B., Wang, H., Chen, S., Xue, Y., Chen, W., Liu, M., & Hu, Y. Q. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.23055 · Cited by 4
Implementing Gait Kinematic Trajectory Forecasting Models on an Embedded System
Shayne, M., Molina, L. A., Hu, B., & Chomiak, T. (2024) · Sensors · doi:10.3390/s24082649 · Cited by 5
Hu, P., Steingrimsson, J. A., Cole, E. B., Cormack, J. B., Dunn, B. K., Gatsonis, C., Lee, C., Li, N., Pisano, E. D., He, J., et al. (2025) · JNCI Monographs · doi:10.1093/jncimonographs/lgae049 · Cited by 4
AI in Healthcare: Bridging the Gap between Research and Clinical Implementation
Lanka, S., & Madala, P. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24oct089 · Cited by 5
Oncointerpreter.ai enables interactive, personalized summarization of cancer diagnostics data
Tripathi, A., Ecker, B. L., Boland, P. M., Ghodoussipour, S. B., Riedlinger, G. R., & De, S. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae284 · Cited by 4
Huang, E. S. (2022) · Med · doi:10.1016/j.medj.2022.03.005 · Cited by 4
Ethical Issues of Artificial Intelligence & Assisted Reproductive Technologies
Zammit, R. (2023) · The International Journal of Prenatal & Life Sciences · doi:10.24946/ijpls/2023132305 · Cited by 4
National Multi-Institutional Validation of a Surgical Transfusion Risk Prediction Model
Lou, S. S., Liu, Y., Cohen, M. E., Ko, C. Y., Hall, B. L., & Kannampallil, T. (2023) · Journal of the American College of Surgeons · doi:10.1097/xcs.0000000000000874 · Cited by 4
Wei, W. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2023.11.127 · Cited by 4
Design of Intelligent Recommendation System for Graduate Employment Based on Artificial Intelligence
Lan, Z. (2023) · International Conference on Evolutionary Algorithms and Soft Computing Techniques (EASCT) · doi:10.1109/easct59475.2023.10392969 · Cited by 4
Bui, D. P., Bajema, K. L., Huang, Y., Yan, L., Li, Y., Rajeevan, N., Berry, K., Rowneki, M., Argraves, S., Hynes, D. M., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307235 · Cited by 4
Mathis, M. R., Singh, K., & Kheterpal, S. (2022) · Anesthesiology · doi:10.1097/aln.0000000000004250 · Cited by 4
Self-citationPerformance drift is a major barrier to the safe use of machine learning in cardiac surgeryPreprint
Dong, T., Sinha, S., Zhai, B., Fudulu, D. P., Chan, J., Narayan, P., Judge, A., Caputo, M., Dimagli, A., Benedetto, U., et al. (2023) · medRxiv · doi:10.1101/2023.01.21.23284795 · Cited by 4
Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems
Kopanitsa, G. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001534 · Cited by 3
“This supports a workflow-aware interpretation of post-deployment fragility: degradation may be linked to how data are produced within evolving workflows and information systems, not only to who the patients are [8,18,33,35].”
SURGE-ahead postoperative delirium prediction: external validation and open-source library
Kocar, T. D., Wolf, P., Leinert, C., Brefka, S., Fotteler, M., Uihlein, A., Wezel, F., Wehling, M., Rahbari, N. N., Kestler, H. A., et al. (2025) · European Geriatric Medicine · doi:10.1007/s41999-025-01180-5 · Cited by 3
Peng, J.-C., Liang, J.-R., Zhu, M., Wang, C., & Gao, Y. (2026) · Digital Health · doi:10.1177/20552076251412651 · Cited by 3
Shen, Y., Chen, Q., He, X., Agrawal, R. V., Grzybowski, A. E., Jin, K., & Ye, X. (2026) · Ophthalmology and Therapy · doi:10.1007/s40123-026-01316-1 · Cited by 3
Metadata
- Authors
- Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., & Saria, S.
- Affiliations
- Harvard University (US)
- Johns Hopkins University (US)
- University of Michigan (US)
- Yale University (US)
- Boston University (US)
- Bayesian Health, New York, NY
- ORCID
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2021-07-15
- Volume
- 385
- Issue
- 3
- Pages
- 283-286
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- PMC
- PMC8665481
- OpenAlex
- W3177617320
- Cited by
- 855
- References
- 2
- Citations by year
- 2026: 336 · 2025: 141 · 2024: 141 · 2023: 135 · 2022: 83 · 2021: 18
- FWCI
- 27.15
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Science Foundation (1840088)
- Alfred P. Sloan Foundation (FG-2018-10877)
- National Institute of General Medical Sciences (T32GM007753)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- distribution shift, clinical trials, machine learning, clinician bias, model generalization, bias mitigation, clinical decision support
- MeSH terms
- Machine Learning, Information Technology, Artificial Intelligence, Humans, Medical Informatics, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-02, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-02 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
