A Path for Translation of Machine Learning Products into Healthcare Delivery

Mark Sendak; Joshua D’Arcy; Sehj Kashyap; Michael C. Gao; Marshall Nichols; Kristin Corey; William Ratliff; Suresh Balu

doi:10.33590/emjinnov/19-00172

Cited by 172Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareElectronic Health Records Systems

A Path for Translation of Machine Learning Products into Healthcare Delivery

Open access

Mark Sendak, Joshua D’Arcy, Sehj Kashyap, Michael C. Gao, Marshall Nichols, Kristin Corey, William Ratliff, Suresh Balu·2020·EMJ Innovations

DOI 10.33590/emjinnov/19-00172

Open this paper in CoChat — free account Open PDF

Abstract

Despite enormous enthusiasm, machine learning models are rarely translated into clinical care and there is minimal evidence of clinical or economic impact. New conference venues and academic journals have emerged to promote the proliferating research; however, the translational path remains unclear. This review undertakes the first in-depth study to identify how machine learning models that ingest structured electronic health record data can be applied to clinical decision support tasks and translated into clinical practice. The authors complement their own work with the experience of 21 machine learning products that address problems across clinical domains and across geographic populations. Four phases of translation emerge: design and develop, evaluate and validate, diffuse and scale, and continuing monitoring and maintenance. The review highlights the varying approaches taken across each phase by teams building machine learning products and presents a discussion of challenges and opportunities. The translational path and associated findings are instructive to researchers and developers building machine learning products, policy makers regulating machine learning products, and health system leaders who are considering adopting a machine learning product.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.33590/emjinnov/19-00172

Cites

79 of 79 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Little evidence exists on the optimal integration of sepsis models and other machine learning models into clinical workflows.
“As a result, little evidence exists on the optimal integration of these sepsis models and other machine learning models into clinical workflows [9].”
From Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study (2020) · cites this paper as [9]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The third international consensus definitions for sepsis and septic shock (Sepsis-3).
Singer, M., Deutschman, C. S., Seymour, C. W., Shankar-Hari, M., Annane, D., Bauer, M., Bellomo, R., Bernard, G. R., Chiche, J.-D., Coopersmith, C. M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0287 · Cited by 28,250 · cited 1× · see citing works
“96 Similar to other technology innovations, the product lifecycle continues and will need to adapt to changes in market dynamics and organisational needs.”
Background
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1× · see citing works
“106 Second, the dataset used to train a model 107 or the outcome that a model predicts 108 can have significant implications for how models lessen or worsen disparities in healthcare.”
Method
Machine Learning in Medicine
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579 · see citing works
Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 1× · see citing works
“92,93 To date, no machine learning product ingesting EHR data has successfully diffused and scaled across healthcare.”
Background
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506 · cited 1× · see citing works
“17 Machine learning models can be trained in a supervised and unsupervised fashion.”
Background
The practical implementation of artificial intelligence technologies in medicine
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019) · Nature Medicine · doi:10.1038/s41591-018-0307-0 · Cited by 2,246 · cited 1× · see citing works
“21,103,104 Data sources and data transformations also impact model performance across studies.”
Background
Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices
Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638 · see citing works
Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.3763 · Cited by 1,395 · see citing works
A Predictive Model for Progression of Chronic Kidney Disease to Kidney Failure
Tangri, N. (2011) · JAMA · doi:10.1001/jama.2011.451 · Cited by 1,359 · see citing works
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341 · cited 2× · see citing works
“106 Second, the dataset used to train a model 107 or the outcome that a model predicts 108 can have significant implications for how models lessen or worsen disparities in healthcare.”
BackgroundMethod
Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308 · see citing works
Do no harm: a roadmap for responsible machine learning for health care
Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0548-6 · Cited by 1,266 · cited 2× · see citing works
“This review also advances prior work to propose best practices for teams building machine learning models within a healthcare setting 14 and for teams conducting quality improvement work following the learning health system framework.”
BackgroundSelf-citation
A clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086 · see citing works
Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review
Goldstein, B. A., Navar, A. M., Pencina, M. J., & Ioannidis, J. P. A. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw042 · Cited by 936 · cited 2× · see citing works
“10,21 Configurations are categorised as either centralised or decentralised.”
Background
Clinical decision support in the era of artificial intelligence
Shortliffe, E. H., & Sepúlveda, M. J. (2018) · JAMA · doi:10.1001/jama.2018.17163 · Cited by 920 · see citing works
An interpretable machine learning model for accurate prediction of sepsis in the ICU
Nemati, S., Holder, A., Razmi, F., Stanley, M. D., Clifford, G. D., & Buchman, T. G. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000002936 · Cited by 832 · see citing works
Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review
Xiao, C., Choi, E., & Sun, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy068 · Cited by 792 · see citing works
Multinational Assessment of Accuracy of Equations for Predicting Risk of Kidney Failure: A Meta-analysis
Tangri, N., Grams, M. E., Levey, A. S., Coresh, J., Appel, L. J., Astor, B. C., Chodick, G., Collins, A. J., Djurdjev, O., Elley, C. R., et al. (2016) · JAMA · doi:10.1001/jama.2015.18202 · Cited by 737 · see citing works
Reporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701 · cited 1× · see citing works
“21,103,104 Data sources and data transformations also impact model performance across studies.”
Background
Deep Learning—A Technology With the Potential to Transform Health Care
Hinton, G. (2018) · JAMA · doi:10.1001/jama.2018.11100 · Cited by 667 · see citing works
Multicenter Comparison of Machine Learning Methods and Conventional Regression for Predicting Clinical Deterioration on the Wards
Churpek, M. M., Yuen, T. C., Winslow, C., Meltzer, D. O., Kattan, M. W., & Edelson, D. P. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000001571 · Cited by 638 · see citing works
A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data
Kahn, M. G., Callahan, T. J., Barnard, J., Bauck, A. E., Brown, J., Davidson, B. N., Estiri, H., Goerg, C., Holve, E., Johnson, S. G., et al. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1244 · Cited by 636 · see citing works
A targeted real-time early warning score (TREWScore) for septic shock.
Henry, K. E., Hager, D. N., Pronovost, P. J., & Saria, S. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aab3719 · Cited by 598 · see citing works
The “inconvenient truth” about AI in healthcare
Panch, T., Mattie, H., & Celi, L. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0155-4 · Cited by 473 · cited 1× · see citing works
“89 Finally, before being integrated into clinical care, a machine learning product needs to evaluated and validated in a ‘silent’ mode, “in which predictions are made in real-time and exposed to a group of clinical experts.””
Background
Machine-Learning-Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index
Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., Dugas, A., Linton, B., Kirsch, T., & Kelen, G. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2017.08.005 · Cited by 424 · see citing works
Implementing the Learning Health System: From Concept to Action
Greene, S. M., Reid, R. J., & Larson, E. B. B. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-3-201208070-00012 · Cited by 398 · cited 1× · see citing works
“15 However, there is not a unifying translational path to inform teams beyond success within a single setting to diffuse and scale across healthcare.”
Background
Artificial intelligence and the implementation challenge
Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13659 · Cited by 389 · cited 1× · see citing works
“92,93 To date, no machine learning product ingesting EHR data has successfully diffused and scaled across healthcare.”
Background
Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: A randomised clinical trial.
Shimabukuro, D. W., Barton, C. W., Feldman, M. D., Mataraso, S. J., & Das, R. (2017) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2017-000234 · Cited by 382 · cited 1× · see citing works
“Only one product, InSight (Dascena, Oakland, California, USA), 40 conducted a single-site, randomised control trial with 142 patients and demonstrated positive results.”
Background
Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU
Mao, Q., Jay, M., Hoffman, J. L., Calvert, J., Barton, C., Shimabukuro, D., Shieh, L., Chettipally, U., Fletcher, G., Kerem, Y., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-017833 · Cited by 336 · see citing works
Calibration drift in regression and machine learning models for acute kidney injury
Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D., & Matheny, M. E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx030 · Cited by 312 · see citing works
With an eye to AI and autonomous diagnosis
Keane, P. A., & Topol, E. J. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0048-y · Cited by 304 · cited 1× · see citing works
“97,98 IDx-DR is now actively being scaled and diffused.”
Background
Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study
Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., et al. (2020) · JMIR Medical Informatics · doi:10.2196/15182 · Cited by 287 · see citing works
Self-citation
A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice*
Giannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269 · see citing works
Multicenter Development and Validation of a Risk Stratification Tool for Ward Patients
Churpek, M. M., Yuen, T. C., Winslow, C. J., Robicsek, A. A., Meltzer, D. O., Gibbons, R. D., & Edelson, D. P. (2014) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201406-1022oc · Cited by 258 · see citing works
Lost in thought—the limits of the human mind and the future of medicine
Obermeyer, Z., & Lee, T. H. (2017) · New England Journal of Medicine · doi:10.1056/nejmp1705348 · Cited by 252 · cited 1× · see citing works
“19 The distinction between decision support and automation is critical: “while it may be assumed that decision support is simply a stepping stone on the progression towards full automation, the truth is that decision support systems have fundamentally different considerations that must be accounted…”
Background
Development and validation of a continuous measure of patient condition using the Electronic Medical Record
Rothman, M. J., Rothman, S. I., & Beals, J. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.06.011 · Cited by 225 · see citing works
Development and validation of machine learning models to identify high-risk surgical patients using automatically curated electronic health record data (Pythia): a retrospective, single-site study
Corey, K. M., Kashyap, S., Lorenzi, E., Lagoo-Deenadayalan, S. A., Heller, K., Whalen, K., Balu, S., Heflin, M. T., McDonald, S. R., Swaminathan, M., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002701 · Cited by 204 · see citing works
Self-citation
A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately?
Rawson, T. M., Moore, L. S. P., Hernandez, B., Charani, E., Castro‐Sánchez, E., Herrero, P., Hayhoe, B. W. J., Hope, W. W., Georgiou, P., & Holmes, A. H. (2017) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2017.02.028 · Cited by 190 · see citing works
Early detection of impending physiologic deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record
Escobar, G. J., LaGuardia, J. C., Turk, B. J., Ragins, A. I., Kipnis, P., & Draper, D. (2012) · Journal of Hospital Medicine · doi:10.1002/jhm.1929 · Cited by 185 · see citing works
Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU
Kipnis, P., Turk, B. J., Wulf, D. A., LaGuardia, J. C., Liu, V. X., Churpek, M. M., Brufau, S. R., & Escobar, G. J. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.09.013 · Cited by 167 · cited 1× · see citing works
“There is a head-to-head comparison of the Advance Alert Monitor and eCart on the same dataset, 105 but this practice is exceedingly rare.”
Background
Clinician perception of a machine learning–based early warning system designed to predict severe sepsis and septic shock
Ginestra, J. C., Giannini, H. M., Schweickert, W. D., Meadows, L., Lynch, M. J., Pavan, K., Chivers, C. J., Draugelis, M., Donnelly, P. J., Fuchs, B. D., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003803 · Cited by 159 · see citing works
Big Data, Big Tech, and Protecting Patient Privacy
Cohen, I. G., & Mello, M. M. (2019) · JAMA · doi:10.1001/jama.2019.11365 · Cited by 135 · cited 1× · see citing works
“106 Second, the dataset used to train a model 107 or the outcome that a model predicts 108 can have significant implications for how models lessen or worsen disparities in healthcare.”
Method
Development and validation of a predictive model for detection of colorectal cancer in primary care by analysis of complete blood counts: a binational retrospective study
Kinar, Y., Kalkstein, N., Akiva, P., Levin, B., Half, E., Goldshtein, I., Chodick, G., & Shalev, V. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv195 · Cited by 128 · see citing works
A Generalizable, Data-Driven Approach to Predict Daily Risk ofClostridium difficileInfection at Two Large Academic Health Centers
Oh, J., Makar, M., Fusco, C., McCaffrey, R., Rao, K. K., Ryan, E. E., Washer, L., West, L. R., Young, V. B., Guttag, J. V., et al. (2018) · Infection Control and Hospital Epidemiology · doi:10.1017/ice.2018.16 · Cited by 124 · see citing works
Early Colorectal Cancer Detected by Machine Learning Model Using Gender, Age, and Complete Blood Count Data
Hornbrook, M. C., Goshen, R., Choman, E. N., O'Keeffe-Rosetti, M. C., Kinar, Y., Liles, E. G., & Rust, K. C. (2017) · Digestive Diseases and Sciences · doi:10.1007/s10620-017-4722-8 · Cited by 123 · see citing works
A study in transfer learning: leveraging data from multiple hospitals to enhance hospital-specific predictions
Wiens, J., Guttag, J., & Horvitz, E. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002162 · Cited by 123 · see citing works
Measuring the modified early warning score and the Rothman Index: Advantages of utilizing the electronic medical record in an early warning system
Finlay, G. D., Rothman, M. J., & Smith, R. A. (2013) · Journal of Hospital Medicine · doi:10.1002/jhm.2132 · Cited by 119 · see citing works
A nonparametric updating method to correct clinical prediction model drift
Davis, S. E., Greevy, R., Fonnesbeck, C., Lasko, T., Walsh, C. G., & Matheny, M. E. (2019) · J. Am. Medical Informatics Assoc. · doi:10.1093/jamia/ocz127 · Cited by 112 · see citing works
Impact of an Electronic Clinical Decision Support Tool for Emergency Department Patients With Pneumonia
Dean, N. C., Jones, B. E., Jones, J. P., Ferraro, J. P., Post, H., Aronsky, D., Vines, C. G., Allen, T. L., & Haug, P. J. (2015) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2015.02.003 · Cited by 103 · see citing works
An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis DetectionPreprint
Futoma, J., Hariharan, S., Sendak, M., Brajer, N., Clement, M. E., Bedoya, A., O’Brien, C. L., & Heller, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05894 · Cited by 98 · see citing works
Self-citation
Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices
Abramoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2019) · Yearbook of pediatric endocrinology · doi:10.1530/ey.16.12.1 · Cited by 92 · see citing works
Standardizing the Early Identification of Acute Kidney Injury: The NHS England National Patient Safety Alert
Selby, N. M., Hill, R. G., Fluck, R., & on behalf of the NHS England ‘Think Kidneys'' AKI Programme (2015) · The Nephron journals/Nephron journals · doi:10.1159/000439146 · Cited by 90 · see citing works
Prognostic models will be victims of their own success, unless…
Lenert, M. C., Matheny, M. E., & Walsh, C. G. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz145 · Cited by 90 · see citing works
A longitudinal analysis of data quality in a large pediatric data research network
Khare, R., Utidjian, L., Ruth, B. J., Kahn, M. G., Burrows, E. K., Marsolo, K. A., Patibandla, N., Razzaghi, H., Colvin, R., Ranade, D., et al. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx033 · Cited by 81 · see citing works
Electronic medical record-based multicondition models to predict the risk of 30 day readmission or death among adult medicine patients: validation and comparison to existing models
Amarasingham, R., Velasco, F. T., Xie, B., Clark, C., Ma, Y., Zhang, S., Bhat, D., Lucena, B., Huesch, M. D., & Halm, E. A. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-015-0162-6 · Cited by 71 · see citing works
Patient Risk Stratification for Hospital-Associated C. diff as a Time-Series Classification Task
Wiens, J., Horvitz, E., & Guttag, J. V. (2012) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 68 · see citing works
Performance analysis of a machine learning flagging system used to identify a group of individuals at a high risk for colorectal cancer
Kinar, Y., Akiva, P., Choman, E. N., Kariv, R., Shalev, V., Levin, B., Narod, S. A., & Goshen, R. (2017) · PLoS ONE · doi:10.1371/journal.pone.0171759 · Cited by 64 · see citing works
A Real-Time Early Warning System for Monitoring Inpatient Mortality Risk: Prospective Study Using Electronic Medical Record Data
Ye, C., Wang, O., Liu, M., Zheng, L., Xia, M., Hao, S., Jin, B., Jin, H., Zhu, C., Huang, C.-J., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/13719 · Cited by 62 · see citing works
Real-Time Risk Prediction on the Wards: A Feasibility Study
Kang, M. A., Churpek, M. M., Zadravecz, F. J., Adhikari, R., Twu, N. M., & Edelson, D. P. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000001716 · Cited by 61 · cited 1× · see citing works
“90 Although represented as a single arrow, the ‘Clinical Integration’ step (Step 2c) can often be the most difficult step in the entire translational path.”
Background
Development, Validation and Deployment of a Real Time 30 Day Hospital Readmission Risk Assessment Tool in the Maine Healthcare Information Exchange
Hao, S., Wang, Y., Jin, B., Shin, A. Y., Zhu, C., Huang, M., Zheng, L., Luo, J. J., Hu, Z., Fu, C., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0140271 · Cited by 58 · see citing works
Evaluation of a prediction model for colorectal cancer: retrospective analysis of 2.5 million patient records
Birks, J., Bankhead, C. R., Holt, T., Fuller, A., & Patnick, J. (2017) · Cancer Medicine · doi:10.1002/cam4.1183 · Cited by 55 · see citing works
Challenges and opportunities in software-driven medical devices
Gordon, W. J., & Stern, A. D. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0426-z · Cited by 51 · see citing works
Tutorial: Safe and Reliable Machine LearningPreprint
Saria, S., & Subbaswamy, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07204 · Cited by 46 · see citing works
Stealth research: Lack of peer‐reviewed evidence from healthcare unicorns
Cristea, I. A., Cahan, E. M., & Ioannidis, J. P. A. (2019) · European Journal of Clinical Investigation · doi:10.1111/eci.13072 · Cited by 45 · cited 1× · see citing works
“99 While stealth science is not uncommon amongst biomedical innovations, lack of transparency is particularly concerning with machine learning.”
Background
Barriers to Achieving Economies of Scale in Analysis of EHR Data: A Cautionary Tale
Sendak, M. P., Balu, S., & Schulman, K. A. (2017) · Applied Clinical Informatics · doi:10.4338/aci-2017-03-cr-0046 · Cited by 45 · see citing works
Self-citation
Evaluation of a digitally-enabled care pathway for acute kidney injury management in hospital emergency admissions
Connell, A., Montgomery, H., Martin, P., Nightingale, C., Sadeghi‐Alavijeh, O., King, D., Karthikesalingam, A., Hughes, C., Back, T., Ayoub, K., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0100-6 · Cited by 42 · see citing works
CDS in a Learning Health Care System: Identifying Physicians' Reasons for Rejection of Best-Practice Recommendations in Pneumonia through Computerized Clinical Decision Support
Jones, B. E., Collingridge, D. S., Vines, C. G., Post, H., Holmen, J. R., Allen, T., Haug, P. J., Weir, C. R., & Dean, N. C. (2019) · Applied Clinical Informatics · doi:10.1055/s-0038-1676587 · Cited by 35 · see citing works
Novel Approach to Inpatient Fall Risk Prediction and Its Cross-Site Validation Using Time-Variant Data
Cho, I., Boo, E.-H., Chung, E., Bates, D. W., & Dykes, P. C. (2018) · Journal of Medical Internet Research · doi:10.2196/11505 · Cited by 27 · see citing works
Clinical considerations when applying machine learning to decision-support tasks versus automation
Jamieson, T., & Avi Goldfarb, B. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2019-009514 · Cited by 24 · see citing works
Computer-Assisted Flagging of Individuals at High Risk of Colorectal Cancer in a Large Health Maintenance Organization Using the ColonFlag Test
Goshen, R., Choman, E. N., Ran, A., Muller, E., Kariv, R., Chodick, G., Ash, N., Narod, S. A., & Shalev, V. (2018) · JCO Clinical Cancer Informatics · doi:10.1200/cci.17.00130 · Cited by 23 · see citing works
Performance and Utilization of an Emergency Department Electronic Screening Tool for Pneumonia
Dean, N. C., Jones, B. E., Ferraro, J. P., Vines, C. G., & Haug, P. J. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.3299 · Cited by 23 · see citing works
Implementation of Real-Time Electronic Clinical Decision Support for Emergency Department Patients with Pneumonia Across a Healthcare System.
Dean, N. C., Vines, C. G., Rubin, J., Collingridge, D. S., Mankivsky, M., Srivastava, R., Jones, B. E., Kuttler, K. G., Walker, M., Jenson, N., et al. (2019) · PubMed Central · Cited by 10 · see citing works
An investigation of sepsis surveillance and emergency treatment on patient mortality outcomes: An observational cohort study
Amland, R. C., & Sutariya, B. B. (2018) · JAMIA Open · doi:10.1093/jamiaopen/ooy013 · Cited by 7 · see citing works
A Comparison Of Data Quality Assessment Checks In Six Data Sharing Networks
Callahan, T. J., Bauck, A. E., Bertoch, D., Brown, J. L., Khare, R., Ryan, P. B., Staab, J., Zozus, M. N., & Kahn, M. G. (2017) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1287 · Cited by 7 · see citing works
Data Science for Molecular Diagnostics Applications: From Academia to Clinic to Industry
Riemenschneider, M., Wienbeck, J., Scherag, A., & Heider, D. (2018) · Systems Medicine · doi:10.1089/sysm.2018.0002 · Cited by 6 · cited 1× · see citing works
“The review builds upon related work that highlights how academic and industry partners collaborate to develop machine learning products, 12 as well as the need for engagement from front-line clinicians and standard reporting.”
Background
Preventing Unrecognized Deterioration and Honoring Patients’ Goals of Care by Embedding an Automated Early-Warning System in Hospital Workflows
Dummett, B. A. (2018) · The Permanente Journal · doi:10.7812/tpp/18-071-09 · Cited by 2 · see citing works
Changes in Nursing Activity After Implementing a CDS Service Predicting the Risk of Falling Based on Electronic Medical Records Data.
Cho, I., & Jin, I. (2019) · AMIA · Cited by 2 · see citing works

Flagged first, then by citations

1 of the 80 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

American Medical Association (AMA). Augmented Intelligence in Health Care. 2018. Available at: https://www. ama-assn.org/system/files/2019-01/ augmented-intelligence-policy-report. pdf. Last accessed: 11 December 2019.
The Rothman Index. PeraHealth Secures $14 Million in Financing. 2019. Available at: https://www.perahealth. com/press-releases/2017/01/ perahealth-secures-14-million-in- financing/. Last accessed: 14 August 2019.
Giannini HM et al. A machine learning algorithm to predict severe sepsis and septic shock. Crit Care Med. 2019;47(11):1485-92.
Corey K et al. Model ensembling vs data pooling: Alternative ways to merge hospital information across sites. Proceedings of Machine Learning for Healthcare. 2019. Available at: https:// static1.squarespace.com/ static/59d5ac1780bd5ef9c396eda6/ t/5d473e91b0f598000 1a24186/1564950161664/Corey.pdf. Last accessed: 11 December 2019.

Sorted by citations

Artificial intelligence in healthcare: transforming the practice of medicine
Bajwa, J., Munir, U., Nori, A., & Williams, B. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2021-0095 · Cited by 1,856 · see citing works
“25 > Economic utility quantifies the net benefit relative to the cost from the investment in the AI system.”
Background
Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI
Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01772-9 · Cited by 1,039 · see citing works
Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence
Collins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852 · see citing works
Machine learning for medical imaging: methodological failures and recommendations for the future
Varoquaux, G. P., & Cheplygina, V. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00592-y · Cited by 720 · see citing works
“DATA, AN IMPERFECT WINDOW ON THE CLINIC Datasets may be biased: re ﬂ ect an application only partly Available datasets only partially re ﬂ ect the clinical situation for a particular medical condition, leading to dataset bias 16 .”
Significance of machine learning in healthcare: Features, pillars and applications
Javaid, M., Haleem, A., Singh, R. P., Suman, R., & Rab, S. (2022) · International Journal of Intelligent Networks · doi:10.1016/j.ijin.2022.05.002 · Cited by 698 · see citing works
Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI
Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070904 · Cited by 665 · see citing works
Artificial Intelligence for Mental Health Care: Clinical Applications, Barriers, Facilitators, and Artificial Wisdom
Lee, E., Torous, J. B., Choudhury, M. D., Depp, C. A., Graham, S. A., Kim, H., Paulus, M. P., Krystal, J. H., & Jeste, D. V. (2021) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2021.02.001 · Cited by 477 · see citing works
“Big data (e.g., EHR, clinical notes, sensor data, social media data) are inherently messy and require considerable transformation before they are usable (105).”
Background
A short guide for medical professionals in the era of artificial intelligence
Meskó, B., & Görög, M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00333-z · Cited by 409 · see citing works
The limits of fair medical imaging AI in real-world generalization
Yang, Y., Zhang, H., Gichoya, J. W., Katabi, D., & Ghassemi, M. (2024) · Nature Medicine · doi:10.1038/s41591-024-03113-4 · Cited by 285 · see citing works
“As artificial intelligence (AI) models are increasingly deployed in real-world clinical settings 1,2 , it is crucial to evaluate not only model performance but also potential biases toward specific demographic groups 3,4 .”
From promise to practice: towards the realisation of AI-informed mental health care
Koutsouleris, N., Hauser, T. U., Skvortsova, V., & De Choudhury, M. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00153-4 · Cited by 251 · see citing works
Integrating a Machine Learning System Into Clinical Workflows: Qualitative Study
Sandhu, S., Lin, A. L., Brajer, N., Sperling, J., Ratliff, W., Bedoya, A. D., Balu, S., O'Brien, C., & Sendak, M. P. (2020) · Journal of Medical Internet Research · doi:10.2196/22421 · Cited by 168 · see citing works
Self-citation
Potential applications and performance of machine learning techniques and algorithms in clinical practice: A systematic review
Nwanosike, E. M., Conway, B. R., Merchant, H. A., & Hasan, S. S. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104679 · Cited by 140 · see citing works
Language models are an effective representation learning technique for electronic health record data
Steinberg, E., Jung, K., Fries, J. A., Corbin, C. K., Pfohl, S. R., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103637 · Cited by 132 · see citing works
Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks
Crossnohere, N. L., Elsaid, M. I., Paskett, J., Bose‐Brill, S., & Bridges, J. F. P. (2022) · Journal of Medical Internet Research · doi:10.2196/36823 · Cited by 131 · see citing works
“For AI, the translational pathway is less well-defined and overseen, but generally includes stages such as development, design, validation, reporting, implementation, and scaling [14].”
Artificial intelligence sepsis prediction algorithm learns to say “I don’t know”
Shashikumar, S. P., Wardi, G., Malhotra, A., & Nemati, S. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00504-6 · Cited by 130 · see citing works
“However, most existing published sepsis prediction models 16,17 are either based on data from a single hospital 5,7 – 10,13,15,18,19 or multiple hospitals from the same healthcare system 11,14 where the care processes are mostly standardized.”
Method
Use of deep learning to develop continuous-risk models for adverse event prediction from electronic health records
Tomašev, N., Harris, N., Baur, S., Mottram, A., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Magliulo, V., et al. (2021) · Nature Protocols · doi:10.1038/s41596-021-00513-5 · Cited by 104 · see citing works
Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework
van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad088 · Cited by 109 · see citing works
Systematic Review of Health Economic Evaluations Focused on Artificial Intelligence in Healthcare: The Tortoise and the Cheetah
Voets, M. M., Veltman, J., Slump, C. H., Siesling, S., & Koffijberg, H. (2022) · Value in Health · doi:10.1016/j.jval.2021.11.1362 · Cited by 105 · see citing works
Translating promise into practice: a review of machine learning in suicide research and prevention
Kirtley, O. J., van Mens, K., Hoogendoorn, M., Kapur, N., & de Beurs, D. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00254-6 · Cited by 100 · see citing works
Artificial intelligence in medical device software and high-risk medical devices – a review of definitions, expert recommendations and regulatory initiatives
Fraser, A. G., Biasin, E., Bijnens, B. H., Bruining, N., Caiani, E. G., Cobbaert, K., Davies, R. H., Gilbert, S., Hovestadt, L., Kamenjašević, E., et al. (2023) · Expert Review of Medical Devices · doi:10.1080/17434440.2023.2184685 · Cited by 103 · see citing works
“Methodological guidelines Park 2020 JAMIA Open [65] Cross-specialty Recommendations for a phased approach to medical AI evaluation Sendak 2020 EMJ Innovation [66] European Medical Journal Cross-specialty Recommendations for translation of ML products into healthcare delivery McCradden 2020 Nature Medicine [67] Cross-s…”
Method
Translation of AI into oncology clinical practice
Naqa, I. E., Karolak, A. M., Luo, Y., Folio, L. R., Tarhini, A. A., Rollison, D. E. M., & Parodi, K. (2023) · Oncogene · doi:10.1038/s41388-023-02826-z · Cited by 83 · see citing works
“● AI gold rush and the lack of distinction among entrepreneurs about requirement differences between a medical AI device and a consumer-tech AI product [52].”
Background
Peeking into a black box, the fairness and generalizability of a MIMIC-III benchmarking model.
Röösli, E., Bozkurt, S., & Hernandez-Boussard, T. (2022) · Scientific Data · doi:10.1038/s41597-021-01110-7 · Cited by 73 · see citing works
Development and Prospective Validation of a Deep Learning Algorithm for Predicting Need for Mechanical Ventilation
Shashikumar, S. P., Wardi, G., Paul, P., Paul, P., Carlile, M., Brenner, L. N., Hibbert, K. A., North, C. M., Mukerji, S. S., Robbins, G. K., et al. (2020) · CHEST Journal · doi:10.1016/j.chest.2020.12.009 · Cited by 70 · see citing works
“Previously, ML algorithms have been implemented into clinical workflow with improved clinical, statistical, or economic usefulness.(38) Additionally, we have included varying prediction windows to illustrate how VentNet performs at various time frames to illustrate potential uses (e-Fig 1).”
Leading with AI in critical care nursing: challenges, opportunities, and the human factor
Hassan, E. A., & El-Ashry, A. M. (2024) · BMC Nursing · doi:10.1186/s12912-024-02363-4 · Cited by 72 · see citing works
“As Sendak et al. (2020) [43] noted, effective AI tools in healthcare should be designed with end-users in mind, ensuring that outputs are easily interpretable and actionable.”
Background
Advancing Artificial Intelligence in Health Settings Outside the Hospital and Clinic
Aggarwal, N., Ahmed, M. K., Basu, S., Curtin, J. J., Evans, B. J., Matheny, M. E., Nundy, S., Sendak, M., Shachar, C., Shah, R. U., et al. (2020) · NAM Perspectives · doi:10.31478/202011f · Cited by 63 · see citing works
“For example, there is ongoing research into a generalizable approach to train site-specifi c Clostridium diffi cile models (a model of within-hospital infection) by which each hospital has a local model [120,121].”
BackgroundSelf-citation
DECIDE-AI: a new reporting guideline and its relevance to artificial intelligence studies in radiology
Vasey, B., Novak, A., Ather, S., Ibrahim, M., & McCulloch, P. G. (2023) · Clinical Radiology · doi:10.1016/j.crad.2022.09.131 · Cited by 58 · see citing works
Digital Twins in Personalized Medicine: Bridging Innovation and Clinical Reality
Silva, A., & Vale, N. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15110503 · Cited by 72 · see citing works
“Even the most advanced DT runs the risk of being overlooked or underutilized in the absence of clinician-centered design [41,42].”
Organizational Governance of Emerging Technologies: AI Adoption in Healthcare
Kim, J. Y., Boag, W., Gulamali, F., Hasan, A., Hogg, H. D. J., Lifson, M., Mulligan, D., Patel, M., Raji, I. D., Sehgal, A., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594089 · Cited by 55 · see citing works
“First, we developed an interview guide and identified which types of internal and external roles to interview based on four relatively distinct stages of AI adoption: (1) problem identification and procurement, (2) development and adaptation, (3) clinical integration, and (4) lifecycle management [28] (Table 4 in Appe…”
MethodSelf-citation
Health Care Professionals’ Experience of Using AI: Systematic Review With Narrative Synthesis
Ayorinde, A., Mensah, D. O., Walsh, J., Ghosh, I., Ibrahim, S. A., Hogg, J., Peek, N., & Griffiths, F. (2024) · Journal of Medical Internet Research · doi:10.2196/55766 · Cited by 55 · see citing works
Development and preliminary testing of health equity across the AI lifecycle (HEAAL): a framework for healthcare delivery organizations to mitigate the risk of AI solutions worsening health inequities
Kim, J. Y., Hasan, A., Kellogg, K. C., Ratliff, W., Murray, S. G., Suresh, H., Valladares, A., Shaw, K., Tobey, D., Vidal, D. E., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000390 · Cited by 61 · see citing works
“However, the proliferation of healthcare AI tools has outpaced regulatory frameworks, accountability measures, and governance standards to ensure safe, effective, and equitable use [3,9,10].”
BackgroundSelf-citation
Real-world implementation of precision psychiatry: Transdiagnostic risk calculator for the automatic detection of individuals at-risk of psychosis
Oliver, D., Spada, G., Colling, C., Broadbent, M., Baldwin, H., Patel, R., Stewart, R., Stahl, D., Dobson, R., McGuire, P., et al. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2020.05.007 · Cited by 49 · see citing works
“For example, since most risk prediction models are developed on “artificial” retrospective datasets, the in-vitro phase seems particularly suited to address barriers relating to accessibility of predictors, outcome data and model refinements that are needed to use risk calculators in real-world EHRs prospectively (Bou…”
Attitudes Toward the Adoption of 2 Artificial Intelligence–Enabled Mental Health Tools Among Prospective Psychotherapists: Cross-sectional Study
Kleine, A.-K., Kokje, E., Lermer, E., & Gaube, S. (2023) · JMIR Human Factors · doi:10.2196/46859 · Cited by 49 · see citing works
“In spite of the growing efforts to create user-friendly artificial intelligence (AI) applications, their use in clinical care remains limited [1].”
Background
A normative framework for artificial intelligence as a sociotechnical system in healthcare
McCradden, M. D., Joshi, S. D., Anderson, J. A., & London, A. J. (2023) · Patterns · doi:10.1016/j.patter.2023.100864 · Cited by 49 · see citing works
PI Prob: A risk prediction and clinical guidance system for evaluating patients with recurrent infections
Rider, N. L., Cahill, G., Motazedi, T., Wei, L., Kurian, A., Noroski, L. M., Seeborg, F. O., Chinn, I. K., & Roberts, K. E. (2021) · PLoS ONE · doi:10.1371/journal.pone.0237285 · Cited by 49 · see citing works
“Lastly, we wish to underscore that our view of AI’s role in healthcare is that it should be rigorously validated but dutifully implemented to augment clinical decision making and extend clinical efforts for making use of big data in a way that serves the patient and provider [37,38].”
Background
Machine learning in transfusion medicine: A scoping review
Maynard, S., Farrington, J., Alimam, S., Evans, H. G., Li, K., Wong, W. K., & Stanworth, S. J. (2023) · Transfusion · doi:10.1111/trf.17582 · Cited by 46 · see citing works
Clinical use of artificial intelligence requires AI-capable organizations
Novak, L. L., Russell, R. G., Garvey, K., Patel, M., Thomas Craig, K. J., Snowdon, J., & Miller, B. (2023) · JAMIA Open · doi:10.1093/jamiaopen/ooad028 · Cited by 43 · see citing works
The unmet promise of trustworthy AI in healthcare: why we fail at clinical translation
Bürger, V. K., Amann, J., Bui, C. K. T., Fehr, J., & Madai, V. I. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1279629 · Cited by 44 · see citing works
Towards a Framework for Evaluating the Safety, Acceptability and Efficacy of Ai Systems for Health: An Initial SynthesisPreprint
Morley, J., Morton, C. E., Karpathakis, K., Taddeo, M., & Floridi, L. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3826358 · Cited by 46 · see citing works
A simulation-based evaluation of machine learning models for clinical decision support: application and analysis using hospital readmission
Mišić, V. V., Rajaram, K., & Gabel, E. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00468-7 · Cited by 37 · see citing works
“…recently proposed frameworks for translating machine learning into clinical practice have high-lighted the demonstration of economic utility 20 and rigorous evaluation of impact in terms of clinical outcomes and cost 21 as important steps; such rigorous evaluation is likely to be of…”
Application of Artificial Intelligence Methods on Osteoporosis Classification with Radiographs—A Systematic Review
Liu, R. W., Ong, W., Makmur, A., Kumar, N., Low, X. Z., Ge, S., Liang, T. Y., Ting, D. F. K., Tan, J. H., & Hallinan, J. T. P. D. (2024) · Bioengineering · doi:10.3390/bioengineering11050484 · Cited by 32 · see citing works
“Several systems have demonstrated diagnostic capabilities comparable to medical professionals in various clinical conditions [36], with many software applications being approved for clinical use [37,38].”
We're implementing AI now, so why not ask us what to do? – How AI providers perceive and navigate the spread of diagnostic AI in complex healthcare systems
Gillner, S. (2024) · Social Science & Medicine · doi:10.1016/j.socscimed.2023.116442 · Cited by 30 · see citing works
Designing a Clinical Decision Support Tool That Leverages Machine Learning for Suicide Risk Prediction: Development Study in Partnership With Native American Care Providers
Haroz, E. E., Grubin, F., Goklish, N. A., Pioche, S., Cwik, M. F., Barlow, A., Waugh, E., Usher, J., Lenert, M. C., & Walsh, C. G. (2021) · JMIR Public Health and Surveillance · doi:10.2196/24377 · Cited by 27 · see citing works
“Despite robust interest, machine learning models have rarely been translated into clinical care [21].”
Background
Artificial intelligence perspective in the future of endocrine diseases
Hasanzad, M., Meybodi, H. R. A., Sarhangi, N., & Larijani, B. A. (2022) · Journal of Diabetes & Metabolic Disorders · doi:10.1007/s40200-021-00949-2 · Cited by 30 · see citing works
“But it should meet the standards and requirements of evidence-based medicine [11, 12].”
Background
Harnessing AI for enhanced evidence-based laboratory medicine (EBLM)
Pillay, T. S., Topçu, D. İ., & Yenice, S. (2025) · Clinica Chimica Acta · doi:10.1016/j.cca.2025.120181 · Cited by 31 · see citing works
The Use of Artificial Intelligence for Cancer Therapeutic Decision-Making
Elemento, O., Khozin, S., & Sternberg, C. N. (2025) · NEJM AI · doi:10.1056/aira2401164 · Cited by 28 · see citing works
Appropriate use of machine learning in healthcare
Ozaydin, B., Berner, E. S., & Cimino, J. J. (2021) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2021.100041 · Cited by 27 · see citing works
Investigating psychotherapists’ attitudes towards artificial intelligence in psychotherapy
Wagner, J., & Schwind, A.-S. (2025) · BMC Psychology · doi:10.1186/s40359-025-03071-7 · Cited by 29 · see citing works
“However, a significant gap remains between AI’s potential and its practical implementation [53].”
Governance for safe and responsible AI in healthcare organisations: a scoping review of frameworks
Wang, A., Freeman, S., & Magrabi, F. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02679-2 · Cited by 22 · see citing works
“…performance against a baseline or gold standard on statistical outcomes like accuracy, sensitivity, speci ﬁ city, generalisability, and reliability, as well as clinical outcomes, usability and adoption, cost-analyses, and ethics analyses (e.g. fairness evaluation) 33,40,50,52,59,61,64,65,71 – 73 .”
Implementation of prognostic machine learning algorithms in paediatric chronic respiratory conditions: a scoping review
Filipow, N., Main, E., Sebire, N. J., Booth, J., Taylor, A. M., Davies, G., & Stanojevic, S. (2022) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2021-001165 · Cited by 22 · see citing works
A Comprehensive Survey of Machine Learning in Healthcare: Predicting Heart and Liver Disease, Tuberculosis Detection in Chest X-Ray Images
Banapuram, C., Naik, A. C., Vanteru, M. K., Kumar, S., V, & Vaigandla, K. K. (2024) · International Journal of Electronics and Communication Engineering · doi:10.14445/23488549/ijece-v11i5p116 · Cited by 21 · see citing works
“The potential ramifications of ML techniques are considerable since they can enhance the overall well-being of a substantial global population [5-7].”
Structural equation modeling for influencing factors on behavioral intention to adopt medical AI among Chinese nurses: a nationwide cross-sectional study
Dai, Q., Li, M., Shi, S., Yang, M., Wang, Z., Liao, J., Li, Z., Liu, J., Deng, J. J., & Tao, L. (2025) · BMC Nursing · doi:10.1186/s12912-025-03748-9 · Cited by 25 · see citing works
“Effective AI tools in healthcare must prioritize end-user needs to ensure actionable and comprehensible outcomes [60].”
The Role and Impact of Artificial Intelligence in Addressing Sexually Transmitted Infections, Nonvenereal Genital Diseases, Sexual Health, and Wellness
Mehta, N., Gupta, S., & Kularathne, Y. (2023) · Indian Dermatology Online Journal · doi:10.4103/idoj.idoj_426_23 · Cited by 21 · see citing works
“[43] Ways to ensure the swift translation from research to practical use include standardization of regulatory frameworks and guidelines enabling to assess them as per established parameters and better regulatory approaches than classifying these models as medical devices (software as a Medical…”
Method
Building competency in artificial intelligence and bias mitigation for nurse scientists and aligned health researchers
Cary, M. P., Grady, S. D., McMillian-Bohler, J. M., Bessias, S., Silcox, C., Silva, S. G., Guilamo‐Ramos, V., McCall, J., Sperling, J. B., & Goldstein, B. A. (2025) · Nursing Outlook · doi:10.1016/j.outlook.2025.102395 · Cited by 21 · see citing works
Interpretable Machine Learning Models for Clinical Decision-Making in a High-Need, Value-Based Primary Care Setting
Bhatt, S. A., Cohon, A., Rose, J., Majerczyk, N., Cozzi, B., Crenshaw, D., & Myers, G. (2021) · NEJM Catalyst · doi:10.1056/cat.21.0008 · Cited by 20 · see citing works
Artificial intelligence and clinical deterioration
Malycha, J., Bacchi, S., & Redfern, O. (2022) · Current Opinion in Critical Care · doi:10.1097/mcc.0000000000000945 · Cited by 20 · see citing works
Development of an automated, general-purpose prediction tool for postoperative respiratory failure using machine learning: A retrospective cohort study
Kiyatkin, M. E., Aasman, B., Fazzari, M. J., Rudolph, M. I., Melo, M. F. V., Eikermann, M., & Gong, M. N. (2023) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2023.111194 · Cited by 21 · see citing works
Expert gaze as a usability indicator of medical AI decision support systems: a preliminary study
Castner, N., Arsiwala-Scheppach, L. T., Mertens, S., Krois, J., Thaqi, E., Kasneci, E., Wahl, S., & Schwendicke, F. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01192-8 · Cited by 20 · see citing works
“6,28 – 30).”
Development and validation of interpretable machine learning models for inpatient fall events and electronic medical record integration
Shim, S., Yu, J. Y., Jekal, S. Y., Song, Y. J., Moon, K. T., Lee, J. H., Yeom, K. M., Park, S. H., Cho, I., Song, M. R., et al. (2022) · Clinical and Experimental Emergency Medicine · doi:10.15441/ceem.22.354 · Cited by 19 · see citing works
“” Another challenge is that previous models have not been well-prepared for electronic medical record (EMR) integration for practical application.(15,16) To solve these limitations, the authors developed a model for predicting falls using interpretable ML and integrating the model into the EMR system to perform nursin…”
Background
Multi-scale digital twins for personalized medicine
Vallée, A. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1753906 · Cited by 21 · see citing works
“…trials may be insufﬁcient to assess continuously updating MSDTs. Alternative approaches, such as digital twin randomized simulations, real-world evidence collection, and adaptive trial designs, must be developed and accepted by regulatory authorities to ensure rigorous yet flexible evaluation (33).”
A scoping review of silent trials for medical artificial intelligence
Tikhomirov, L., Semmler, C., Prizant, N., Bhasin, S., Kenyon, G., van der Vegt, A., Erdman, L., Kurian, N. C., Thompson, H., Palmer, L. J., et al. (2026) · Nature Health · doi:10.1038/s44360-025-00048-z · Cited by 17 · see citing works
Self-citation
Computer-Based Diagnosis of Celiac Disease by Quantitative Processing of Duodenal Endoscopy Images
Molder, A., Balaban, D. V., Molder, C. C., Jinga, M., & Robin, A. (2023) · Diagnostics · doi:10.3390/diagnostics13172780 · Cited by 19 · see citing works
“medical experts [21] and software applications are starting to be certified for clinical use [22,23].”
Background
Prediction of evening fatigue severity in outpatients receiving chemotherapy: less may be more
Kober, K. M., Roy, R., Dhruva, A., Conley, Y. P., Chan, R. J., Cooper, B. A., Olshen, A. B., & Miaskowski, C. A. (2021) · Fatigue Biomedicine Health & Behavior · doi:10.1080/21641846.2021.1885119 · Cited by 16 · see citing works
“of ML models are challenging [97], our findings suggest that the language used to assess fatigue is important.”
Background
Strengthening the use of artificial intelligence within healthcare delivery organizations: balancing regulatory compliance and patient safety
Sendak, M., Liu, V. X., Beecy, A., Vidal, D. E., Shaw, K., Lifson, M. A., Tobey, D., Valladares, A., Loufek, B. T., Mogri, M., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae119 · Cited by 14 · see citing works
Self-citation
Rethinking model prototyping through the MedMNIST+ dataset collection
Doerrich, S., Salvo, F. D., Brockmann, J., & Ledig, C. (2025) · Scientific Reports · doi:10.1038/s41598-025-92156-9 · Cited by 14 · see citing works
“Simultaneously, there is an accumulating interest in integrating machine learning techniques into medical imaging, where DL models are approaching performance comparable to medical experts on specific tasks 7 and software applications are beginning to receive clinical certifications 8 .”
Leveraging machine learning for enhanced and interpretable risk prediction of venous thromboembolism in acute ischemic stroke care
Jiang, Y., Li, A., Li, Z., Li, Y., Li, R., Zhao, Q., & Li, G. (2025) · PLoS ONE · doi:10.1371/journal.pone.0302676 · Cited by 14 · see citing works
“The model’s predictions can also inform multidisciplinary discussions, improving treatment planning and resource allocation [37].”
Prediction of clinical outcomes after percutaneous coronary intervention: Machine-learning analysis of the National Inpatient Sample
Galimzhanov, A. M., Matetić, A., Tenekecioğlu, E., & Mamas, M. A. (2023) · International Journal of Cardiology · doi:10.1016/j.ijcard.2023.131339 · Cited by 12 · see citing works
How Artificial Intelligence is Transforming Medicine: The Future of Pharmaceutical Research
Sharma, P., Jain, V., & Tailang, M. (2023) · Advanced technologies and societal change · doi:10.1007/978-981-99-1949-9_7 · Cited by 12 · see citing works
Advancements in automated classification of chronic obstructive pulmonary disease based on computed tomography imaging features through deep learning approaches
Zhu, Z. (2024) · Respiratory Medicine · doi:10.1016/j.rmed.2024.107809 · Cited by 12 · see citing works
Artificial Intelligence in Healthcare
Naqvi, G., Nasir, T., Azam, H., & Zafar, L. (2023) · Pakistan Journal of Humanities and Social Sciences · doi:10.52131/pjhss.2023.1102.0443 · Cited by 13 · see citing works
Visualizing machine learning-based predictions of postpartum depression risk for lay audiences
Desai, P., Harkins, S., Rahman, S., Kumar, S., Hermann, A. D., Joly, R., Zhang, Y., Pathak, J., Kim, J., D’Angelo, D., et al. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad198 · Cited by 10 · see citing works
Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record DataPreprint
Steinberg, E., Jung, K., Fries, J., Corbin, C. K., Pfohl, S. R., & Shah, N. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05295 · Cited by 10 · see citing works
Empowering photodynamic therapy with artificial intelligence: current trends and future directions
Paul, A., Xavierselvan, M., Aebisher, D., Kubrak, T. P., Bartusik‐Aebisher, D., & Mallidi, S. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1771804 · Cited by 8 · see citing works
“Developing such integrated systems demands interdisciplinary collaboration among AI researchers, biomedical engineers, clinical physicists, and treating physicians that remains challenging to coordinate (248).”
BEDS-Bench: Behavior of EHR-models under Distributional Shift--A BenchmarkPreprint
Avati, A., Seneviratne, M. G., Xue, E., Xu, Z., Lakshminarayanan, B., & Dai, A. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08189 · Cited by 8 · see citing works
Precision strength training: Data-driven artificial intelligence approach to strength and conditioningPreprint
Teikari, P., & Pietrusz, A. (2021) · doi:10.31236/osf.io/w734a · Cited by 11 · see citing works
Development and external validation of a machine learning model for predicting drug-induced immune thrombocytopenia in a real-world hospital cohort
Dung, H. V., Tan, V. M., Ðiê̹u, N. T., Linh, P. V., Khai, N. V., Ngan, T. T., & Phương, N. T. T. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03107-3 · Cited by 7 · see citing works
“Such systems have demonstrated clinical benefit in prior studies involving adverse drug reaction prediction [26, 35].”
Artificial Intelligence in Healthcare: Lost In Translation?Preprint
Madai, V. I., & Higgins, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.13454 · Cited by 7 · see citing works
“This path from early-stage PoC to a ﬁnal medical product is referred to as translation [9, 10].”
Encouraging dissemination of research on the use of artificial intelligence and related innovative technologies in clinical pharmacy practice and education: call for papers
Hoti, K., & Weidmann, A. E. (2024) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-024-01777-z · Cited by 7 · see citing works
“…learning driven clinical decision support systems (CDSS) based on electronic health records are tested for use in predicting patient outcomes [10], personalizing treatment plans, identifying adverse drug reactions [11, 12], predicting potential medication-related problems before they occur…”
A Governance Framework for the Implementation and Operation of AI Applications in a University Hospital
Apfelbacher, T., Koçman, S. E., Prokosch, H., & Christoph, J. (2024) · Studies in health technology and informatics · doi:10.3233/shti240527 · Cited by 6 · see citing works
Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
Wang, S. M., Hogg, H. D. J., Sangvai, D. G., Patel, M. R., Weissler, E. H., Kellogg, K. C., Ratliff, W., Balu, S., & Sendak, M. (2023) · JMIR Formative Research · doi:10.2196/43963 · Cited by 6 · see citing works
“However, this suggests that high-resource health care delivery settings that bring together clinical and technical experts to integrate ML-driven CDS are most likely to succeed [48].”
Self-citation
From Clinic to Computer and Back Again: Practical Considerations When Designing and Implementing Machine Learning Solutions for Pediatrics
Nagaraj, S., Harish, V., McCoy, L. G., Morgado, F., Stedman, I., Lu, S. M., Drysdale, E., Brudno, M., & Singh, D. A. (2020) · Current Treatment Options in Pediatrics · doi:10.1007/s40746-020-00205-4 · Cited by 6 · see citing works
Data Management and Network Architecture Effect on Performance Variability in Direct Attenuation Correction via Deep Learning for Cardiac SPECT: A Feasibility Study
Torkaman, M., Yang, J., Shi, L., Wang, R., Miller, E. J., Sinusas, A. J., Liu, C., Gullberg, G. T., & Seo, Y. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3138372 · Cited by 6 · see citing works
“However, despite all the efforts and enthu-siasm, deep learning models are rarely translated into clinical care and the evidence of clinical impact remains limited [12].”
Background
Using Cough Sounds for the Recognition of Multiclass Respiratory Diseases With Artificial Intelligence: A Review
Zhang, P.-F., Ting, H.-N., & Chang, S. (2025) · IEEE Access · doi:10.1109/access.2025.3584401 · Cited by 4 · see citing works
“To address these risks, defined implementation pathways [209], [210] and a three-stage roadmap for clinical integration of AI have been proposed [211], [212], alongside guidelines that promote standardized reporting [207], [213], [214], [215], [216], reproducibility, and reliability in healthcare…”
Bridging the governance gap in health artificial intelligence: integrating nursing perspectives into the National Academy of Medicine Artificial Intelligence Code of Conduct
Cary, M. P., Russell, R. G., Silcox, C., Lytle, K. S., Lehmann, L. S., Hightower, M., Economou-Zavlanos, N., & Guilamo‐Ramos, V. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf224 · Cited by 4 · see citing works
Forecasting Erroneous Neural Machine Translation of Disease Symptoms: Development of Bayesian Probabilistic Classifiers for Cross-Lingual Health Translation
Ji, M., Xie, W., Huang, R., & Qian, X. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18189873 · Cited by 4 · see citing works
Algorithmic Fairness in Chest X-ray Diagnosis: A Case Study
Zhang, H., Hartvigsen, T., & Ghassemi, M. (2023) · doi:10.21428/2c646de5.816cae0e · Cited by 15 · see citing works
Artificial Intelligence in Predicting, Diagnosing and Preventing Sexually Transmitted Infections (STIs)
Taneja, J., Ghosh, J., Kant, R., & Christodoulides, M. (2025) · Venereology · doi:10.3390/venereology4020005 · Cited by 4 · see citing works
“Sendak et al. [8] emphasized the importance of rigorous validation studies with diverse populations, user-centered design principles and continuous performance monitoring, which are essential conditions for the responsible deployment of AI/ML in sexual and reproductive health.”
Systems Thinking Approach to an Artificial Intelligence Reality within Healthcare: From Hype to Value
Strachna, O., & Asan, O. (2021) · IEEE International Symposium on Systems Engineering · doi:10.1109/isse51541.2021.9582546 · Cited by 4 · see citing works
Machine Learning for Smart Health Services in the Framework of Industry 5.0
Kumar, N., Tripathi, P., Nanda, R. P., Tiwari, S., & Sharma, S. (2024) · Advances in web technologies and engineering book series · doi:10.4018/979-8-3693-0782-3.ch013 · Cited by 4 · see citing works
Enhancing Obstetric Decision-Making With AI: A Systematic Review of AI Models for Predicting Mode of Delivery
Elhabeeb, S. M. A., Ali, S. H. M., Babikir, M. M. A. E., Mohammed, F. S. A., Ali, S. H. M., Mohamed, N. A. A. E., & Elsheikh, N. E. A. (2025) · Cureus · doi:10.7759/cureus.83655 · Cited by 5 · see citing works
“Prior systematic reviews by Sendak et al. [33] and Nagendran et al. [34] have emphasized the importance of external validation to ensure that AI models maintain their performance across different patient populations, clinical workflows, and healthcare systems.”
Improving data processing in medical education through machine learning
Shyndaliyev, N. T., Ainur, O., Shadinova, K., Barakova, A., & Nurmukhanbetova, N. (2025) · CARDIOVASCULAR THERAPY AND PREVENTION · doi:10.15829/1728-8800-2025-4446 · Cited by 3 · see citing works
The impact of feature combinations on machine learning models for in-hospital mortality prediction
Stenwig, E., Rossi, P. S., Salvi, G., & Skjærvold, N. K. (2025) · Scientific Reports · doi:10.1038/s41598-025-26611-y · Cited by 4 · see citing works
Health economics evaluation of artificial intelligence in the field of oncology: a scoping review
Tun, H. M., Rahman, H. A., Naing, L., & Malik, O. A. (2026) · Health Economics Review · doi:10.1186/s13561-026-00751-x · Cited by 6 · see citing works
“In LMICs, AI technologies in oncology can improve two main issues: addressing healthcare personnel shortages and overcoming barriers resulting from limited medical equipment [38, 43, 44].”
How deep learning influences workflows and roles in virtual surgical planning
Hofer, B., Kittler, M. G., & Laukens, K. (2023) · Discover Health Systems · doi:10.1007/s44250-023-00041-x · Cited by 3 · see citing works
“The translational path remains unclear [40].”
The current state of digital manikins to support pain self-reporting: a systematic literature review
Ali, S. M., Mountain, D. C., Lee, R. R., Murphy, D., Chiarotto, A., Wong, D., Dixon, W. G., & van der Veer, S. N. (2025) · PAIN Reports · doi:10.1097/pr9.0000000000001274 · Cited by 3 · see citing works
AI-DRIVEN CLINICAL DECISION SUPPORT SYSTEMS: EVALUATING IMPACT ON DIAGNOSIS AND TREATMENT ACCURACY
Mittal, M., & Raja, V. A. J. (2025) · doi:10.34218/ijeaai_03_01_002 · Cited by 3 · see citing works
Machine learning for predictive analytics
Kashyap, S., Corey, K., Kansal, A., & Sendak, M. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-820273-9.00003-8 · Cited by 3 · see citing works
Self-citation
Data-driven approaches to generating knowledge: Machine learning, artificial intelligence, and predictive modeling
Matheny, M. E., Lucila Ohno-machado, A., Davis, S. E., & Nemati, S. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-91200-6.00031-0 · Cited by 3 · see citing works
SurgeryLSTM: a time-aware neural model for accurate and explainable length of stay prediction after spine surgery
Cho, H. N., Sutari, S., Lopez, A. M., Bow, H. C., & Zheng, K. W. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf079 · Cited by 2 · see citing works
Performance of machine learning algorithms in predicting the need for surgical fixation in pediatric craniomaxillofacial trauma
Thornton, B., Patel, K. P., Ma, B., & Castro-Núñez, J. A. (2025) · International Journal of Oral and Maxillofacial Surgery · doi:10.1016/j.ijom.2025.10.005 · Cited by 2 · see citing works
Automated detection of primary soft tissue sarcomas of the extremities using artificial intelligence and ChatGPT
Voigtländer, H., Schmitz, F., Strauss, D. S., Kauczor, H. ‐., Voigtlander, S., Sauerwein, S., & Sedaghat, S. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1674509 · Cited by 2 · see citing works

Machine learning in translation
(2021) · Nature Biomedical Engineering · doi:10.1038/s41551-021-00758-1 · Cited by 7 · see citing works
Do no harm: a roadmap for responsible machine learning for health care
Wiens, J., Saria, S., Sendak, M., Ghassemi, M., Liu, V. X., Doshi-Velez, F., Jung, K., Heller, K., Kale, D., Saeed, M., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0548-6 · Cited by 1,266 · see citing works
Self-citation
Machine Learning in Health Care: A Critical Appraisal of Challenges and Opportunities
Sendak, M., Gao, M. C., Nichols, M., Lin, A. L., & Balu, S. (2019) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.287 · Cited by 87 · see citing works
Self-citation
How to develop machine learning models for healthcare
Chen, P.-H. C., Liu, Y., & Peng, L. (2019) · Nature Materials · doi:10.1038/s41563-019-0345-0 · Cited by 274 · see citing works
Deploying clinical machine learning? Consider the following...Preprint
Lu, C., Chang, K., Singh, P., Pomerantz, S. R., Doyle, S., Kakarmath, S. S., Bridge, C. P., & Kalpathy–Cramer, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06919 · Cited by 3 · see citing works

Authors: Sendak, M., D’Arcy, J., Kashyap, S., Gao, M. C., Nichols, M., Corey, K., Ratliff, W., & Balu, S.
Affiliations: Duke University (US)
ORCID: Mark Sendak
Sehj Kashyap
Michael C. Gao
Marshall Nichols
Kristin Corey
Suresh Balu
Venue: EMJ Innovations
Publisher: European Medical Group
Published: 2020-01-27
ISSN: 2513-8634 (ISSN-L)
DOI: 10.33590/emjinnov/19-00172
OpenAlex: W3201352722
Cited by: 172
References: 80
Citations by year: 2026: 46 · 2025: 40 · 2024: 18 · 2023: 24 · 2022: 14 · 2021: 22 · 2020: 8
FWCI: 5.01
Type: journal article
Language: en
Open access: diamond
doi.org
Licence: cc-by-nc
Locations: EMJ Innovation (publishedVersion, cc-by-nc)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders: University of Washington
Topics: Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords: clinical decision support, electronic health records, clinical implementation, model validation, health technology adoption, health system leadership

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Path for Translation of Machine Learning Products into Healthcare Delivery

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks