Calibration drift in regression and machine learning models for acute kidney injury
Open accessAbstract
Objective: Predictive analytics create opportunities to incorporate personalized risk estimates into clinical decision support. Models must be well calibrated to support decision-making, yet calibration deteriorates over time. This study explored the influence of modeling methods on performance drift and connected observed drift with data shifts in the patient population. Materials and Methods: Using 2003 admissions to Department of Veterans Affairs hospitals nationwide, we developed 7 parallel models for hospital-acquired acute kidney injury using common regression and machine learning methods, validating each over 9 subsequent years. Results: Discrimination was maintained for all models. Calibration declined as all models increasingly overpredicted risk. However, the random forest and neural network models maintained calibration across ranges of probability, capturing more admissions than did the regression models. The magnitude of overprediction increased over time for the regression models while remaining stable and small for the machine learning models. Changes in the rate of acute kidney injury were strongly linked to increasing overprediction, while changes in predictor-outcome associations corresponded with diverging patterns of calibration drift across methods. Conclusions: Efficient and effective updating protocols will be essential for maintaining accuracy of, user confidence in, and safety of personalized risk predictions to support decision-making. Model updating protocols should be tailored to account for variations in calibration drift across methods and respond to periods of rapid performance drift rather than be limited to regularly scheduled annual or biannual intervals.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ongoing monitoring is essential because the performance and effect(s) of AI models can change after deployment.
“Ongoing monitoring is essential because the performance and effect(s) of AI models can change after deployment 27,28 .”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [27]
The Discussion (2012 – Q4) recommends repeated validation and monitoring because model calibration can drift over time. It does not establish that the effects of deployed AI models change; the limitations section says the effect of calibration drift on clinical utility remains uncertain. (LLM, conf=0.86)
Experimental — accuracy not yet measuredProprietary models are difficult to assess owing to their opaque nature and have been shown to decline in performance over time.
“This limited information is of concern because proprietary models are difficult to assess owing to their opaque nature and have been shown to decline in performance over time.15,16”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [16]
The Results (passages [1] and [2]) report that calibration deteriorated over nine years in the study’s AKI prediction models, supporting the performance-over-time part of the claim. The paper does not establish that proprietary models are difficult to assess because they are opaque; it also reports that discrimination was largely maintained. (LLM, conf=0.87)
Experimental — accuracy not yet measuredCalibration failure is structural and progressive rather than sporadic.
“This extends prior reports of temporal miscalibration [32–34] by showing calibration failure is structural and progressive rather than sporadic.”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · see citing works
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957 · see citing works
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537 · see citing works
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287 · see citing works
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648 · see citing works
KDIGO Clinical Practice Guidelines for Acute Kidney Injury
Khwaja, A. (2012) · Nephron Clinical Practice · doi:10.1159/000339789 · Cited by 8,039 · see citing works
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.2307/1271436 · Cited by 6,553 · see citing works
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170 · see citing works
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · see citing works
Acute Renal Failure in Critically Ill Patients A Multinational, Multicenter Study
Uchino, S. (2005) · JAMA · doi:10.1001/jama.294.7.813 · Cited by 4,183 · see citing works
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281 · see citing works
Meinshausen, N., & Bühlmann, P. (2010) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2010.00740.x · Cited by 2,314 · see citing works
Risk Prediction Models for Hospital Readmission
Kansagara, D. L., Englander, H. L., Salanitro, A. H., Kagen, D. I., Theobald, C. N., Freeman, M., & Kripalani, S. B. (2011) · JAMA · doi:10.1001/jama.2011.1515 · Cited by 1,712 · see citing works
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508 · see citing works
Steyerberg, E. W. (2009) · Statistics for Biology and Health · doi:10.1007/978-0-387-77244-8 · Cited by 1,491 · see citing works
Statistical modeling: The two cultures
Breiman, L. (2001) · Cited by 1,314 · see citing works
Risk prediction models: II. External validation, model updating, and impact assessment
Moons, K. G. M., Kengne, A. P., Grobbee, D. E., Royston, P., Vergouwe, Y., Altman, D. G., & Woodward, M. (2012) · Heart · doi:10.1136/heartjnl-2011-301247 · Cited by 1,195 · see citing works
Coca, S. G., Yusuf, B., Shlipak, M. G., Garg, A. X., & Parikh, C. R. (2009) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2008.11.034 · Cited by 1,156 · see citing works
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934 · see citing works
A calibration hierarchy for risk models was defined: from utopia to empirical data
Van Calster, B., Nieboer, D., Vergouwe, Y., De Cock, B., Pencina, M. J., & Steyerberg, E. W. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.005 · Cited by 910 · see citing works
Brivet, F. G., Kleinknecht, D., Loirat, P., & Landais, P. (1996) · Critical Care Medicine · doi:10.1097/00003246-199602000-00003 · Cited by 859 · see citing works
Urine Output Assessment as a Clinical Quality Measure
Macedo, E. (2015) · The Nephron journals/Nephron journals · doi:10.1159/000437312 · Cited by 798 · see citing works
Collins, G. S., de Groot, J. A., Dutton, S., Omar, O., Shanyinde, M., Tajar, A., Voysey, M., Wharton, R., Yu, L.-M., Moons, K. G., et al. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-40 · Cited by 729 · see citing works
The Top Ten Algorithms in Data Mining
(2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420089653 · Cited by 682 · see citing works
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600 · see citing works
Debray, T. P. A., Vergouwe, Y., Koffijberg, H., Nieboer, D., Steyerberg, E. W., & Moons, K. G. M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.018 · Cited by 597 · see citing works
Validation, updating and impact of clinical prediction rules: A review
Toll, D. B., Janssen, K. J. M., Vergouwe, Y., & Moons, K. G. M. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.008 · Cited by 584 · see citing works
Steyerberg, E. W., Borsboom, G. J. J. M., van Houwelingen, H. C., Eijkemans, M. J. C., & Habbema, J. D. F. (2004) · Statistics in Medicine · doi:10.1002/sim.1844 · Cited by 552 · see citing works
Liaño, F., Junco, E., Pascual, J., Madero, R., & Verde, E. (1998) · PubMed · Cited by 442 · see citing works
Machine Learning for Detection and Diagnosis of Disease
Sajda, P. (2006) · Annual Review of Biomedical Engineering · doi:10.1146/annurev.bioeng.8.061505.095802 · Cited by 414 · see citing works
Updating methods improved the performance of a clinical prediction model in new patients
Janssen, K. J. M., Moons, K. G. M., Kalkman, C. J., Grobbee, D. E., & Vergouwe, Y. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.04.018 · Cited by 381 · see citing works
Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance
Van Calster, B., & Vickers, A. J. (2015) · Medical Decision Making · doi:10.1177/0272989x14547233 · Cited by 371 · cited 1× · see citing works
“Errors in individual predicted probabilities can lead to overconfidence, inappropriate treatment, or poor allocation of resources.(1,8,39,40) Although the importance of calibration is emphasized in current recommendations, many studies focus solely on discrimination, and the few existing evaluations of calibration dri…”
BackgroundA Risk Prediction Score for Kidney Failure or Mortality in Rhabdomyolysis
McMahon, G. M., Zeng, X., & Waikar, S. S. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.9774 · Cited by 326 · see citing works
Perlin, J., Kolodner, R. M., & Roswell, R. H. (2005) · A Nudge Too Far? A Nudge at All? On Paying People to Be Healthy · doi:10.12927/hcpap..17381 · Cited by 265 · see citing works
Malley, J. D., Kruppa, J., Dasgupta, A., Malley, K. G., & Ziegler, A. (2011) · Methods of Information in Medicine · doi:10.3414/me00-01-0052 · Cited by 242 · see citing works
Gurm, H. S., Seth, M., Kooiman, J., & Share, D. A. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.03.026 · Cited by 214 · see citing works
Calibrating predictive model estimates to support personalized medicine
Jiang, X., Osl, M., Kim, J., & Lucila Ohno-machado, A. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000291 · Cited by 209 · see citing works
Reporting performance of prognostic models in cancer: a review.
Mallett, S., Royston, P., Waters, R., Dutton, S., & Altman, D. G. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-21 · Cited by 195 · see citing works
Implementing Electronic Health Care Predictive Analytics: Considerations And Challenges
Amarasingham, R., Patzer, R. E., Huesch, M. D., Nguyen, N. Q., & Xie, B. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0352 · Cited by 180 · see citing works
Integrating Predictive Analytics Into High-Value Care
Parikh, R. B., Kakad, M., & Bates, D. W. (2016) · JAMA · doi:10.1001/jama.2015.19417 · Cited by 169 · see citing works
Perlin, J. B., Kolodner, R. M., & Roswell, R. H. (2004) · PubMed · Cited by 159 · see citing works
Hickey, G. L., Grant, S. W., Murphy, G. J., Bhabra, M., Pagano, D., McAllister, K., Buchan, I., & Bridgewater, B. (2013) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezs584 · Cited by 154 · see citing works
Risk Factors for Acute Kidney Injury in Severe Rhabdomyolysis
Rodríguez, E., Soler, M. J., Rap, O., Barrios, C., Orfila, M. A., & Pascual, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0082992 · Cited by 111 · see citing works
Matheny, M. E., Miller, R. A., Ikizler, T. A., Waitman, L. R., Denny, J. C., Schildcrout, J. S., Dittus, R. S., & Peterson, J. F. (2010) · Medical Decision Making · doi:10.1177/0272989x10364246 · Cited by 104 · see citing works
Self-citationPark, M. H., Shim, H. S., Kim, W. H., Kim, H., Kim, D. J., Lee, S.-H., Kim, C. S., Gwak, M. S., & Kim, G. S. (2015) · PLoS ONE · doi:10.1371/journal.pone.0136230 · Cited by 93 · see citing works
Prediction of acute kidney injury within 30 days of cardiac surgery
Ng, S., Sanagou, M., Wolfe, R. S., Cochrane, A. D., Smith, J. A., & Reid, C. M. (2013) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2013.06.049 · Cited by 92 · see citing works
Legrand, M. M., Pirracchio, R., Rosa, A., Petersen, M. L., van der Laan, M. J., Fabiani, J., Fernandez‐Gerlinger, M., Podglajen, I., Safran, D., Cholley, B., et al. (2013) · Critical Care · doi:10.1186/cc13041 · Cited by 91 · see citing works
Risk prediction with machine learning and regression methods
Steyerberg, E. W., van der Ploeg, T., & CALSTER, B. V. (2014) · Biometrical Journal · doi:10.1002/bimj.201300297 · Cited by 89 · see citing works
Kristović, D., Horvatić, I., Husedžinović, I., Sutlić, Ž., Rudež, I., Barić, D., Unić, D., Blažeković, R., & Crnogorac, M. (2015) · Interactive Cardiovascular and Thoracic Surgery · doi:10.1093/icvts/ivv162 · Cited by 85 · see citing works
A spline-based tool to assess and visualize the calibration of multiclass risk predictions
Van Hoorde, K., Van Huffel, S., Timmerman, D., Bourne, T., & Van Calster, B. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.12.016 · Cited by 85 · see citing works
Discrimination and calibration of mortality risk prediction models in interventional cardiology
Matheny, M. E., Lucila Ohno-machado, A., & Resnic, F. S. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.02.007 · Cited by 77 · see citing works
Self-citationAdaptation of Clinical Prediction Models for Application in Local Settings
Kappen, T. H., Vergouwe, Y., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., & Moons, K. G. M. (2012) · Medical Decision Making · doi:10.1177/0272989x12439755 · Cited by 70 · see citing works
Lucila Ohno-machado, A., Resnic, F. S., & Matheny, M. E. (2006) · Annual Review of Biomedical Engineering · doi:10.1146/annurev.bioeng.8.061505.095842 · Cited by 70 · see citing works
Self-citationMinne, L., Eslami, S., de Keizer, N., de Jonge, E., de Rooij, S. E., & Abu-Hanna, A. (2012) · Intensive Care Medicine · doi:10.1007/s00134-011-2390-2 · Cited by 69 · see citing works
Slankamenac, K., Beck‐Schimmer, B., Breitenstein, S., Puhan, M. A., & Clavien, P. (2013) · World Journal of Surgery · doi:10.1007/s00268-013-2159-6 · Cited by 66 · see citing works
Moving From Clinical Trials to Precision Medicine
Pencina, M. J., & Peterson, E. D. (2016) · JAMA · doi:10.1001/jama.2016.4839 · Cited by 65 · see citing works
Simplified Clinical Risk Score to Predict Acute Kidney Injury After Aortic Surgery
Kim, W. H., Lee, S. M., Choi, J. W., Kim, E. H., Lee, J., Jung, J. W., Ahn, J., Sung, K. I., Kim, C. S., & Cho, H. S. (2013) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2013.04.007 · Cited by 62 · see citing works
Performance of Apache III over Time in Australia and New Zealand: A Retrospective Cohort Study
Paul, E. M., Bailey, M., van Lint, A., & Pilcher, D. V. (2012) · Anaesthesia and Intensive Care · doi:10.1177/0310057x1204000609 · Cited by 58 · see citing works
Brown, J. R., MacKenzie, T. A., Maddox, T. M., Fly, J., Tsai, T. T., Plomondon, M. E., Nielson, C. D., Siew, E. D., Resnic, F. S., Baker, C. R., et al. (2015) · Journal of the American Heart Association · doi:10.1161/jaha.115.002136 · Cited by 56 · see citing works
Self-citationCronin, R. M., VanHouten, J. P., Siew, E. D., Eden, S., Fihn, S. D., Nielson, C. D., Peterson, J. F., Baker, C. R., Ikizler, T. A., Speroff, T., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv051 · Cited by 53 · see citing works
Self-citationAmarasingham, R., Audet, A. J., Bates, D. W., Cohen, I. G., Entwistle, M., Escobar, G. J., Liu, V. X., Etheredge, L. M., Lo, B., Lucila Ohno-machado, A., et al. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1163 · Cited by 46 · see citing works
Predicting Acute Kidney Injury Among Burn Patients in the 21st Century
Schneider, D. F., Dobrowolsky, A. B., Shakir, I. A., Sinacore, J. M., Mosier, M. M., & Gamelli, R. L. (2012) · Journal of Burn Care & Research · doi:10.1097/bcr.0b013e318239cc24 · Cited by 36 · see citing works
Gurm, S., Seth, M., Kooiman, J., & Share, D. A. (2013) · European Heart Journal · doi:10.1093/eurheartj/eht308.p1262 · Cited by 31 · see citing works
Wang, Y., Cheng, H., Tong, Y. A., & Chen, Y. (2013) · Nephrology · doi:10.1111/nep.12092 · Cited by 28 · see citing works
Cook, D., Joyce, C., Barnett, R. J., Birgan, S., Playford, H. R., Cockings, J. G. L., & Hurford, R. (2002) · Anaesthesia and Intensive Care · doi:10.1177/0310057x0203000307 · Cited by 28 · see citing works
Harrison, D. A., Lone, N., Haddow, C., MacGillivray, M., Khan, A., Cook, B., & Rowan, K. (2014) · BMC Anesthesiology · doi:10.1186/1471-2253-14-116 · Cited by 22 · see citing works
Eslami, S., de Keizer, N. F., de Jonge, E., de Rooij, S. E., Abu–Hanna, A., & Minne, L. (2012) · Methods of Information in Medicine · doi:10.3414/me11-02-0044 · Cited by 21 · see citing works
Breidthardt, T., Christ‐Crain, M., Stolz, D., Bingisser, R. M., Drexler, B., Klima, T. B., Balmelli, C., Schüetz, P., Haaf, P., Schärer, M., et al. (2012) · The American Journal of Medicine · doi:10.1016/j.amjmed.2011.07.010 · Cited by 19 · see citing works
Madan, P., Elayda, M. A., Lee, V., & Wilson, J. M. (2010) · International Journal of Cardiology · doi:10.1016/j.ijcard.2010.02.005 · Cited by 10 · see citing works
Hand, D. J. (2009) · Chapman & Hall/CRC data mining and knowledge discovery series · doi:10.1201/9781420089653.ch9 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Calibration: the Achilles heel of predictive analytics
On behalf of Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative, Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019) · BMC Medicine · doi:10.1186/s12916-019-1466-7 · Cited by 2,650 · see citing works
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · see citing works
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · see citing works
Evaluation of clinical prediction models (part 1): from development to external validation
Collins, G. S., Dhiman, P., Ma, J., Schlussel, M. M., Archer, L., Van Calster, B., Harrell, F. E., Martin, G. P., Moons, K. G. M., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-074819 · Cited by 798 · see citing works
A Systematic Review on Supervised and Unsupervised Machine Learning Algorithms for Data Science
Alloghani, M., Al-Jumeily, D., Mustafina, J., Hussain, A., & Aljaaf, A. J. (2020) · Unsupervised and Semi-Supervised Learning · doi:10.1007/978-3-030-22475-2_1 · Cited by 702 · see citing works
Ethical Machine Learning in Healthcare
Chen, I. Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K., & Ghassemi, M. (2021) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092820-114757 · Cited by 645 · see citing works
The myth of generalisability in clinical research and machine learning in health care
Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., & Celi, L. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30186-2 · Cited by 467 · see citing works
A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies
Thelen, A., Zhang, X., Fink, O., Lu, Y., Ghosh, S., Youn, B. D., Todd, M. D., Mahadevan, S., Hu, C. K., & Hu, Z. Y. (2022) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-022-03425-4 · Cited by 471 · see citing works
Feng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00611-y · Cited by 423 · see citing works
Shifting machine learning for healthcare from development to deployment and from models to data
Zhang, A., Xing, L., Zou, J., & Wu, J. C. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00898-y · Cited by 420 · see citing works
The development of a machine learning inpatient acute kidney injury prediction model.
Koyner, J. L., Carey, K. A., Edelson, D. P., & Churpek, M. M. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003123 · Cited by 323 · see citing works
Song, X., Liu, X., Liu, F., & Wang, C. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104484 · Cited by 301 · see citing works
There is no such thing as a validated prediction model
Van Calster, B., Steyerberg, E. W., Wynants, L., & van Smeden, M. (2023) · BMC Medicine · doi:10.1186/s12916-023-02779-w · Cited by 286 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · see citing works
“However, distribution shifts still pose a problem to selective prediction methods; for instance, it is difficult to maintain desired abstention rates under distribution shifts (Kompa et al., 2020), and confidence estimates have been found to drift over time (e.g., Davis et al. (2017)).”
Use of machine learning to analyse routinely collected intensive care unit data: a systematic review
Shillan, D., Sterne, J. A. C., Champneys, A., & Gibbison, B. (2019) · Critical Care · doi:10.1186/s13054-019-2564-9 · Cited by 251 · see citing works
Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review
Plana, D., Shung, D. L., Grimshaw, A. A., Saraf, A., Sung, J. J. Y., & Kann, B. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.33946 · Cited by 236 · see citing works
Predictive analytics in health care: how can we know it works?
Van Calster, B., Wynants, L., Timmerman, D., Steyerberg, E. W., & Collins, G. S. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz130 · Cited by 223 · see citing works
Jiang, L., Wu, Z., Xu, X., Zhan, Y., Jin, X., Wang, L., & Qiu, Y. (2021) · Journal of International Medical Research · doi:10.1177/03000605211000157 · Cited by 219 · see citing works
"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196 · see citing works
Ellahham, S., Ellahham, N., & Simsekler, M. C. E. (2020) · American Journal of Medical Quality · doi:10.1177/1062860619878515 · Cited by 177 · see citing works
A Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172 · see citing works
Detection of calibration drift in clinical prediction models to inform model updating
Davis, S. E., Greevy, R. A., Lasko, T. A., Walsh, C. G., & Matheny, M. E. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103611 · Cited by 160 · see citing works
Self-citationShreve, J. T., Khanani, S. A., & Haddad, T. C. (2022) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk_350652 · Cited by 144 · see citing works
Artificial intelligence for diabetic retinopathy screening, prediction and management
Gunasekeran, D. V., Ting, D. S. W., Tan, G. S. W., & Wong, T. Y. (2020) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000000693 · Cited by 133 · see citing works
Jenkins, D. A., Martin, G. P., Sperrin, M., Riley, R. D., Debray, T. P. A., Collins, G. S., & Peek, N. (2021) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00090-3 · Cited by 131 · see citing works
Song, X., Yu, A. S. L., Kellum, J. A., Waitman, L. R., Matheny, M. E., Simpson, S. Q., Hu, Y., & Liu, M. (2020) · Nature Communications · doi:10.1038/s41467-020-19551-w · Cited by 120 · see citing works
Self-citationWalsh, C. G., Chaudhry, B., Dua, P., Goodman, K. W., Kaplan, B., Kavuluru, R., Solomonides, A., & Subbian, V. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooz054 · Cited by 118 · see citing works
Rahmani, K., Thapa, R., Tsou, P., Chetty, S. C., Barnes, G., Lam, C., & Tso, C. F. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104930 · Cited by 115 · see citing works
Singh, K., Valley, T. S., Tang, S., Li, B. Y., Kamran, F., Sjoding, M. W., Wiens, J., Otles, E., Donnelly, J. P., Wei, M. Y., et al. (2021) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202006-698oc · Cited by 111 · see citing works
A nonparametric updating method to correct clinical prediction model drift
Davis, S. E., Greevy, R., Fonnesbeck, C., Lasko, T., Walsh, C. G., & Matheny, M. E. (2019) · J. Am. Medical Informatics Assoc. · doi:10.1093/jamia/ocz127 · Cited by 112 · see citing works
Self-citationDeisenhofer, A., Barkham, M., Beierl, E. T., Schwartz, B., Doorn, K. A., Beevers, C. G., Berwian, I. M., Blackwell, S. E., Bockting, C., Brakemeier, E., et al. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104443 · Cited by 115 · see citing works
Dynamic models to predict health outcomes: current status and methodological challenges
Jenkins, D. A., Sperrin, M., Martin, G., & Peek, N. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0045-2 · Cited by 96 · see citing works
Walsh, C. G., Sharman, K., & Hripcsak, G. M. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.10.008 · Cited by 102 · see citing works
Guo, L. L., Pfohl, S. R., Fries, J., Posada, J., Fleming, S. L., Aftandilian, C., Shah, N., & Sung, L. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1735184 · Cited by 96 · see citing works
Prognostic models will be victims of their own success, unless…
Lenert, M. C., Matheny, M. E., & Walsh, C. G. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz145 · Cited by 90 · see citing works
Self-citationZhang, L., Guo, W., Lv, C., Guo, M., Yang, M., Fu, Q., & Liu, X. (2023) · Animal Research and One Health · doi:10.1002/aro2.44 · Cited by 88 · see citing works
Detecting and Remediating Harmful Data Shifts for the Responsible Deployment of Clinical AI Models
Subasri, V., Krishnan, A., Kore, A., Dhalla, A., Pandya, D., Wang, B., Malkin, D., Razak, F., Verma, A. A., Goldenberg, A., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.13685 · Cited by 87 · see citing works
Demystifying artificial intelligence in pharmacy
Nelson, S. D., Walsh, C. G., Olsen, C. A., McLaughlin, A. J., LeGrand, J., Schutz, N., & Lasko, T. A. (2020) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxaa218 · Cited by 87 · see citing works
Self-citationKuo, Y., Chan, N. B., Leung, J. M. Y., Meng, H. M. L., So, A. M., Tsoi, K. K. F., & Graham, C. A. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104143 · Cited by 83 · see citing works
Introduction to Artificial Intelligence and Machine Learning for Pathology
Harrison, J. H., Gilbertson, J. R., Hanna, M. G., Olson, N. H., Seheult, J. N., Sorace, J. M., & Stram, M. N. (2021) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2020-0541-cp · Cited by 83 · see citing works
Introduction to Machine Learning in Digital Healthcare Epidemiology
Roth, J. A., Battegay, M., Juchler, F., Vogt, J. E., & Widmer, A. F. (2018) · Infection Control and Hospital Epidemiology · doi:10.1017/ice.2018.265 · Cited by 81 · see citing works
Prospective Validation of an Electronic Health Record–Based, Real-Time Suicide Risk Model
Walsh, C. G., Johnson, K. B., Ripperger, M., Sperry, S., Harris, J., Clark, N., Fielstein, E., Novak, L., Robinson, K., & Stead, W. W. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.1428 · Cited by 78 · see citing works
Nestor, B. A., McDermott, M. B. A., Boag, W., Berner, G., Naumann, T., Hughes, M. C., Goldenberg, A., & Ghassemi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00690 · Cited by 78 · see citing works
Nagaraj, S. B., Pena, M. J., Ju, W., & Heerspink, H. J. L. (2020) · Diabetes Obesity and Metabolism · doi:10.1111/dom.14178 · Cited by 77 · see citing works
Monitoring performance of clinical artificial intelligence in health care: a scoping review
Andersen, E. S., Birk-Korch, J. B., Hansen, R. S., Fly, L. H., Röttger, R., Arcani, D. M. C., Brasen, C. L., Brandslund, I., & Madsen, J. S. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-24-00042 · Cited by 73 · see citing works
Rashidi, H. H., Sen, S., Palmieri, T. L., Blackmon, T., Wajda, J., & Tran, N. K. (2020) · Scientific Reports · doi:10.1038/s41598-019-57083-6 · Cited by 73 · see citing works
Convolutional Neural Network Model for Intensive Care Unit Acute Kidney Injury Prediction
Le, S., Allen, A. O., Calvert, J. S., Palevsky, P. M., Braden, G. L., Patel, S., Pellegrini, E., Green‐Saxena, A., Hoffman, J. L., & Das, R. (2021) · Kidney International Reports · doi:10.1016/j.ekir.2021.02.031 · Cited by 67 · see citing works
Singh, H., Mhasawade, V., & Chunara, R. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000023 · Cited by 67 · see citing works
Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare
Procter, R., Tolmie, P., & Rouncefield, M. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3577009 · Cited by 70 · see citing works
Artificial Intelligence in Acute Kidney Injury Risk Prediction
Gameiro, J., Branco, T., & Lopes, J. A. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9030678 · Cited by 65 · see citing works
Callahan, A., McElfresh, D., Banda, J. M., Bunney, G., Char, D., Chen, J., Corbin, C. K., Dash, D., Downing, N. L., Jain, S. S., et al. (2024) · NEJM Catalyst · doi:10.1056/cat.24.0131 · Cited by 61 · see citing works
Vagliano, I., Chesnaye, N. C., Leopold, J. H., Jager, K. J., Abu–Hanna, A., & Schut, M. C. (2022) · Clinical Kidney Journal · doi:10.1093/ckj/sfac181 · Cited by 65 · see citing works
Kamran, F., Tang, S., Otles, E., McEvoy, D. S., Saleh, S. N., Gong, J., Li, B. Y., Dutta, S., Liu, X., Medford, R. J., et al. (2022) · BMJ · doi:10.1136/bmj-2021-068576 · Cited by 60 · see citing works
Debray, T. P. A., Collins, P. G. S., Riley, R. D., Snell, K. I. E., CALSTER, B. V., Reitsma, J. B., & Moons, K. G. M. (2023) · BMJ · doi:10.1136/bmj-2022-071058 · Cited by 62 · see citing works
Xie, Y., Gunasekeran, D. V., Balaskas, K., Keane, P. A., Sim, D. A., Bachmann, L. M., Macrae, C., & Ting, D. S. W. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.22 · Cited by 59 · see citing works
Developing more generalizable prediction models from pooled studies and large clustered data sets
de Jong, V. M. T., Moons, K. G. M., Eijkemans, M. J. C., Riley, R. D., & Debray, T. P. A. (2021) · Statistics in Medicine · doi:10.1002/sim.8981 · Cited by 56 · see citing works
EHR foundation models improve robustness in the presence of temporal distribution shift
Guo, L., Steinberg, E., Fleming, S. L., Posada, J., Lemmon, J., Pfohl, S. R., Shah, N. H., Fries, J., & Sung, L. (2023) · Scientific Reports · doi:10.1038/s41598-023-30820-8 · Cited by 59 · see citing works
Zhang, Y.-H., Xu, D.-M., Gao, J., Wang, R., Yan, K., Liang, H., Xu, J., Zhao, Y.-L., Zheng, X.-Z., Xu, L.-Y., et al. (2025) · Nature Communications · doi:10.1038/s41467-024-55629-5 · Cited by 52 · see citing works
Machine Learning Operations in Health Care: A Scoping Review
Rajagopal, A., Ayanian, S., Ryu, A. J., Qian, R., Legler, S. R., Peeler, E. A., Issa, M., Coons, T. J., & Kawamoto, K. (2024) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2024.06.009 · Cited by 52 · see citing works
Hanna, M. G., Olson, N. H., Zarella, M. D., Dash, R. C., Herrmann, M. D., Furtado, L. V., Stram, M. N., Raciti, P. M., Hassell, L. A., Mays, A., et al. (2023) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2023-0042-cp · Cited by 51 · see citing works
Calibration Drift Among Regression and Machine Learning Models for Hospital Mortality.
Davis, S. E., Lasko, T. A., Chen, G., & Matheny, M. E. (2017) · PubMed Central · Cited by 50 · see citing works
Self-citationSuriyakumar, V. M., Papernot, N., Goldenberg, A., & Ghassemi, M. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445934 · Cited by 55 · see citing works
Chi, S., Tian, Y., Wang, F., Zhou, T., Jin, S., & Li, J. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102256 · Cited by 46 · see citing works
Implications of the Use of Artificial Intelligence Predictive Models in Health Care Settings
Vaid, A., Sawant, A. S., Suárez‐Fariñas, M., Lee, J., Kaul, S. K., Kovatch, P., Freeman, R., Jiang, J. J., Jayaraman, P., Fayad, Z. A., et al. (2023) · Annals of Internal Medicine · doi:10.7326/m23-0949 · Cited by 46 · see citing works
Davis, S. E., Dorn, C., Park, D. J., & Matheny, M. E. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf039 · Cited by 43 · see citing works
Self-citationClinical deployment environments: Five pillars of translational machine learning for health
Harris, S. K., Bonnici, T., Keen, T., Lilaonitkul, W., White, M. J., & Swanepoel, N. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.939292 · Cited by 43 · see citing works
Considerations in the reliability and fairness audits of predictive models for advance care planning
Lu, J., Sattler, A. L., Wang, S. X. Y., Khaki, A. R., Callahan, A., Fleming, S. L., Fong, R., Ehlert, B., Li, R., Shieh, L. Y., et al. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.943768 · Cited by 41 · see citing works
Matheny, M. E., Ricket, I. M., Goodrich, C. A., Shah, R. U., Stabler, M. E., Perkins, A. M., Dorn, C. A., Denton, J. N., Bray, B. E., Gouripeddi, R., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.35782 · Cited by 39 · see citing works
Self-citationHe, Z., Zhou, J., Liu, Z., Dong, S., Zhang, Y., Shen, T. H., Zheng, S., & Xu, X. (2021) · Hepatobiliary & pancreatic diseases international · doi:10.1016/j.hbpd.2021.02.001 · Cited by 39 · see citing works
He, J., Hu, Y. Q., Zhang, X., Wu, L., Waitman, L. R., & Liu, M. (2018) · JAMIA Open · doi:10.1093/jamiaopen/ooy043 · Cited by 38 · see citing works
Machine learning, the kidney, and genotype–phenotype analysis
Sealfon, R. S. G., Mariani, L. H., Kretzler, M., & Troyanskaya, O. G. (2020) · Kidney International · doi:10.1016/j.kint.2020.02.028 · Cited by 37 · see citing works
Mišić, V. V., Rajaram, K., & Gabel, E. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00468-7 · Cited by 37 · see citing works
Rahimi, A. K., Canfell, O. J., Chan, W., Sly, B. P., Pole, J. D., Sullivan, C. M., & Shrapnel, S. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104758 · Cited by 35 · see citing works
"The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep LearningPreprint
Sendak, M., Elish, M. C., Gao, M. C., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O’Brien, C. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08089 · Cited by 35 · see citing works
Why do probabilistic clinical models fail to transport between sites
Lasko, T. A., Strobl, E. V., & Stead, W. W. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01037-4 · Cited by 34 · see citing works
Self-citationRamessur, R., Raja, L., Kilduff, C. L. S., Kang, S., Li, J.-P. O., Thomas, P. B. M., & Sim, D. A. (2021) · Asia-Pacific Journal of Ophthalmology · doi:10.1097/apo.0000000000000406 · Cited by 34 · see citing works
Open questions and research gaps for monitoring and updating AI-enabled tools in clinical settings
Davis, S. E., Walsh, C. G., & Matheny, M. E. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.958284 · Cited by 33 · see citing works
Self-citationMachine Learning for Acute Kidney Injury Prediction in the Intensive Care Unit
Gottlieb, E. R., Samuel, M. C., Bonventre, J. V., Celi, L. A., & Mattie, H. (2022) · Advances in Chronic Kidney Disease · doi:10.1053/j.ackd.2022.06.005 · Cited by 32 · see citing works
Tohidinezhad, F., Bontempi, D., Zhang, Z., Dingemans, A. C., Aerts, J. G. J. V., Bootsma, G. P., Vansteenkiste, J., Hashemi, S. M. S., Smit, E. F., Gietema, H. A., et al. (2023) · European Journal of Cancer · doi:10.1016/j.ejca.2023.01.027 · Cited by 32 · see citing works
Validation of Risk Prediction Models to Inform Clinical Decisions After Acute Kidney Injury
Sawhney, S., Tan, Z., Black, C., Marks, A., McLernon, D. J., Ronksley, P. E., & James, M. T. (2021) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2020.12.008 · Cited by 31 · see citing works
Ameen, S., Wong, M. C., Yee, K., & Turner, P. A. (2022) · Applied Sciences · doi:10.3390/app12073341 · Cited by 30 · see citing works
Liou, L., Scott, E. R., Parchure, P., Ouyang, Y., Egorova, N., Freeman, R., Hofer, I. S., Nadkarni, G. N., Timsina, P., Kia, A., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01141-5 · Cited by 30 · see citing works
Kuris, E. O., Veeramani, A., McDonald, C. L., DiSilvestro, K. J., Zhang, A. S., Cohen, E. M., & Daniels, A. H. (2021) · World Neurosurgery · doi:10.1016/j.wneu.2021.02.114 · Cited by 29 · see citing works
Not All Clinical AI Monitoring Systems Are Created Equal: Review and Recommendations
Feng, J., Xia, F., Singh, K., & Pirracchio, R. (2025) · NEJM AI · doi:10.1056/aira2400657 · Cited by 28 · see citing works
Zhuo, X.-Y., Lei, S.-H., Sun, L., Bai, Y., Wu, J., Zheng, Y.-J., Liu, K., Liu, W.-F., & Zhao, B.-C. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.02.018 · Cited by 28 · see citing works
Silverman, A. L., Shung, D. L., Stidham, R. W., Kochhar, G. S., & Iacucci, M. (2024) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2024.05.048 · Cited by 30 · see citing works
Ma, W., Chen, C., Zheng, S., Qin, J., Zhang, H., & Dou, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16437-8_30 · Cited by 28 · see citing works
Predicting 30-Day Hospital Readmission Risk in a National Cohort of Patients with Cirrhosis
Koola, J., Ho, S. B., Cao, A., Chen, G., Perkins, A. M., Davis, S. E., & Matheny, M. E. (2019) · Digestive Diseases and Sciences · doi:10.1007/s10620-019-05826-w · Cited by 31 · see citing works
Self-citationArtificial Intelligence in Perioperative Care: Opportunities and Challenges
Han, L., Char, D. S., Aghaeepour, N., & The Stanford Anesthesia AI Working Group (2024) · Anesthesiology · doi:10.1097/aln.0000000000005013 · Cited by 27 · see citing works
Văduva, A.-G., Oprea, S., Niculae, A.-M., Bârã, A., & ANDREESCU, A. I. (2024) · Electronics · doi:10.3390/electronics13224527 · Cited by 27 · see citing works
Bülow, R. D., Dimitrov, D., Boor, P., & Sáez-Rodríguez, J. (2021) · Seminars in Immunopathology · doi:10.1007/s00281-021-00847-y · Cited by 27 · see citing works
Walsh, C. G., McKillop, M. M., Lee, P., Harris, J. W., Simpson, C., & Novak, L. L. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab092 · Cited by 26 · see citing works
Haroz, E. E., Walsh, C. G., Goklish, N. A., Cwik, M. F., O’Keefe, V. M., & Barlow, A. (2019) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12598 · Cited by 25 · see citing works
Zhuang, Y., Dyas, A. R., Meguid, R. A., Henderson, W. G., Bronsert, M. R., Madsen, H. J., & Colborn, K. L. (2023) · Annals of Surgery · doi:10.1097/sla.0000000000006106 · Cited by 25 · see citing works
Large Language Models with Vision on Diagnostic Radiology Board Exam Style Questions
Sun, S. H., Chen, K., Anavim, S., Phillipi, M., Yeh, L., Huynh, K. N., Cortes, G., Tran, J., Tran, M. D., Yaghmai, V., et al. (2024) · Academic Radiology · doi:10.1016/j.acra.2024.11.028 · Cited by 25 · see citing works
Netayawijit, P., Chansanam, W., & Sorn-in, K. (2025) · Healthcare · doi:10.3390/healthcare13202588 · Cited by 25 · see citing works
Reps, J., Ryan, P. B., Rijnbeek, P. R., & Schuemie, M. J. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00501-2 · Cited by 25 · see citing works
Sheyn, D. D., Ju, M., Zhang, S., Anyaeche, C., Hijaz, A., Mangel, J. M., Mahajan, S. T., Conroy, B., El‐Nashar, S. A., & Ray, S. (2019) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000003517 · Cited by 25 · see citing works
Bednorz, A., Mak, J. K. L., Jylhävä, J., & Religa, D. (2023) · Clinical Interventions in Aging · doi:10.2147/cia.s400887 · Cited by 24 · see citing works
Cama-Olivares, A., Braun, C. G., Takeuchi, T., O’Hagan, E. C., Kaiser, K. A., Ghazi, L., Chen, J., Forni, L. G., Kane‐Gill, S. L., Ostermann, M., et al. (2025) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000702 · Cited by 27 · see citing works
- Authors
- Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D., & Matheny, M. E.
- Affiliations
- Vanderbilt University (US)
- Corresponding author
- Matheny, Michael E
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-03-31
- Volume
- 24
- Issue
- 6
- Pages
- 1052-1061
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC6080675
- OpenAlex
- W2604834158
- Cited by
- 312
- References
- 72
- Citations by year
- 2026: 58 · 2025: 36 · 2024: 39 · 2023: 31 · 2022: 45 · 2021: 41 · 2020: 29 · 2019: 20 · 2018: 10 · 2017: 3
- FWCI
- 15.17
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Library of Medicine (5T15LM007450-15, 1R21LM011664-01)
- Veterans Health Administration (VA HSR&D CDA-08-020, VA HSR&D IIR 11-292, VA HSR&D IIR 13-052, VA HSR&D IIR 13-073)
- Topics
- Health Sciences › Medicine › Nephrology › Acute Kidney Injury Research
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- acute kidney injury, performance drift, hospital-acquired acute kidney injury, risk prediction, random forest, artificial neural networks, regression models, clinical decision support, veterans, distribution shift
- MeSH terms
- Clinical Decision-Making, Machine Learning, Aged, Bayes Theorem, Decision Support Techniques, Female, Hospitals, Veterans, Humans, Male, Middle Aged, United States, Logistic Models, Acute Kidney Injury
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
