The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Calibration drift in regression and machine learning models for acute kidney injury
Open accessAbstract
Objective: Predictive analytics create opportunities to incorporate personalized risk estimates into clinical decision support. Models must be well calibrated to support decision-making, yet calibration deteriorates over time. This study explored the influence of modeling methods on performance drift and connected observed drift with data shifts in the patient population. Materials and Methods: Using 2003 admissions to Department of Veterans Affairs hospitals nationwide, we developed 7 parallel models for hospital-acquired acute kidney injury using common regression and machine learning methods, validating each over 9 subsequent years. Results: Discrimination was maintained for all models. Calibration declined as all models increasingly overpredicted risk. However, the random forest and neural network models maintained calibration across ranges of probability, capturing more admissions than did the regression models. The magnitude of overprediction increased over time for the regression models while remaining stable and small for the machine learning models. Changes in the rate of acute kidney injury were strongly linked to increasing overprediction, while changes in predictor-outcome associations corresponded with diverging patterns of calibration drift across methods. Conclusions: Efficient and effective updating protocols will be essential for maintaining accuracy of, user confidence in, and safety of personalized risk predictions to support decision-making. Model updating protocols should be tailored to account for variations in calibration drift across methods and respond to periods of rapid performance drift rather than be limited to regularly scheduled annual or biannual intervals.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Ongoing monitoring is essential because the performance and effect(s) of AI models can change after deployment 27,28 .”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This limited information is of concern because proprietary models are difficult to assess owing to their opaque nature and have been shown to decline in performance over time.15,16”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This extends prior reports of temporal miscalibration [32–34] by showing calibration failure is structural and progressive rather than sporadic.”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
KDIGO Clinical Practice Guidelines for Acute Kidney Injury
Khwaja, A. (2012) · Nephron Clinical Practice · doi:10.1159/000339789 · Cited by 8,297
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.2307/1271436 · Cited by 7,001
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Flagged first, then by citations
Cited by
Sorted by citations
Calibration: the Achilles heel of predictive analytics
On behalf of Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative, Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019) · BMC Medicine · doi:10.1186/s12916-019-1466-7 · Cited by 2,650
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956
Evaluation of clinical prediction models (part 1): from development to external validation
Collins, G. S., Dhiman, P., Ma, J., Schlussel, M. M., Archer, L., Van Calster, B., Harrell, F. E., Martin, G. P., Moons, K. G. M., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-074819 · Cited by 798
A Systematic Review on Supervised and Unsupervised Machine Learning Algorithms for Data Science
Alloghani, M., Al-Jumeily, D., Mustafina, J., Hussain, A., & Aljaaf, A. J. (2020) · Unsupervised and Semi-Supervised Learning · doi:10.1007/978-3-030-22475-2_1 · Cited by 702
Ethical Machine Learning in Healthcare
Chen, I. Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K., & Ghassemi, M. (2021) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092820-114757 · Cited by 645
The myth of generalisability in clinical research and machine learning in health care
Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., & Celi, L. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30186-2 · Cited by 467
A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies
Thelen, A., Zhang, X., Fink, O., Lu, Y., Ghosh, S., Youn, B. D., Todd, M. D., Mahadevan, S., Hu, C. K., & Hu, Z. Y. (2022) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-022-03425-4 · Cited by 511
Feng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00611-y · Cited by 423
Shifting machine learning for healthcare from development to deployment and from models to data
Zhang, A., Xing, L., Zou, J., & Wu, J. C. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00898-y · Cited by 420
The development of a machine learning inpatient acute kidney injury prediction model.
Koyner, J. L., Carey, K. A., Edelson, D. P., & Churpek, M. M. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003123 · Cited by 323
Song, X., Liu, X., Liu, F., & Wang, C. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104484 · Cited by 301
There is no such thing as a validated prediction model
Van Calster, B., Steyerberg, E. W., Wynants, L., & van Smeden, M. (2023) · BMC Medicine · doi:10.1186/s12916-023-02779-w · Cited by 286
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518
“However, distribution shifts still pose a problem to selective prediction methods; for instance, it is difficult to maintain desired abstention rates under distribution shifts (Kompa et al., 2020), and confidence estimates have been found to drift over time (e.g., Davis et al. (2017)).”
Use of machine learning to analyse routinely collected intensive care unit data: a systematic review
Shillan, D., Sterne, J. A. C., Champneys, A., & Gibbison, B. (2019) · Critical Care · doi:10.1186/s13054-019-2564-9 · Cited by 251
Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review
Plana, D., Shung, D. L., Grimshaw, A. A., Saraf, A., Sung, J. J. Y., & Kann, B. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.33946 · Cited by 236
Predictive analytics in health care: how can we know it works?
Van Calster, B., Wynants, L., Timmerman, D., Steyerberg, E. W., & Collins, G. S. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz130 · Cited by 223
Jiang, L., Wu, Z., Xu, X., Zhan, Y., Jin, X., Wang, L., & Qiu, Y. (2021) · Journal of International Medical Research · doi:10.1177/03000605211000157 · Cited by 232
"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196
Ellahham, S., Ellahham, N., & Simsekler, M. C. E. (2020) · American Journal of Medical Quality · doi:10.1177/1062860619878515 · Cited by 177
A Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172
Detection of calibration drift in clinical prediction models to inform model updating
Davis, S. E., Greevy, R. A., Lasko, T. A., Walsh, C. G., & Matheny, M. E. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103611 · Cited by 160
Self-citationShreve, J. T., Khanani, S. A., & Haddad, T. C. (2022) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk_350652 · Cited by 151
Artificial intelligence for diabetic retinopathy screening, prediction and management
Gunasekeran, D. V., Ting, D. S. W., Tan, G. S. W., & Wong, T. Y. (2020) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000000693 · Cited by 133
Jenkins, D. A., Martin, G. P., Sperrin, M., Riley, R. D., Debray, T. P. A., Collins, G. S., & Peek, N. (2021) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00090-3 · Cited by 131
Song, X., Yu, A. S. L., Kellum, J. A., Waitman, L. R., Matheny, M. E., Simpson, S. Q., Hu, Y., & Liu, M. (2020) · Nature Communications · doi:10.1038/s41467-020-19551-w · Cited by 120
Self-citationWalsh, C. G., Chaudhry, B., Dua, P., Goodman, K. W., Kaplan, B., Kavuluru, R., Solomonides, A., & Subbian, V. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooz054 · Cited by 118
Rahmani, K., Thapa, R., Tsou, P., Chetty, S. C., Barnes, G., Lam, C., & Tso, C. F. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104930 · Cited by 115
Singh, K., Valley, T. S., Tang, S., Li, B. Y., Kamran, F., Sjoding, M. W., Wiens, J., Otles, E., Donnelly, J. P., Wei, M. Y., et al. (2021) · Annals of the American Thoracic Society · doi:10.1513/annalsats.202006-698oc · Cited by 111
A nonparametric updating method to correct clinical prediction model drift
Davis, S. E., Greevy, R., Fonnesbeck, C., Lasko, T., Walsh, C. G., & Matheny, M. E. (2019) · J. Am. Medical Informatics Assoc. · doi:10.1093/jamia/ocz127 · Cited by 112
Self-citationDeisenhofer, A., Barkham, M., Beierl, E. T., Schwartz, B., Doorn, K. A., Beevers, C. G., Berwian, I. M., Blackwell, S. E., Bockting, C., Brakemeier, E., et al. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104443 · Cited by 123
Dynamic models to predict health outcomes: current status and methodological challenges
Jenkins, D. A., Sperrin, M., Martin, G., & Peek, N. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0045-2 · Cited by 96
Walsh, C. G., Sharman, K., & Hripcsak, G. M. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.10.008 · Cited by 102
Guo, L. L., Pfohl, S. R., Fries, J., Posada, J., Fleming, S. L., Aftandilian, C., Shah, N., & Sung, L. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1735184 · Cited by 96
Prognostic models will be victims of their own success, unless…
Lenert, M. C., Matheny, M. E., & Walsh, C. G. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz145 · Cited by 97
Self-citationZhang, L., Guo, W., Lv, C., Guo, M., Yang, M., Fu, Q., & Liu, X. (2023) · Animal Research and One Health · doi:10.1002/aro2.44 · Cited by 110
Detecting and Remediating Harmful Data Shifts for the Responsible Deployment of Clinical AI Models
Subasri, V., Krishnan, A., Kore, A., Dhalla, A., Pandya, D., Wang, B., Malkin, D., Razak, F., Verma, A. A., Goldenberg, A., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.13685 · Cited by 87
Demystifying artificial intelligence in pharmacy
Nelson, S. D., Walsh, C. G., Olsen, C. A., McLaughlin, A. J., LeGrand, J., Schutz, N., & Lasko, T. A. (2020) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxaa218 · Cited by 98
Self-citationKuo, Y., Chan, N. B., Leung, J. M. Y., Meng, H. M. L., So, A. M., Tsoi, K. K. F., & Graham, C. A. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104143 · Cited by 86
Introduction to Artificial Intelligence and Machine Learning for Pathology
Harrison, J. H., Gilbertson, J. R., Hanna, M. G., Olson, N. H., Seheult, J. N., Sorace, J. M., & Stram, M. N. (2021) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2020-0541-cp · Cited by 91
Introduction to Machine Learning in Digital Healthcare Epidemiology
Roth, J. A., Battegay, M., Juchler, F., Vogt, J. E., & Widmer, A. F. (2018) · Infection Control and Hospital Epidemiology · doi:10.1017/ice.2018.265 · Cited by 86
Prospective Validation of an Electronic Health Record–Based, Real-Time Suicide Risk Model
Walsh, C. G., Johnson, K. B., Ripperger, M., Sperry, S., Harris, J., Clark, N., Fielstein, E., Novak, L., Robinson, K., & Stead, W. W. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.1428 · Cited by 78
Nestor, B., McDermott, M. B. A., Boag, W., Berner, G., Naumann, T., Hughes, M. C., Goldenberg, A., & Ghassemi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00690 · Cited by 89
Nagaraj, S. B., Pena, M. J., Ju, W., & Heerspink, H. J. L. (2020) · Diabetes Obesity and Metabolism · doi:10.1111/dom.14178 · Cited by 80
Monitoring performance of clinical artificial intelligence in health care: a scoping review
Andersen, E. S., Birk-Korch, J. B., Hansen, R. S., Fly, L. H., Röttger, R., Arcani, D. M. C., Brasen, C. L., Brandslund, I., & Madsen, J. S. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-24-00042 · Cited by 73
Rashidi, H. H., Sen, S., Palmieri, T. L., Blackmon, T., Wajda, J., & Tran, N. K. (2020) · Scientific Reports · doi:10.1038/s41598-019-57083-6 · Cited by 77
Convolutional Neural Network Model for Intensive Care Unit Acute Kidney Injury Prediction
Le, S., Allen, A. O., Calvert, J. S., Palevsky, P. M., Braden, G. L., Patel, S., Pellegrini, E., Green‐Saxena, A., Hoffman, J. L., & Das, R. (2021) · Kidney International Reports · doi:10.1016/j.ekir.2021.02.031 · Cited by 67
Singh, H., Mhasawade, V., & Chunara, R. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000023 · Cited by 67
Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare
Procter, R., Tolmie, P., & Rouncefield, M. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3577009 · Cited by 78
Artificial Intelligence in Acute Kidney Injury Risk Prediction
Gameiro, J., Branco, T., & Lopes, J. A. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9030678 · Cited by 69
Callahan, A., McElfresh, D., Banda, J. M., Bunney, G., Char, D., Chen, J., Corbin, C. K., Dash, D., Downing, N. L., Jain, S. S., et al. (2024) · NEJM Catalyst · doi:10.1056/cat.24.0131 · Cited by 61
Vagliano, I., Chesnaye, N. C., Leopold, J. H., Jager, K. J., Abu–Hanna, A., & Schut, M. C. (2022) · Clinical Kidney Journal · doi:10.1093/ckj/sfac181 · Cited by 68
Kamran, F., Tang, S., Otles, E., McEvoy, D. S., Saleh, S. N., Gong, J., Li, B. Y., Dutta, S., Liu, X., Medford, R. J., et al. (2022) · BMJ · doi:10.1136/bmj-2021-068576 · Cited by 60
Debray, T. P. A., Collins, P. G. S., Riley, R. D., Snell, K. I. E., CALSTER, B. V., Reitsma, J. B., & Moons, K. G. M. (2023) · BMJ · doi:10.1136/bmj-2022-071058 · Cited by 64
Xie, Y., Gunasekeran, D. V., Balaskas, K., Keane, P. A., Sim, D. A., Bachmann, L. M., Macrae, C., & Ting, D. S. W. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.22 · Cited by 59
Developing more generalizable prediction models from pooled studies and large clustered data sets
de Jong, V. M. T., Moons, K. G. M., Eijkemans, M. J. C., Riley, R. D., & Debray, T. P. A. (2021) · Statistics in Medicine · doi:10.1002/sim.8981 · Cited by 56
EHR foundation models improve robustness in the presence of temporal distribution shift
Guo, L., Steinberg, E., Fleming, S. L., Posada, J., Lemmon, J., Pfohl, S. R., Shah, N. H., Fries, J., & Sung, L. (2023) · Scientific Reports · doi:10.1038/s41598-023-30820-8 · Cited by 65
Zhang, Y., Xu, D., Gao, J., Wang, R., Yan, K., Liang, H., Xu, J., Zhao, Y., Zheng, X., Xu, L., et al. (2025) · Nature Communications · doi:10.1038/s41467-024-55629-5 · Cited by 58
Machine Learning Operations in Health Care: A Scoping Review
Rajagopal, A., Ayanian, S., Ryu, A. J., Qian, R., Legler, S. R., Peeler, E. A., Issa, M., Coons, T. J., & Kawamoto, K. (2024) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2024.06.009 · Cited by 52
Hanna, M. G., Olson, N. H., Zarella, M. D., Dash, R. C., Herrmann, M. D., Furtado, L. V., Stram, M. N., Raciti, P. M., Hassell, L. A., Mays, A., et al. (2023) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2023-0042-cp · Cited by 52
Calibration Drift Among Regression and Machine Learning Models for Hospital Mortality.
Davis, S. E., Lasko, T. A., Chen, G., & Matheny, M. E. (2017) · PubMed Central · Cited by 50
Self-citationSuriyakumar, V. M., Papernot, N., Goldenberg, A., & Ghassemi, M. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445934 · Cited by 62
Chi, S., Tian, Y., Wang, F., Zhou, T., Jin, S., & Li, J. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102256 · Cited by 46
Implications of the Use of Artificial Intelligence Predictive Models in Health Care Settings
Vaid, A., Sawant, A. S., Suárez‐Fariñas, M., Lee, J., Kaul, S. K., Kovatch, P., Freeman, R., Jiang, J. J., Jayaraman, P., Fayad, Z. A., et al. (2023) · Annals of Internal Medicine · doi:10.7326/m23-0949 · Cited by 50
Davis, S. E., Dorn, C., Park, D. J., & Matheny, M. E. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf039 · Cited by 43
Self-citationClinical deployment environments: Five pillars of translational machine learning for health
Harris, S. K., Bonnici, T., Keen, T., Lilaonitkul, W., White, M. J., & Swanepoel, N. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.939292 · Cited by 46
Considerations in the reliability and fairness audits of predictive models for advance care planning
Lu, J., Sattler, A. L., Wang, S. X. Y., Khaki, A. R., Callahan, A., Fleming, S. L., Fong, R., Ehlert, B., Li, R., Shieh, L. Y., et al. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.943768 · Cited by 43
Matheny, M. E., Ricket, I. M., Goodrich, C. A., Shah, R. U., Stabler, M. E., Perkins, A. M., Dorn, C. A., Denton, J. N., Bray, B. E., Gouripeddi, R., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.35782 · Cited by 42
Self-citationHe, Z., Zhou, J., Liu, Z., Dong, S., Zhang, Y., Shen, T. H., Zheng, S., & Xu, X. (2021) · Hepatobiliary & pancreatic diseases international · doi:10.1016/j.hbpd.2021.02.001 · Cited by 42
He, J., Hu, Y. Q., Zhang, X., Wu, L., Waitman, L. R., & Liu, M. (2018) · JAMIA Open · doi:10.1093/jamiaopen/ooy043 · Cited by 38
Machine learning, the kidney, and genotype–phenotype analysis
Sealfon, R. S. G., Mariani, L. H., Kretzler, M., & Troyanskaya, O. G. (2020) · Kidney International · doi:10.1016/j.kint.2020.02.028 · Cited by 37
Mišić, V. V., Rajaram, K., & Gabel, E. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00468-7 · Cited by 37
Rahimi, A. K., Canfell, O. J., Chan, W., Sly, B. P., Pole, J. D., Sullivan, C. M., & Shrapnel, S. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104758 · Cited by 35
"The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep LearningPreprint
Sendak, M., Elish, M. C., Gao, M. C., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O’Brien, C. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08089 · Cited by 35
Why do probabilistic clinical models fail to transport between sites
Lasko, T. A., Strobl, E. V., & Stead, W. W. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01037-4 · Cited by 41
Self-citationRamessur, R., Raja, L., Kilduff, C. L. S., Kang, S., Li, J.-P. O., Thomas, P. B. M., & Sim, D. A. (2021) · Asia-Pacific Journal of Ophthalmology · doi:10.1097/apo.0000000000000406 · Cited by 36
Open questions and research gaps for monitoring and updating AI-enabled tools in clinical settings
Davis, S. E., Walsh, C. G., & Matheny, M. E. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.958284 · Cited by 33
Self-citationMachine Learning for Acute Kidney Injury Prediction in the Intensive Care Unit
Gottlieb, E. R., Samuel, M. C., Bonventre, J. V., Celi, L. A., & Mattie, H. (2022) · Advances in Chronic Kidney Disease · doi:10.1053/j.ackd.2022.06.005 · Cited by 36
Tohidinezhad, F., Bontempi, D., Zhang, Z., Dingemans, A. C., Aerts, J. G. J. V., Bootsma, G. P., Vansteenkiste, J., Hashemi, S. M. S., Smit, E. F., Gietema, H. A., et al. (2023) · European Journal of Cancer · doi:10.1016/j.ejca.2023.01.027 · Cited by 31
Validation of Risk Prediction Models to Inform Clinical Decisions After Acute Kidney Injury
Sawhney, S., Tan, Z., Black, C., Marks, A., McLernon, D. J., Ronksley, P. E., & James, M. T. (2021) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2020.12.008 · Cited by 32
Ameen, S., Wong, M. C., Yee, K., & Turner, P. A. (2022) · Applied Sciences · doi:10.3390/app12073341 · Cited by 31
Liou, L., Scott, E. R., Parchure, P., Ouyang, Y., Egorova, N., Freeman, R., Hofer, I. S., Nadkarni, G. N., Timsina, P., Kia, A., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01141-5 · Cited by 30
Kuris, E. O., Veeramani, A., McDonald, C. L., DiSilvestro, K. J., Zhang, A. S., Cohen, E. M., & Daniels, A. H. (2021) · World Neurosurgery · doi:10.1016/j.wneu.2021.02.114 · Cited by 29
Not All Clinical AI Monitoring Systems Are Created Equal: Review and Recommendations
Feng, J., Xia, F., Singh, K., & Pirracchio, R. (2025) · NEJM AI · doi:10.1056/aira2400657 · Cited by 28
Zhuo, X.-Y., Lei, S.-H., Sun, L., Bai, Y., Wu, J., Zheng, Y.-J., Liu, K., Liu, W.-F., & Zhao, B.-C. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.02.018 · Cited by 28
Silverman, A. L., Shung, D. L., Stidham, R. W., Kochhar, G. S., & Iacucci, M. (2024) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2024.05.048 · Cited by 30
Ma, W., Chen, C., Zheng, S., Qin, J., Zhang, H., & Dou, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16437-8_30 · Cited by 28
Predicting 30-Day Hospital Readmission Risk in a National Cohort of Patients with Cirrhosis
Koola, J., Ho, S. B., Cao, A., Chen, G., Perkins, A. M., Davis, S. E., & Matheny, M. E. (2019) · Digestive Diseases and Sciences · doi:10.1007/s10620-019-05826-w · Cited by 31
Self-citationArtificial Intelligence in Perioperative Care: Opportunities and Challenges
Han, L., Char, D. S., Aghaeepour, N., & The Stanford Anesthesia AI Working Group (2024) · Anesthesiology · doi:10.1097/aln.0000000000005013 · Cited by 26
Văduva, A.-G., Oprea, S., Niculae, A.-M., Bârã, A., & ANDREESCU, A. I. (2024) · Electronics · doi:10.3390/electronics13224527 · Cited by 27
Bülow, R. D., Dimitrov, D., Boor, P., & Sáez-Rodríguez, J. (2021) · Seminars in Immunopathology · doi:10.1007/s00281-021-00847-y · Cited by 28
Walsh, C. G., McKillop, M. M., Lee, P., Harris, J. W., Simpson, C., & Novak, L. L. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab092 · Cited by 26
Haroz, E. E., Walsh, C. G., Goklish, N. A., Cwik, M. F., O’Keefe, V. M., & Barlow, A. (2019) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12598 · Cited by 25
Zhuang, Y., Dyas, A. R., Meguid, R. A., Henderson, W. G., Bronsert, M. R., Madsen, H. J., & Colborn, K. L. (2023) · Annals of Surgery · doi:10.1097/sla.0000000000006106 · Cited by 25
Large Language Models with Vision on Diagnostic Radiology Board Exam Style Questions
Sun, S. H., Chen, K., Anavim, S., Phillipi, M., Yeh, L., Huynh, K. N., Cortes, G., Tran, J., Tran, M. D., Yaghmai, V., et al. (2024) · Academic Radiology · doi:10.1016/j.acra.2024.11.028 · Cited by 29
Netayawijit, P., Chansanam, W., & Sorn-in, K. (2025) · Healthcare · doi:10.3390/healthcare13202588 · Cited by 41
Reps, J., Ryan, P. B., Rijnbeek, P. R., & Schuemie, M. J. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00501-2 · Cited by 25
Sheyn, D. D., Ju, M., Zhang, S., Anyaeche, C., Hijaz, A., Mangel, J. M., Mahajan, S. T., Conroy, B., El‐Nashar, S. A., & Ray, S. (2019) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000003517 · Cited by 27
Bednorz, A., Mak, J. K. L., Jylhävä, J., & Religa, D. (2023) · Clinical Interventions in Aging · doi:10.2147/cia.s400887 · Cited by 27
Cama-Olivares, A., Braun, C. G., Takeuchi, T., O’Hagan, E. C., Kaiser, K. A., Ghazi, L., Chen, J., Forni, L. G., Kane‐Gill, S. L., Ostermann, M., et al. (2025) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000702 · Cited by 30
Metadata
- Authors
- Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D., & Matheny, M. E.
- Affiliations
- Vanderbilt University (US)
- Corresponding author
- Matheny, Michael E
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-03-31
- Volume
- 24
- Issue
- 6
- Pages
- 1052-1061
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC6080675
- OpenAlex
- W2604834158
- Cited by
- 312
- References
- 72
- Citations by year
- 2026: 58 · 2025: 36 · 2024: 39 · 2023: 31 · 2022: 45 · 2021: 41 · 2020: 29 · 2019: 20 · 2018: 10 · 2017: 3
- FWCI
- 15.17
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Library of Medicine (5T15LM007450-15, 1R21LM011664-01)
- Veterans Health Administration (VA HSR&D CDA-08-020, VA HSR&D IIR 11-292, VA HSR&D IIR 13-052, VA HSR&D IIR 13-073)
- Topics
- Health Sciences › Medicine › Nephrology › Acute Kidney Injury Research
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- acute kidney injury, performance drift, hospital-acquired acute kidney injury, risk prediction, random forest, artificial neural networks, regression models, clinical decision support, veterans, distribution shift
- MeSH terms
- Clinical Decision-Making, Machine Learning, Aged, Bayes Theorem, Decision Support Techniques, Female, Hospitals, Veterans, Humans, Male, Middle Aged, United States, Logistic Models, Acute Kidney Injury
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
