Internal and External Validation of a Machine Learning Risk Score for Acute Kidney Injury
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Importance: Acute kidney injury (AKI) is associated with increased morbidity and mortality in hospitalized patients. Current methods to identify patients at high risk of AKI are limited, and few prediction models have been externally validated. Objective: To internally and externally validate a machine learning risk score to detect AKI in hospitalized patients. Design, Setting, and Participants: This diagnostic study included 495 971 adult hospital admissions at the University of Chicago (UC) from 2008 to 2016 (n = 48 463), at Loyola University Medical Center (LUMC) from 2007 to 2017 (n = 200 613), and at NorthShore University Health System (NUS) from 2006 to 2016 (n = 246 895) with serum creatinine (SCr) measurements. Patients with an SCr concentration at admission greater than 3.0 mg/dL, with a prior diagnostic code for chronic kidney disease stage 4 or higher, or who received kidney replacement therapy within 48 hours of admission were excluded. A simplified version of a previously published gradient boosted machine AKI prediction algorithm was used; it was validated internally among patients at UC and externally among patients at NUS and LUMC. Main Outcomes and Measures: Prediction of Kidney Disease Improving Global Outcomes SCr-defined stage 2 AKI within a 48-hour interval was the primary outcome. Discrimination was assessed by the area under the receiver operating characteristic curve (AUC). Results: The study included 495 971 adult admissions (mean [SD] age, 63 [18] years; 87 689 [17.7%] African American; and 266 866 [53.8%] women) across 3 health systems. The development of stage 2 or higher AKI occurred in 15 664 of 48 463 patients (3.4%) in the UC cohort, 5711 of 200 613 (2.8%) in the LUMC cohort, and 3499 of 246 895 (1.4%) in the NUS cohort. In the UC cohort, 332 patients (0.7%) required kidney replacement therapy compared with 672 patients (0.3%) in the LUMC cohort and 440 patients (0.2%) in the NUS cohort. The AUCs for predicting at least stage 2 AKI in the next 48 hours were 0.86 (95% CI, 0.86-0.86) in the UC cohort, 0.85 (95% CI, 0.84-0.85) in the LUMC cohort, and 0.86 (95% CI, 0.86-0.86) in the NUS cohort. The AUCs for receipt of kidney replacement therapy within 48 hours were 0.96 (95% CI, 0.96-0.96) in the UC cohort, 0.95 (95% CI, 0.94-0.95) in the LUMC cohort, and 0.95 (95% CI, 0.94-0.95) in the NUS cohort. In time-to-event analysis, a probability cutoff of at least 0.057 predicted the onset of stage 2 AKI a median (IQR) of 27 (6.5-93) hours before the eventual doubling in SCr concentrations in the UC cohort, 34.5 (19-85) hours in the NUS cohort, and 39 (19-108) hours in the LUMC cohort. Conclusions and Relevance: In this study, the machine learning algorithm demonstrated excellent discrimination in both internal and external validation, supporting its generalizability and potential as a clinical decision support tool to improve AKI detection and outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
A clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086
The development of a machine learning inpatient acute kidney injury prediction model.
Koyner, J. L., Carey, K. A., Edelson, D. P., & Churpek, M. M. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003123 · Cited by 323
Self-citation
Cited by
Acute kidney injury in the critically ill: an updated review on pathophysiology and management
Pickkers, P., Darmon, M., Hoste, E., Joannidis, M., Legrand, M., Ostermann, M., Prowle, J. R., Schneider, A., & Schetz, M. (2021) · Intensive Care Medicine · doi:10.1007/s00134-021-06454-7 · Cited by 549
Song, X., Yu, A. S. L., Kellum, J. A., Waitman, L. R., Matheny, M. E., Simpson, S. Q., Hu, Y., & Liu, M. (2020) · Nature Communications · doi:10.1038/s41467-020-19551-w · Cited by 120
Rajendran, S., Xu, Z., Pan, W., Ghosh, A., & Wang, F. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000117 · Cited by 61
Liu, X., DuMontier, C., Hu, P., Liu, C., Yeung, W., Mao, Z., Ho, V., Thoral, P. J., Kuo, P.-C., Hu, J., et al. (2023) · The Journals of Gerontology: Series A · doi:10.1093/gerona/glac107 · Cited by 14
Yang, M., Liu, S., Hao, T., Ma, C., Chen, H., Li, Y., Wu, C., Xie, J., Qiu, H., Li, J., et al. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102785 · Cited by 10
Koyner, J. L., Martin, J., Carey, K. A., Caskey, J., Edelson, D. P., Mayampurath, A., Dligach, D., Afshar, M., & Churpek, M. M. (2025) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.0000000695 · Cited by 10
Self-citation
Metadata
- Authors
- Churpek, M. M., Carey, K. A., Edelson, D. P., Singh, T., Astor, B. C., Gilbert, E. R., Winslow, C., Shah, N., Afshar, M., & Koyner, J. L.
- Venue
- JAMA Network Open
- Publisher
- American Medical Association (AMA)
- Published
- 2020-08-11
- Volume
- 3
- Issue
- 8
- Pages
- e2012892
- ISSN
- 2574-3805 (ISSN-L)
- OpenAlex
- W3048873596
- Cited by
- 119
- References
- 3
- FWCI
- 9.53
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.