The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Calibration drift in regression and machine learning models for acute kidney injury
Open accessAbstract
Objective: Predictive analytics create opportunities to incorporate personalized risk estimates into clinical decision support. Models must be well calibrated to support decision-making, yet calibration deteriorates over time. This study explored the influence of modeling methods on performance drift and connected observed drift with data shifts in the patient population. Materials and Methods: Using 2003 admissions to Department of Veterans Affairs hospitals nationwide, we developed 7 parallel models for hospital-acquired acute kidney injury using common regression and machine learning methods, validating each over 9 subsequent years. Results: Discrimination was maintained for all models. Calibration declined as all models increasingly overpredicted risk. However, the random forest and neural network models maintained calibration across ranges of probability, capturing more admissions than did the regression models. The magnitude of overprediction increased over time for the regression models while remaining stable and small for the machine learning models. Changes in the rate of acute kidney injury were strongly linked to increasing overprediction, while changes in predictor-outcome associations corresponded with diverging patterns of calibration drift across methods. Conclusions: Efficient and effective updating protocols will be essential for maintaining accuracy of, user confidence in, and safety of personalized risk predictions to support decision-making. Model updating protocols should be tailored to account for variations in calibration drift across methods and respond to periods of rapid performance drift rather than be limited to regularly scheduled annual or biannual intervals.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Ongoing monitoring is essential because the performance and effect(s) of AI models can change after deployment 27,28 .”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This limited information is of concern because proprietary models are difficult to assess owing to their opaque nature and have been shown to decline in performance over time.15,16”
From External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients (2021) · cites this paper as [16]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This extends prior reports of temporal miscalibration [32–34] by showing calibration failure is structural and progressive rather than sporadic.”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648
KDIGO Clinical Practice Guidelines for Acute Kidney Injury
Khwaja, A. (2012) · Nephron Clinical Practice · doi:10.1159/000339789 · Cited by 8,297
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.2307/1271436 · Cited by 7,001
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Flagged first, then by citations
Cited by
Sorted by citations
Items 301–313 of 313
Huy, N. T., Minh, P. Q., & Bich, L. T. (2022) · Journal of Artificial Intelligence for Healthcare Systems · doi:10.68159/f249917157
Georges, O., Beyls, C., Dominicis, F. D., Merlusca, G., Pfister, A. W., Wolf, J. D., & Arab, O. A. (2026) · Research Square · doi:10.21203/rs.3.rs-10754857/v1
Liberski, P. S., Żerdziński, K., Gałuszewski, M., Skrzypek, M., & Krzych, Ł. J. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15187013
Qin, L.-H., Lü, Y.-X., Liang, W., Song, R., Chen, P.-Y., Xu, W., Zeng, G., Chen, J., Zhao, Y., & Long, L. (2026) · Academic Radiology · doi:10.1016/j.acra.2026.08.110
SOMA: a preventive health governance framework for adaptive, consumer-facing digital health guidance
Mueller, M. R. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1902578
Wang, X., Xiong, D., Duan, B., Ding, G., Huang, Y., & Wang, Q. (2026) · iScience · doi:10.1016/j.isci.2026.117697
Li, Z., Chen, K., Ji, Y., Wu, H., & Lu, Q. (2026) · Skin · doi:10.2738/skin.2026.0011
Balagam, P. K. (2026) · Research Square · doi:10.21203/rs.3.rs-11122103/v1
BAYDİLİ, I., Taşçı, B., Taşçı, G., Doğan, Ş., & Tuncer, T. (2026) · Bioengineering · doi:10.3390/bioengineering13101175 · Cited by 1
Assessing Concept and Virtual Drift in a Deep Learning Model for Antibiotic Resistance Prediction
Vouriot, L., Rebaudet, S., Sardi, F. S., Lebsir, M., Boussier, J., Armand-Lefèvre, L., Gaudart, J., & Urena, R. (2026) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms69103.2026.00281
Omigbodun, F. T. (2026) · Journal of Medicine Surgery and Public Health · doi:10.1016/j.glmedi.2026.100258
李红旗, Liu, F., & Tang, L. (2026) · Buildings · doi:10.3390/buildings16214200
Liu, C., Chen, C.-J., & Ko, Y.-L. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1965076
Metadata
- Authors
- Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D., & Matheny, M. E.
- Affiliations
- Vanderbilt University (US)
- Corresponding author
- Matheny, Michael E
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-03-31
- Volume
- 24
- Issue
- 6
- Pages
- 1052-1061
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC6080675
- OpenAlex
- W2604834158
- Cited by
- 312
- References
- 72
- Citations by year
- 2026: 58 · 2025: 36 · 2024: 39 · 2023: 31 · 2022: 45 · 2021: 41 · 2020: 29 · 2019: 20 · 2018: 10 · 2017: 3
- FWCI
- 15.17
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Library of Medicine (5T15LM007450-15, 1R21LM011664-01)
- Veterans Health Administration (VA HSR&D CDA-08-020, VA HSR&D IIR 11-292, VA HSR&D IIR 13-052, VA HSR&D IIR 13-073)
- Topics
- Health Sciences › Medicine › Nephrology › Acute Kidney Injury Research
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- acute kidney injury, performance drift, hospital-acquired acute kidney injury, risk prediction, random forest, artificial neural networks, regression models, clinical decision support, veterans, distribution shift
- MeSH terms
- Clinical Decision-Making, Machine Learning, Aged, Bayes Theorem, Decision Support Techniques, Female, Hospitals, Veterans, Humans, Male, Middle Aged, United States, Logistic Models, Acute Kidney Injury
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
