CoChat
Sign Up
Cited by 77Machine Learning in HealthcareArtificial Intelligence in HealthcareChronic Kidney Disease and Diabetes

Longitudinal Risk Prediction of Chronic Kidney Disease in Diabetic Patients using Temporal-Enhanced Gradient Boosting Machine: Retrospective Cohort Study

Open access
Song, Xing, Waitman, Lemuel Russ, Yu, Alan SL, Robbins, David C, Hu, Yong, Liu, Mei·2020·JMIR Medical Informatics
DOI 10.2196/15510PMC PMC7055762

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Artificial intelligence–enabled electronic health record (EHR) analysis can revolutionize medical practice from diagnosis and prediction of complex diseases to making recommendations in patient care, especially for chronic conditions such as chronic kidney disease (CKD), which is one of the most frequent complications in patients with diabetes and is associated with substantial morbidity and mortality. Objective: Longitudinal prediction of health outcome requires effective representation of temporal data in EHR. In this study, we proposed a novel temporal-enhanced gradient boosting machine (GBM) model that dynamically updates and ensembles learners based on new events in patient timelines to improve the prediction accuracy of CKD among patients with diabetes. Methods: Using a broad spectrum of deidentified EHR data on a retrospective cohort of 14,039 adult patients with type 2 diabetes and GBM as the base learner, we validated our proposed Landmark-Boosting model against three state-of-the-art temporal models for rolling predictions of 1-year CKD risk. Results: The proposed model uniformly outperformed other models, achieving an area under receiver operating curve of 0.83 (95% CI 0.76-0.85), 0.78 (95% CI 0.75-0.82), and 0.82 (95% CI 0.78-0.86) in predicting CKD risk with automatic accumulation of new data in later years (years 2, 3, and 4 since diabetes mellitus onset, respectively). The Landmark-Boosting model also maintained the best calibration across moderate- and high-risk groups and over time. The experimental results demonstrated that the proposed temporal model can not only accurately predict 1-year CKD risk but also improve performance over time with additionally accumulated data, which is essential for clinical use to improve renal management of patients with diabetes. Conclusions: Incorporation of temporal information in EHR data can significantly improve predictive model performance and will particularly benefit patients who follow-up with their physicians as recommended.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/15510

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A time-enhanced gradient boosting machine trained on EHR data from 14,039 patients with T2D achieved AUCs of 0.83, 0.78, and 0.82 for predicting DN onset over 2-, 3-, and 4-year horizons.

    “a time-enhanced gradient boosting machine (GBM) trained on EHR data from 14,039 patients with T2D achieved AUCs of 0.83, 0.78, and 0.82 for predicting DN onset over 2-, 3-, and 4-year horizons”

    From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [70]

    The Abstract (Results) reports the stated AUCs at years 2, 3, and 4 since diabetes onset, and Methods (Study Cohort) reports 14,039 patients. But Methods (Experimental Design) says the model made rolling predictions of kidney-disease risk over the next year at each of those time points—not predictions over 2-, 3-, and 4-year horizons. The paper calls the outcome diabetic kidney disease (DKD), rather than DN. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Most DN clinical feature-based studies did not specify the prediction horizon, with only a few targeting new-onset DN over 1- or 3-year periods.

    “Notably, most studies did not specify the prediction horizon, with only a few targeting new-onset DN over 1- or 3-year periods [65, 70].”

    From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [70]

    The Methods > Experimental Design section describes this study’s rolling prediction of one-year diabetic kidney disease risk, supporting it as an example of a study with a specified horizon. The passages found do not establish that most other studies omitted a prediction horizon or that only a few targeted one- or three-year periods. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 43 · Cited by 77

Research works printed but not linked: 39

  1. [1]Kramer H Screening for kidney disease in adults with diabetes and prediabetes Curr Opin Nephrol Hypertens 2005 5 14 3 249 52 10.1097/01.mnh.0000165891.67878.7f 15821418 15821418 · DOI
  2. [2]Persson F Rossing P Diagnosis of diabetic kidney disease: state of the art and future perspective Kidney Int Suppl (2011) 2018 1 8 1 2 7 10.1016/j.kisu.2017.10.003 30675433 30675433 PMC6336222 · DOI
  3. [4]Molitch ME DeFronzo RA Franz MJ Keane WF Mogensen CE Parving H Steffes MW American Diabetes Association Nephropathy in diabetes Diabetes Care 2004 1 27 Suppl 1 S79 83 10.2337/diacare.27.2007.s79 14693934 14693934 · DOI
  4. [5]Gross JL de Azevedo MJ Silveiro SP Canani LH Caramori ML Zelmanovitz T Diabetic nephropathy: diagnosis, prevention, and treatment Diabetes Care 2005 1 28 1 164 76 10.2337/diacare.28.1.164 15616252 15616252 · DOI · Paper page
  5. [6]Orphanou K Stassopoulou A Keravnou E Temporal abstraction and temporal Bayesian networks in clinical domains: a survey Artif Intell Med 2014 3 60 3 133 49 10.1016/j.artmed.2013.12.007 24529699 24529699 · DOI
  6. [7]Zhao J Henriksson A Learning temporal weights of clinical events using variable importance BMC Med Inform Decis Mak 2016 7 21 16 Suppl 2 71 10.1186/s12911-016-0311-6 27459993 27459993 PMC4965710 · DOI
  7. [8]Augusto JC Temporal reasoning for decision support in medicine Artif Intell Med 2005 1 33 1 1 24 10.1016/j.artmed.2004.07.006 15617978 15617978 · DOI
  8. [9]Shahar Y A framework for knowledge-based temporal abstraction Artif Intell 1997 90 1-2 79 133 10.1016/S0004-3702(96)00025-2 · DOI · Paper page
  9. [10]Ghosh S Li J Cao L Ramamohanarao K Septic shock prediction for ICU patients via coupled HMM walking on sequential contrast patterns J Biomed Inform 2017 2 66 19 31 10.1016/j.jbi.2016.12.010 28011233 28011233 · DOI
  10. [11]Lin J Keogh E Wei L Lonardi S Experiencing SAX: a novel symbolic representation of time series Data Min Knowl Discov 2007 15 2 107 44 10.1007/s10618-007-0064-z · DOI · Paper page
  11. [12]Moskovitch R Shahar Y Classification-driven temporal discretization of multivariate time series Data Min Knowl Discov 2014 10 2 29 4 871 913 10.1007/s10618-014-0380-z · DOI
  12. [13]Rasmy L Wu Y Wang N Geng X Zheng WJ Wang F Wu H Xu H Zhi D A study of generalizability of recurrent neural network-based predictive models for heart failure onset risk using a large and heterogeneous EHR data set J Biomed Inform 2018 8 84 11 6 10.1016/j.jbi.2018.06.011 29908902 29908902 PMC6076336 · DOI
  13. [14]Che ZP Purushotham S Cho K Sontag D Liu Y Recurrent neural networks for multivariate time series with missing values Sci Rep 2018 4 17 8 1 6085 10.1038/s41598-018-24271-9 10.1038/s41598-018-24271-9 29666385 29666385 PMC5904216 · DOI · Paper page
  14. [16]Jardine MJ Hata J Woodward M Perkovic V Ninomiya T Arima H Zoungas S Cass A Patel A Marre M Mancia G Mogensen CE Poulter N Chalmers J ADVANCE Collaborative Group Prediction of kidney-related outcomes in patients with type 2 diabetes Am J Kidney Dis 2012 11 60 5 770 8 10.1053/j.ajkd.2012.04.025 22694950 22694950 · DOI
  15. [17]Lin C Li C Liu C Lin W Lin C Yang S Li T Development and validation of a risk prediction model for end-stage renal disease in patients with type 2 diabetes Sci Rep 2017 8 31 7 1 10177 10.1038/s41598-017-09243-9 10.1038/s41598-017-09243-9 28860599 28860599 PMC5579050 · DOI
  16. [18]Hagar Y Albers D Pivovarov R Chase H Dukic V Elhadad N Survival analysis with electronic health record data: experiments with chronic kidney disease Stat Anal Data Min 2014 7 5 385 403 10.1002/sam.11236 10.1002/sam.11236 PMC8112603 33981381 · DOI
  17. [19]Perotte A Ranganath R Hirsch JS Blei D Elhadad N Risk prediction for chronic kidney disease progression using heterogeneous electronic health record data and time series analysis J Am Med Inform Assoc 2015 7 22 4 872 80 10.1093/jamia/ocv024 25896647 25896647 PMC4482276 · DOI · Paper page
  18. [20]Singh A Nadkarni G Gottesman O Ellis SB Bottinger EP Guttag JV Incorporating temporal EHR data in predictive models for risk stratification of renal function deterioration J Biomed Inform 2015 2 53 220 8 10.1016/j.jbi.2014.11.005 25460205 25460205 PMC4520404 · DOI · Paper page
  19. [21]Koyner JL Carey KA Edelson DP Churpek MM The development of a machine learning inpatient acute kidney injury prediction model Crit Care Med 2018 7 46 7 1070 7 10.1097/CCM.0000000000003123 29596073 29596073 · DOI · Paper page
  20. [22]Dafni U Landmark analysis at the 25-year landmark point Circ Cardiovasc Qual Outcomes 2011 5 4 3 363 71 10.1161/CIRCOUTCOMES.110.957951 21586725 21586725 · DOI
  21. [23]Wells BJ Chagin KM Li L Hu B Yu C Kattan MW Using the landmark method for creating prediction models in large datasets derived from electronic health records Health Care Manag Sci 2015 3 18 1 86 92 10.1007/s10729-014-9281-3 24752545 24752545 · DOI
  22. [24]Nichols GA Desai J Lafata JE Lawrence JM O'Connor PJ Pathak RD Raebel MA Reid RJ Selby JV Silverman BG Steiner JF Stewart WF Vupputuri S Waitzfelder B SUPREME-DM Study Group Construction of a multisite DataLink using electronic health records for the identification, surveillance, prevention, and management of diabetes mellitus: the SUPREME-DM project Prev Chronic Dis 2012 9 E110 10.5888/pcd9.110311 22677160 22677160 PMC3457753 · DOI
  23. [25]KDOQI KDOQI clinical practice guidelines and clinical practice recommendations for diabetes and chronic kidney disease Am J Kidney Dis 2007 2 49 2 Suppl 2 S12 154 10.1053/j.ajkd.2006.12.005 17276798 17276798 · DOI · Paper page
  24. [26]American Diabetes Association Standards of medical care in diabetes-2018 abridged for primary care providers Clin Diabetes 2018 1 36 1 14 37 10.2337/cd17-0119 29382975 29382975 PMC5775000 · DOI
  25. [27]Levey AS Coresh J Greene T Stevens LA Zhang YL Hendriksen S Kusek JW van Lente F Chronic Kidney Disease Epidemiology Collaboration Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate Ann Intern Med 2006 8 15 145 4 247 54 10.7326/0003-4819-145-4-200608150-00004 16908915 16908915 · DOI
  26. [28]Waitman LR Warren JJ Manos EL Connolly DW Expressing observations from electronic medical record flowsheets in an i2b2 based clinical data repository to support research and quality improvement AMIA Annu Symp Proc 2011 2011 1454 63 22195209 22195209 PMC3243191
  27. [29]Murphy SN Weber G Mendis M Gainer V Chueh HC Churchill S Kohane I Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2) J Am Med Inform Assoc 2010 17 2 124 30 10.1136/jamia.2009.000893 20190053 20190053 PMC3000779 · DOI
  28. [30]Song X Waitman LR Hu Y Yu AS Robbins D Liu M An exploration of ontology-based EMR data abstraction for diabetic kidney disease prediction AMIA Jt Summits Transl Sci Proc 2019 2019 704 13 31259027 31259027 PMC6568123
  29. [31]Damle R Alavi K The University Healthsystem Consortium clinical database: An emerging resource in colorectal surgery research Semin Colon Rectal Surg 2016 6 27 2 92 5 10.1053/j.scrs.2016.01.006 · DOI
  30. [32]Hutchinson R Liu LP Dietterich TG Incorporating Boosted Regression Trees Into Ecological Latent Variable Models Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence 2011 AAAI'11 August 7-11, 2011 San Francisco, California 1343 8
  31. [33]Johnson R Zhang T Learning nonlinear functions using regularized greedy forest IEEE Trans Pattern Anal Mach Intell 2014 5 36 5 942 54 10.1109/TPAMI.2013.159 26353228 26353228 · DOI
  32. [34]He K Li Y Zhu J Liu H Lee JE Amos CI Hyslop T Jin J Lin H Wei Q Li Y Component-wise gradient boosting and false discovery control in survival analysis with high-dimensional covariates Bioinformatics 2016 1 1 32 1 50 7 10.1093/bioinformatics/btv517 26382192 26382192 PMC4757968 · DOI
  33. [35]Torlay L Perrone-Bertolotti M Thomas E Baciu M Machine learning-XGBoost analysis of language networks to classify patients with epilepsy Brain Inform 2017 9 4 3 159 69 10.1007/s40708-017-0065-7 28434153 28434153 PMC5563301 · DOI · Paper page
  34. [36]Kalousis A Prados J Hilario M Stability of feature selection algorithms: a study on high-dimensional spaces Knowl Inf Syst 2007 12 1 95 116 10.1007/s10115-006-0040-8 · DOI
  35. [37]Friedman JH Greedy function approximation: a gradient boosting machine Ann Stat 2001 10 29 5 1189 232 10.1214/aos/1013203451 · DOI · Paper page
  36. [38]Chen T Guestrin C XGBoost: A Scalable Tree Boosting System Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2016 KDD'16 August 13-17, 2016 San Francisco, CA 785 94
  37. [39]Davis J Goadrich M The relationship between Precision-Recall and ROC curves Proceedings of the 23rd international conference on Machine learning 2006 ICML'06 June 25-29, 2006 Pittsburgh, PA 233 40 10.1145/1143844.1143874 · DOI · Paper page
  38. [40]Hood L Friend SH Predictive, personalized, preventive, participatory (P4) cancer medicine Nat Rev Clin Oncol 2011 3 8 3 184 7 10.1038/nrclinonc.2010.227 21364692 21364692 · DOI
  39. [41]Song X Waitman LR Hu Y Yu AS Robins D Liu M Robust clinical marker identification for diabetic kidney disease with ensemble feature selection J Am Med Inform Assoc 2019 3 1 26 3 242 53 10.1093/jamia/ocy165 30602020 30602020 PMC7792755 · DOI

Authors
Song, X., Waitman, L. R., Yu, A. S., Robbins, D. C., Hu, Y., & Liu, M.
Affiliations
  • University of Kansas Medical Center (US)
  • Jinan University (CN)
Corresponding author
Liu, Mei
ORCID
Venue
JMIR Medical Informatics
Publisher
JMIR Publications Inc.
Published
2020-01-24
Volume
8
Issue
1
Pages
e15510
ISSN
2291-9694 (ISSN-L)
DOI
10.2196/15510
PMC
PMC7055762
OpenAlex
W2989324284
Cited by
77
References
43
Citations by year
2026: 11 · 2025: 17 · 2024: 10 · 2023: 12 · 2022: 16 · 2021: 9 · 2020: 2
FWCI
3.08
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Medical Informatics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Natural Science Foundation of China (91746204)
  • National Institutes of Health (UL1TR002366)
  • Major Research Plan (91746204)
  • National Institute of General Medical Sciences (P20GM130423)
  • National Center for Advancing Translational Sciences
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Medicine › Nephrology › Chronic Kidney Disease and Diabetes
Keywords
chronic kidney disease, type 2 diabetes, CKD prediction, longitudinal prediction, electronic health records, temporal data representation, rolling prediction, model calibration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review