Cited by 136Machine Learning in HealthcareArtificial Intelligence in HealthcareElectronic Health Records Systems

Incorporating temporal EHR data in predictive models for risk stratification of renal function deterioration

Open access
Singh, Anima, Nadkarni, Girish, Gottesman, Omri, Ellis, Stephen B., Bottinger, Erwin P., Guttag, John V.·2015·Journal of Biomedical Informatics

Abstract

Predictive models built using temporal data in electronic health records (EHRs) can potentially play a major role in improving management of chronic diseases. However, these data present a multitude of technical challenges, including irregular sampling of data and varying length of available patient history. In this paper, we describe and evaluate three different approaches that use machine learning to build predictive models using temporal EHR data of a patient. The first approach is a commonly used non-temporal approach that aggregates values of the predictors in the patient's medical history. The other two approaches exploit the temporal dynamics of the data. The two temporal approaches vary in how they model temporal information and handle missing data. Using data from the EHR of Mount Sinai Medical Center, we learned and evaluated the models in the context of predicting loss of estimated glomerular filtration rate (eGFR), the most common assessment of kidney function. Our results show that incorporating temporal information in patient's medical history can lead to better prediction of loss of kidney function. They also demonstrate that exactly how this information is incorporated is important. In particular, our results demonstrate that the relative importance of different predictors varies over time, and that using multi-task learning to account for this is an appropriate way to robustly capture the temporal dynamics in EHR data. Using a case study, we also demonstrate how the multi-task learning based model can yield predictive models with better performance for identifying patients at high risk of short-term loss of kidney function.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jbi.2014.11.005

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 22 · Cited by 136

Research works printed but not linked: 16

  1. [1]Levey AS Stevens LA Coresh J Conceptual model of CKD: applications and implications Am J Kidney Dis 2009 53 Suppl 3 S4 S16 http://dx.doi.org/10.1053/j.ajkd.2008.07.048 19231760 · DOI
  2. [3]Sud M CKD stage at nephrology referral and factors influencing the risks of ESRD and death Am J Kidney Dis 2014 http://dx.doi.org/10.1053/j.ajkd.2013.12.008. pii: S0272-6386(13)01637-5 24485146 · DOI
  3. [4]Chou C-C Adults with late stage 3 chronic kidney disease are at high risk for prevalent silent brain infarction: a population-based study Stroke; J Cerebral Circulat 2011 42 8 2120 5 http://dx.doi.org/10.1161/STROKEAHA.110.597930 21700935 · DOI
  4. [5]Tangri N A predictive model for progression of chronic kidney disease to kidney failure JAMA 2011 305 15 1553 9 http://dx.doi.org/10.1001/jama.2011.451 21482743 · DOI
  5. [6]Desai AS Association between cardiac biomarkers and the development of ESRD in patients with type 2 diabetes mellitus, Anemia, and CKD Am J Kidney Dis 2011 58 5 717 28 http://dx.doi.org/10.1053/j.ajkd.2011.05.020 21820220 · DOI
  6. [7]Landray MJ Prediction of ESRD and death among people with CKD: the Chronic Renal Impairment in Birmingham (CRIB) prospective cohort study Am J Kidney Dis 2010 56 6 1082 94 http://dx.doi.org/10.1053/j.ajkd.2010.07.016 21035932 PMC2991589 · DOI
  7. [8]Echouffo-Tcheugui JB Kengne AP Risk models to predict chronic kidney disease and its progression: a systematic review PLoS Med 2012 9 11 e1001344 http://dx.doi.org/10.1371/journal.pmed.1001344 23185136 PMC3502517 · DOI
  8. [9]Hallan SI Combining GFR and albuminuria to classify CKD improves prediction of ESRD J Am Soc Nephrol 2009 20 5 1069 77 http://dx.doi.org/10.1681/ASN.2008070730 19357254 PMC2678033 · DOI
  9. [10]Hauskrecht MaL Zitao Wu Lei Modeling clinical time series using Gaussian process sequences SIAM international conference on data mining 2013 623 31
  10. [11]Luo L Methods for estimating kidney disease stage transition probabilities using electronic medical records eGEMs (Generating Evidence & Methods to improve patient outcomes) 2013 1 3 http://dx.doi.org/10.13063/2327-9214.104 PMC4371506 25848580 · DOI
  11. [12]Toma T Abu-Hanna A Bosman R Discovery and inclusion of SOFA score episode in mortality prediction J Biomed Inform 2007 40 649 60 http://dx.doi.org/10.1016/j.jbi.2007.03.007 17485242 · DOI
  12. [13]Tudor Toma R-JB Siebes Arno Peek Niels Abu-Hanna Ameen Learning predictive models that use pattern discovery – a bootstrap evaluative approach applied in organ functioning sequences J Biomed Inform 2010 43 578 86 http://dx.doi.org/10.1016/j.jbi.2010.03.00 20332034 · DOI
  13. [14]Levey AS Coresh J Chronic kidney disease The Lancet 2012 379 9811 165 80 http://dx.doi.org/10.1016/S0140-6736(11)60178-5 21840587 · DOI · Paper page
  14. [17]Kissmeyer L Community nephrology: audit of screening for renal insufficiency in a high risk population Nephrol Dial Transplant 1999 14 9 2150 5 http://dx.doi.org/10.1093/ndt/14.9.2150 10489224 · DOI
  15. [19]Ginès P Schrier RW Renal failure in cirrhosis N Engl J Med 2009 361 13 1279 90 http://dx.doi.org/10.1056/NEJMra0809139 19776409 · DOI
  16. [20]Stengel B Chronic kidney disease and cancer: a troubling connection J Nephrol 2010 23 3 253 62 20349418 PMC4823382

Authors
Singh, A., Nadkarni, G., Gottesman, O., Ellis, S. B., Bottinger, E. P., & Guttag, J. V.
Affiliations
  • MIT Computer Science and Artificial Intelligence Laboratory (US)
  • Icahn School of Medicine at Mount Sinai (US)
Corresponding author
Singh, Anima
Venue
Journal of Biomedical Informatics
Publisher
Elsevier BV
Published
2015-02
Volume
53
Pages
220-228
ISSN
1532-0464 (ISSN-L)
OpenAlex
W2015462679
Cited by
136
References
22
Citations by year
2026: 5 · 2025: 8 · 2024: 13 · 2023: 13 · 2022: 22 · 2021: 14 · 2020: 15 · 2019: 13 · 2018: 13 · 2017: 4 · 2016: 13 · 2015: 3
FWCI
6.65
Type
journal article
Language
en
Open access
Locations
  • Journal of Biomedical Informatics (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-nd)
  • PMC
Funders
Andrea and Charles Bronfman Philantropies
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
risk stratification, renal function decline, estimated glomerular filtration rate decline, multi-task learning, missing data handling, irregularly sampled data, chronic kidney disease, risk prediction model
MeSH terms
Machine Learning, Algorithms, Glomerular Filtration Rate, Hospitals, Humans, Kidney, Kidney Diseases, Medical Informatics, New York City, Risk, Software, Time Factors, Models, Statistical, Disease Progression, Area Under Curve, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review