Risk prediction for chronic kidney disease progression using heterogeneous electronic health record data and time series analysis
Open accessAbstract
Background As adoption of electronic health records continues to increase, there is an opportunity to incorporate clinical documentation as well as laboratory values and demographics into risk prediction modeling. Objective The authors develop a risk prediction model for chronic kidney disease (CKD) progression from stage III to stage IV that includes longitudinal data and features drawn from clinical documentation. Methods The study cohort consisted of 2908 primary-care clinic patients who had at least three visits prior to January 1, 2013 and developed CKD stage III during their documented history. Development and validation cohorts were randomly selected from this cohort and the study datasets included longitudinal inpatient and outpatient data from these populations. Time series analysis (Kalman filter) and survival analysis (Cox proportional hazards) were combined to produce a range of risk models. These models were evaluated using concordance, a discriminatory statistic. Results A risk model incorporating longitudinal data on clinical documentation and laboratory test results (concordance 0.849) predicts progression from state III CKD to stage IV CKD more accurately when compared to a similar model without laboratory test results (concordance 0.733, P<.001), a model that only considers the most recent laboratory test results (concordance 0.819, P < .031) and a model based on estimated glomerular filtration rate (concordance 0.779, P < .001). Conclusions A risk prediction model that takes longitudinal laboratory test results and clinical documentation into consideration can predict CKD progression from stage III to stage IV more accurately than three models that do not take all of these variables into consideration.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The Perotte et al. model was not compared with conventional simpler machine learning classifiers, and collective matrix factorization becomes inherently difficult to interpret.
“For example, the Perotte et al.99 model was not compared with conventional simpler machine learning classifiers, and collective matrix factorization becomes inherently difficult to interpret79.”
From Multimodal machine learning in precision health: A scoping review (2022) · cites this paper as [99]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793 · see citing works
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337 · see citing works
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751 · see citing works
Toprak, Ö., Kurt, H., Sarı, Y., Şarkış, C., Us, H., & Kırık, A. (2017) · Kidney & Blood Pressure Research · doi:10.1159/000468530 · Cited by 29,769 · see citing works
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297 · see citing works
A New Equation to Estimate Glomerular Filtration Rate
Levey, A. S., Stevens, L. A., Schmid, C. H., Zhang, Y. (., Castro, A. F., Feldman, H. I., Kusek, J. W., Eggers, P., Van Lente, F., Greene, T., et al. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-9-200905050-00006 · Cited by 25,603 · see citing works
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · see citing works
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,649 · see citing works
Self-citationAhmed, A., Jorna, T., & Bhandari, S. (2016) · The Nephron journals/Nephron journals · doi:10.1159/000447068 · Cited by 11,782 · see citing works
Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization
Go, A. S., Chertow, G. M., Fan, D., McCulloch, C. E., & Hsu, C.-Y. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa041031 · Cited by 11,369 · see citing works
Prevalence of Chronic Kidney Disease in the United States
Coresh, J., Selvin, E., Stevens, L. A., Manzi, J., Kusek, J. W., Eggers, P., Van Lente, F., & Levey, A. S. (2007) · JAMA · doi:10.1001/jama.298.17.2038 · Cited by 4,895 · see citing works
Graphical Models, Exponential Families, and Variational Inference
Wainwright, M. J., & Jordan, M. I. (2007) · now publishers, Inc. eBooks · doi:10.1561/9781601981851 · Cited by 3,063 · see citing works
Graphical Models, Exponential Families, and Variational Inference
Wainwright, M. J., & Jordan, M. I. (2008) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000001 · Cited by 1,842 · see citing works
Stochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,802 · see citing works
Self-citationLevin, A., Bakris, G. L., Molitch, M. E., Smulders, M., Tian, J., Williams, L. A., & Andress, D. L. (2006) · Kidney International · doi:10.1038/sj.ki.5002009 · Cited by 1,511 · see citing works
Pencina, M. J., & D’Agostino, R. B. (2004) · Statistics in Medicine · doi:10.1002/sim.1802 · Cited by 1,446 · see citing works
Inference for the Generalization Error
Nadeau, C., & Bengio, Y. (2003) · Machine Learning · doi:10.1023/a:1024068626366 · Cited by 1,165 · see citing works
What can natural language processing do for clinical decision support?
Demner‐Fushman, D., Chapman, W. W., & McDonald, C. J. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.08.007 · Cited by 690 · see citing works
Tsai, T. T., Patel, U. D., Chang, T. I., et al. (2014) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2013.06.016 · Cited by 598 · see citing works
Understanding the Kalman Filter
Meinhold, R. J., & Singpurwalla, N. D. (1983) · The American Statistician · doi:10.1080/00031305.1983.10482723 · Cited by 587 · see citing works
Inference for the Generalization ErrorPreprint
Bengio, Y., & Nadeau, C. (1999) · RePEc: Research Papers in Economics · Cited by 483 · see citing works
Murff, H. J., FitzHenry, F., Matheny, M. E., Gentry, N., Kotter, K., Crimin, K. S., Dittus, R. S., Rosen, A. K., Elkin, P. L., Brown, S. H., et al. (2011) · JAMA · doi:10.1001/jama.2011.1204 · Cited by 475 · see citing works
Timing of Onset of CKD-Related Metabolic Complications
Moranne, O. P., Froissart, M., Rossert, J., Gauci, C., Boffa, J., Haymann, J., m’rad, M. B., Jacquot, C., Houillier, P., Stengel, B., et al. (2008) · Journal of the American Society of Nephrology · doi:10.1681/asn.2008020159 · Cited by 467 · see citing works
The Timing of Specialist Evaluation in Chronic Kidney Disease and Mortality
Kinchen, K., Sadler, J. H., Fink, N. E., Brookmeyer, R., Klag, M. J., Levey, A. S., & Powe, N. R. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-137-6-200209170-00007 · Cited by 457 · see citing works
Unlocking Clinical Data from Narrative Reports: A Study of Natural Language Processing
Hripcsak, G. M., Friedman, C., Alderson, P. O., DuMouchel, W., Johnson, S. B., & Clayton, P. D. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-122-9-199505010-00007 · Cited by 340 · see citing works
Smart, N. A., Dieberg, G., Ladhani, M., & Titus, T. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007333.pub2 · Cited by 240 · see citing works
Risk Prediction Models for Patients With Chronic Kidney Disease
Tangri, N., Kitsios, G. D., Inker, L. A., Griffith, J. L., Naimark, D. M. J., Walker, S., Rigatto, C., Uhlig, K., Kent, D. M., & Levey, A. S. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-8-201304160-00004 · Cited by 226 · see citing works
Black, C., Sharma, P., Scotland, G., McCullough, K., McGurn, D., Robertson, L. M., Fluck, N., MacLeod, A. M., McNamee, P., Prescott, G. J., et al. (2010) · Health Technology Assessment · doi:10.3310/hta14210 · Cited by 221 · see citing works
Risk Models to Predict Chronic Kidney Disease and Its Progression: A Systematic Review
Echouffo-Tcheugui, J. B., & Kengne, A. P. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001344 · Cited by 199 · see citing works
Identification and Referral of Patients With Progressive CKD: A National Study
Boulware, L. E., Troll, M. U., Jaar, B. G., Myers, D. I., & Powe, N. R. (2006) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2006.04.073 · Cited by 199 · see citing works
Prevalence of chronic kidney disease
Bellizzi, V. (2008) · Giornale Italiano di Nefrologia · Cited by 176 · see citing works
Ananthakrishnan, A. N., Cai, T., Savova, G., Cheng, S.-C., Chen, P., Perez, R. G., Gainer, V. S., Murphy, S. N., Peter Szolovits, B., Xia, Z., et al. (2013) · Inflammatory Bowel Diseases · doi:10.1097/mib.0b013e31828133fd · Cited by 169 · see citing works
Hemmelgarn, B. R. (2010) · JAMA · doi:10.1001/jama.2010.303 · Cited by 163 · see citing works
Gender differences in chronic kidney disease
Iseki, K. (2008) · Kidney International · doi:10.1038/ki.2008.261 · Cited by 140 · see citing works
Jain, A. K., McLeod, I., Huo, C. Y., et al. (2009) · Kidney International · doi:10.1038/ki.2009.158 · Cited by 98 · see citing works
Vijayakrishnan, R., Steinhubl, S. R., Ng, K., Sun, J., Byrd, R. J., Daar, Z. S., Williams, B. A., deFilippi, C. R., Ebadollahi, S., & Stewart, W. F. (2014) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2014.03.008 · Cited by 89 · see citing works
Hug, B. L., Witkowski, D. J., Sox, C. M., Keohane, C. A., Seger, D. L., Yoon, C. S., Matheny, M. E., & Bates, D. W. (2009) · Kidney International · doi:10.1038/ki.2009.353 · Cited by 85 · see citing works
Clinical decision support with automated text processing for cervical cancer screening
Wagholikar, K. B., MacLaughlin, K. L., Henry, M. R., Greenes, R. A., Hankey, R. A., Liu, H., & Chaudhry, R. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-000820 · Cited by 83 · see citing works
Agrawal, V., Ghosh, A. K., Barnes, M. A., & McCullough, P. A. (2008) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2008.06.022 · Cited by 73 · see citing works
Kagoma, Y. K., Weir, M. A., Iansavichus, A. V., Hemmelgarn, B. R., Akbari, A., Patel, U. D., Garg, A. X., & Jain, A. K. (2010) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2010.08.029 · Cited by 73 · see citing works
Levin, A., Djurdjev, O., Duncan, J., Rosenbaum, D., & Werb, R. (2005) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfi209 · Cited by 72 · see citing works
Redundancy-Aware Topic Modeling for Patient Record Notes
Cohen, R., Aviram, I., Elhadad, M., & Elhadad, N. (2014) · PLoS ONE · doi:10.1371/journal.pone.0087555 · Cited by 71 · cited 1× · see citing works
“More detailed discussion of this model and methods for inference can be found in the literature.(30,31) Example topics can be seen in Tables 2 and 3, where they are represented by their most highly associated words.”
MethodSelf-citationElkin, P. L., Froehling, D. A., Wahner‐Roedler, D. L., Brown, S. H., & Bailey, K. R. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-1-201201030-00003 · Cited by 67 · see citing works
Natural Language Processing to identify pneumonia from radiology reports
Dublin, S., Baldwin, E. A., Walker, R. L., Christensen, L. M., Haug, P. J., Jackson, M. L., Nelson, J. C., Ferraro, J. P., Carrell, D., & Chapman, W. W. (2013) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3418 · Cited by 66 · see citing works
Babazono, T., Hanai, K., Suzuki, K. A. O., et al. (2006) · Diabetologia · doi:10.1007/s00125-006-0247-y · Cited by 65 · see citing works
Implementing KDOQI CKD Definition and Staging Guidelines in Southern California Kaiser Permanente
Rutkowski, M., Mann, W., Derose, S. F., Selevan, D. C., Pascual, N. O., Diesto, J., & Crooks, P. (2009) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2008.07.052 · Cited by 63 · see citing works
CKD as an Underrecognized Threat to Patient Safety
Fink, J. C., Brown, J., Hsu, V. D., Seliger, S. L., Walker, L. D., & Zhan, M. (2009) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2008.12.016 · Cited by 60 · see citing works
Greer, R. C., Powe, N. R., Jaar, B. G., Troll, M. U., & Boulware, L. E. (2011) · BMC Nephrology · doi:10.1186/1471-2369-12-1 · Cited by 53 · see citing works
Adverse Safety Events in Chronic Kidney Disease
Chapin, E., Zhan, M., Hsu, V. D., et al. (2009) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.06210909 · Cited by 53 · see citing works
Barriers to successful care for chronic kidney disease
Lenz, O., Mekala, D. P., Patel, D. V., Fornoni, A., Metz, D., & Roth, D. (2005) · BMC Nephrology · doi:10.1186/1471-2369-6-11 · Cited by 44 · see citing works
Automated identification of pneumonia in chest radiograph reports in critically ill patients
Liu, V. X., Clark, M. P., Mendoza, M., Saket, R. R., Gardner, M. N., Turk, B. J., & Escobar, G. J. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-90 · Cited by 31 · see citing works
Patel, T. G., POGACH, L. M., & Barth, R. H. (2009) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2008.07.051 · Cited by 29 · see citing works
Roizen, M. F. (2012) · Yearbook of Anesthesiology and Pain Management · doi:10.1016/j.yane.2012.02.007 · Cited by 28 · see citing works
Characterizing pre-dialysis care in the era of eGFR reporting: a cohort study
Abdel-Kader, K. F. M., Fischer, G. S., Johnston, J. R., Gu, C., Moore, C. G., & Unruh, M. L. (2011) · BMC Nephrology · doi:10.1186/1471-2369-12-12 · Cited by 23 · see citing works
Hemmelgarn, B. R., Manns, B., Straus, S. E., et al. (2012) · Journal of Nephrology · doi:10.5301/jn.5000226 · Cited by 18 · see citing works
eGFR and creatinine clearance in relation to metabolic changes in an unselected patient population
Drion, I., Joosten, H., Dikkeschei, L. D., et al. (2009) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2009.07.002 · Cited by 16 · see citing works
Pottelbergh, G. V., Vaes, B., Jadoul, M. Y., et al. (2012) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2011.12.010 · Cited by 12 · see citing works
Wu, I., Hsu, K., Lee, C., et al. (2013) · BMC Nephrology · doi:10.1186/1471-2369-14-92 · Cited by 9 · see citing works
Clinical Decision Support for Colonoscopy Surveillance Using Natural Language Processing
Wagholikar, K. B., Sohn, S., Wu, S., Kaggal, V. C., Buehler, S., Greenes, R. A., Wu, T., Larson, D. W., Liu, H., Chaudhry, R., et al. (2012) · IEEE International Conference on Healthcare Informatics, Imaging and Systems Biology · doi:10.1109/hisb.2012.11 · Cited by 9 · see citing works
Steinhubl, S. R., Williams, B. A., Sun, J., et al. (2011) · Circulation · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [18]Callaghan FM Jackson MT Demner-Fushman D NLP-derived information improves the estimates of risk of disease compared to estimates based on manually extracted data alone. In: 5th International Symposium on Semantic Mining in Biomedicine. 2012:18–25.
Cited by
Sorted by citations
Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710 · see citing works
A clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086 · see citing works
Multimodal machine learning in precision health: A scoping review
Kline, A., Wang, H., Li, Y., Dennis, S., Hutch, M., Xu, Z., Wang, F., Cheng, F., & Luo, Y. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00712-8 · Cited by 528 · see citing works
“Similar justi fi cations as outlined above could be applied to other areas seen as ‘ under saturated ’ such as hematology with only one paper 49 and nephrology having just three 41,87,99 Fig.”
BackgroundMethodDeep Learning to Predict Patient Future Diseases from the Electronic Health Records
Miotto, R., Li, L., & Dudley, J. T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-30671-1_66 · Cited by 59 · see citing works
Korach, Z. T., Cato, K. D., Collins, S. A., Kang, M. J., Knaplund, C., Dykes, P. C., Wang, L., Schnock, K. O., Garcia, J. P., Jia, H., et al. (2019) · Applied Clinical Informatics · doi:10.1055/s-0039-3401814 · Cited by 32 · see citing works
Cleary, F., Prieto-Merino, D., & Nitsch, D. (2022) · PLOS ONE · doi:10.1371/journal.pone.0264167 · Cited by 6 · see citing works
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · doi:10.21203/rs.2.10083/v1 · Cited by 3 · see citing works
- Authors
- Perotte, A., Ranganath, R., Hirsch, J. S., Blei, D., & Elhadad, N.
- Affiliations
- Columbia University (US)
- Princeton University (US)
- Corresponding author
- Perotte, Adler
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-04-20
- Volume
- 22
- Issue
- 4
- Pages
- 872-880
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC4482276
- OpenAlex
- W2119033610
- Cited by
- 128
- References
- 60
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 18 · 2023: 14 · 2022: 16 · 2021: 11 · 2020: 14 · 2019: 15 · 2018: 12 · 2017: 5 · 2016: 10 · 2015: 2
- FWCI
- 9.48
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Science Foundation (1344668, 0745520, 1009542, 1247664)
- Alfred P. Sloan Foundation (0745520, IIS-1009542)
- Defense Advanced Research Projects Agency (FA8750, FA8750‐14‐2‐0009)
- Advanced Research Projects Agency
- Office of Naval Research (N00014-11-1-0651, IIS-1009542, 0745520, N00014)
- U.S. National Library of Medicine (T15LM007079)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Nephrology › Chronic Kidney Disease and Diabetes
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- chronic kidney disease progression, risk prediction model, electronic health records, clinical documentation, Kalman filter, Cox proportional hazards model, estimated glomerular filtration rate, primary care patients, heterogeneous data integration
- MeSH terms
- Aged, Female, Glomerular Filtration Rate, Humans, Longitudinal Studies, Male, Middle Aged, Models, Theoretical, Primary Health Care, Time, Cohort Studies, Proportional Hazards Models, Survival Analysis, Disease Progression, Risk Assessment, Renal Insufficiency, Chronic, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
