Predicting all‐cause readmissions using electronic health record data from the entire hospitalization: Model development and comparison
Open accessAbstract
BACKGROUND: Incorporating clinical information from the full hospital course may improve prediction of 30‐day readmissions. OBJECTIVE: To develop an all‐cause readmissions risk‐prediction model incorporating electronic health record (EHR) data from the full hospital stay, and to compare “full‐stay” model performance to a “first day” and 2 other validated models, LACE (includes Length of stay, Acute [nonelective] admission status, Charlson Comorbidity Index, and Emergency department visits in the past year), and HOSPITAL (includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type [nonelective], number of Admissions in the past year, and Length of stay). DESIGN: Observational cohort study. SUBJECTS: All medicine discharges between November 2009 and October 2010 from 6 hospitals in North Texas, including safety net, teaching, and nonteaching sites. MEASURES: Thirty‐day nonelective readmissions were ascertained from 75 regional hospitals. RESULTS: Among 32,922 admissions (validation = 16,430), 12.7% were readmitted. In addition to many first‐day factors, we identified hospital‐acquired Clostridium difficile infection (adjusted odds ratio [AOR]: 2.03, 95% confidence interval [CI]: 1.18‐3.48), vital sign instability on discharge (AOR: 1.25, 95% CI: 1.15‐1.36), hyponatremia on discharge (AOR: 1.34, 95% CI: 1.18‐1.51), and length of stay (AOR: 1.06, 95% CI: 1.04‐1.07) as significant predictors. The full‐stay model had better discrimination than other models though the improvement was modest (C statistic 0.69 vs 0.64‐0.67). It was also modestly better in identifying patients at highest risk for readmission (likelihood ratio +2.4 vs. 1.8–2.1) and in reclassifying individuals (net reclassification index 0.02–0.06). CONCLUSIONS: Incorporating clinically granular EHR data from the full hospital stay modestly improves prediction of 30‐day readmissions. Given limited improvement in prediction despite incorporation of data on hospital complications, clinical instabilities, and trajectory, our findings suggest that many factors influencing readmissions remain unaccounted for. Further improvements in readmission models will likely require accounting for psychosocial and behavioral factors not currently captured by EHRs. Journal of Hospital Medicine 2016;11:473–480. © 2016 Society of Hospital Medicine
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
To the best of the authors' knowledge, their models outperform existing EHR models in the medical literature for predicting mortality (0.92-0.94 vs 0.91), unexpected readmission (0.75-0.76 vs 0.69), and increased length of stay (0.85-0.86 vs 0.77).
“To the best of our knowledge, our models outperform existing EHR models in the medical literature for predicting mortality (0.92–0.94 vs 0.91),42 unexpected readmission (0.75–0.76 vs 0.69),43 and increased length of stay (0.85–0.86 vs 0.77).44”
From Scalable and accurate deep learning with electronic health records (2018) · cites this paper as [43]
Citation 43 supports the 0.69 comparator: the Results (Derivation and Validation of the Full-Stay EHR Model for 30-Day Readmission; page unavailable) report a C statistic of 0.69. The paper predicts all-cause 30-day readmission, however, not specifically unexpected readmission. It does not assess the citing authors’ models or their reported 0.75–0.76 values. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Pencina, M. J., D' Agostino, R. B., D' Agostino, R. B., & Vasan, R. S. (2008) · Statistics in Medicine · doi:10.1002/sim.2929 · Cited by 6,493 · see citing works
Rehospitalizations among Patients in the Medicare Fee-for-Service Program
Jencks, S. F., Williams, M. V., & Coleman, E. A. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0803563 · Cited by 5,164 · see citing works
Risk Prediction Models for Hospital Readmission
Kansagara, D. L., Englander, H. L., Salanitro, A. H., Kagen, D. I., Theobald, C. N., Freeman, M., & Kripalani, S. B. (2011) · JAMA · doi:10.1001/jama.2011.1515 · Cited by 1,712 · see citing works
Neighborhood Socioeconomic Disadvantage and 30-Day Rehospitalization
Kind, A., Jencks, S., Brock, J., Yu, M., Bartels, C. M., Ehlenbach, W. J., Greenberg, C., & Smith, M. (2014) · Annals of Internal Medicine · doi:10.7326/m13-2946 · Cited by 1,383 · see citing works
Post-Hospital Syndrome — An Acquired, Transient Condition of Generalized Risk
Krumholz, H. M. (2013) · New England Journal of Medicine · doi:10.1056/nejmp1212324 · Cited by 1,218 · see citing works
Interventions to Reduce 30-Day Rehospitalization: A Systematic Review
Hansen, L. O., Young, R. S., Hinami, K., Leung, A., & Williams, M. V. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00008 · Cited by 1,182 · see citing works
van Walraven, C., Dhalla, I. A., Bell, C. M., Etchells, E. E., Stiell, I. G., Zarnke, K. B., Austin, P. C., & Forster, A. J. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.091117 · Cited by 966 · see citing works
Does Increased Access to Primary Care Reduce Hospital Readmissions?
Weinberger, M., Oddone, E. Ž., & Henderson, W. G. (1996) · New England Journal of Medicine · doi:10.1056/nejm199605303342206 · Cited by 801 · see citing works
Proportion of hospital readmissions deemed avoidable: a systematic review
van Walraven, C., Bennett, C. L., Jennings, A., Austin, P. C., & Forster, A. J. (2011) · Canadian Medical Association Journal · doi:10.1503/cmaj.101860 · Cited by 712 · see citing works
Leening, M. J. G., Vedder, M. M., Witteman, J. C. M., Pencina, M. J., & Steyerberg, E. W. (2014) · Annals of Internal Medicine · doi:10.7326/m13-1522 · Cited by 594 · see citing works
Donzé, J., Aujesky, D., Williams, D., & Schnipper, J. L. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.3023 · Cited by 553 · see citing works
Thirty-Day Readmissions — Truth and Consequences
Joynt, K. E., & Jha, A. K. (2012) · New England Journal of Medicine · doi:10.1056/nejmp1201598 · Cited by 483 · see citing works
Amarasingham, R., Moore, B. J., Tabak, Y. P., Drazner, M. H., Clark, C. A., Zhang, S., Reed, W. G., Swanson, T. S., Ma, Y., & Halm, E. A. (2010) · Medical Care · doi:10.1097/mlr.0b013e3181ef60d9 · Cited by 437 · see citing works
Self-citationCalvillo–King, L., Arnold, D. L., Eubank, K. J., Lo, M., Yunyongying, P., Stieglitz, H., & Halm, E. A. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-012-2235-x · Cited by 436 · see citing works
Self-citationPatients in context--EHR capture of social and behavioral determinants of health
Adler, N. E., & Stead, W. W. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1413945 · Cited by 317 · see citing works
Arbaje, A. I., Wolff, J. L., Yu, Q., Powe, N. R., Anderson, G. F., & Boult, C. (2008) · The Gerontologist · doi:10.1093/geront/48.4.495 · Cited by 305 · see citing works
Hospital-Initiated Transitional Care Interventions as a Patient Safety Strategy
Rennke, S., Nguyen, O. K., Shoeb, M., Magan, Y., Wachter, R. M., & Ranji, S. R. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-5-201303051-00011 · Cited by 271 · see citing works
Self-citationFunctional Impairment and Hospital Readmission in Medicare Seniors
Greysen, S. R., Cenzer, I. S., Auerbach, A. D., & Covinsky, K. E. (2015) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2014.7756 · Cited by 266 · see citing works
Socioeconomic Status And Readmissions: Evidence From An Urban Teaching Hospital
Hu, J., Gonsahn, M. D., & Nerenz, D. R. (2014) · Health Affairs · doi:10.1377/hlthaff.2013.0816 · Cited by 225 · see citing works
Association of impaired functional status at hospital discharge and subsequent rehospitalization
Hoyer, E. H., Needham, D. M., Atanelov, L., Knox, B., Friedman, M., & Brotman, D. J. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2152 · Cited by 184 · see citing works
A conceptual framework for the study of early readmission as an indicator of quality of care
Ashton, C. M., & Wray, N. P. (1996) · Social Science & Medicine · doi:10.1016/s0277-9536(96)00049-4 · Cited by 181 · see citing works
Implementing Electronic Health Care Predictive Analytics: Considerations And Challenges
Amarasingham, R., Patzer, R. E., Huesch, M. D., Nguyen, N. Q., & Xie, B. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0352 · Cited by 180 · see citing works
Self-citationHospital Readmissions—Not Just a Measure of Quality
Kangovi, S., & Grande, D. (2011) · JAMA · doi:10.1001/jama.2011.1562 · Cited by 160 · see citing works
Patel, A., Parikh, R., Howell, E. H., Hsich, E., Landers, S. H., & Gorodeski, E. Z. (2015) · Circulation: Heart Failure · doi:10.1161/circheartfailure.114.001438 · Cited by 133 · see citing works
Predicting 30-Day Readmissions With Preadmission Electronic Health Record Data
Shadmi, E., Flaks‐Manov, N., Hoshen, M. B., Goldman, O., Bitterman, H., & Balicer, R. D. (2015) · Medical Care · doi:10.1097/mlr.0000000000000315 · Cited by 120 · see citing works
Amarasingham, R., Patel, P. C., Toto, K. H., et al. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-001901 · Cited by 119 · see citing works
Self-citationGetting More Performance from Performance Measurement
Cassel, C. K., Conway, P. H., Delbanco, S. F., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmp1408345 · Cited by 97 · see citing works
Adding Socioeconomic Data To Hospital Readmissions Calculations May Produce More Useful Results
Nagasako, E. M., Reidhead, M., Waterman, B. M., & Dunagan, W. C. (2014) · Health Affairs · doi:10.1377/hlthaff.2013.1148 · Cited by 96 · see citing works
Dharmarajan, K., Hsieh, A., Lin, Z., Bueno, H., Ross, J. S., Horwitz, L. I., Barreto-Filho, J. A. S., Kim, N., Suter, L. G., Bernheim, S. M., et al. (2013) · BMJ · doi:10.1136/bmj.f6571 · Cited by 95 · see citing works
Horwitz, L. I., Partovian, C., Lin, Z., Grady, J. N., Herrin, J. S., Conover, M. M., Montague, J. A., Dillaway, C. H., Bartczak, K., Suter, L. G., et al. (2014) · Annals of Internal Medicine · doi:10.7326/m13-3000 · Cited by 89 · see citing works
Singal, A. G., Rahimi, R. S., Clark, C., Ma, Y., Cuthbert, J. A., Rockey, D. C., & Amarasingham, R. (2013) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2013.03.022 · Cited by 83 · see citing works
Self-citationAmarasingham, R., Velasco, F. T., Xie, B., Clark, C., Ma, Y., Zhang, S., Bhat, D., Lucena, B., Huesch, M. D., & Halm, E. A. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-015-0162-6 · Cited by 71 · see citing works
Self-citationNijhawan, A. E., Clark, C., Kaplan, R., Moore, B. J., Halm, E. A., & Amarasingham, R. (2012) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0b013e31826ebc83 · Cited by 62 · see citing works
Self-citationWatson, A. J., O’Rourke, J. A., Jethwani, K., Cami, A., Stern, T. A., Kvedar, J. C., Chueh, H. C., & Zai, A. (2011) · Psychosomatics · doi:10.1016/j.psym.2011.02.007 · Cited by 52 · see citing works
Bradley, E. H., Sipsma, H. L., Horwitz, L. I., Ndumele, C. D., Brewster, A. L., Curry, L. A., & Krumholz, H. M. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-3105-5 · Cited by 46 · see citing works
Nguyen, O. K., Chan, C., Makam, A. N., Stieglitz, H., & Amarasingham, R. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-2969-8 · Cited by 39 · see citing works
Self-citationClinical and sociodemographic risk factors for readmission of Medicare beneficiaries.
Holloway, J. J., Thomas, J. W., & Shapiro, L. N. (1988) · PubMed Central · Cited by 39 · see citing works
Effect of Clinical and Social Risk Factors on Hospital Profiling for Stroke Readmission
Keyhani, S., Myers, L. J., Cheng, E. M., Hebert, P. L., Williams, L. S., & Bravata, D. M. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0361 · Cited by 34 · see citing works
Patient Factors Contributing to Variation in Same-Hospital Readmission Rate
Henke, R. M., Karaca, Z., Lin, H., et al. (2015) · Medical Care Research and Review · doi:10.1177/1077558715577478 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648 · see citing works
“, than did traditional models. To the best of our knowledge, our models outperform existing EHR literature for predicting mortality (0.92-0.94 vs 0.91), [93] unexpected readmission (0.75-0.76 vs 0.69)[71] and increased length of stay (0.85-0.86 vs 0.77) [62]. Direct comparisons to other studies are difficult [98] becaus…”
ResultInfluentialA prescription for note bloat: an effective progress note template
Kahn, D., Stewart, E., Duncan, M., Lee, E., Simon, W., Lee, C., Friedman, J., Mosher, H., Harris, K., Bell, J., et al. (2018) · Journal of Hospital Medicine · doi:10.12788/jhm.2898 · Cited by 65 · see citing works
- Authors
- Nguyen, O. K., Makam, A. N., Clark, C., Zhang, S., Xie, B., Velasco, F., Amarasingham, R., & Halm, E. A.
- Affiliations
- The University of Texas Southwestern Medical Center (US)
- Parkland Center For Clinical Innovation (US)
- Texas Health Resources Dallas Texas
- Venue
- Journal of Hospital Medicine
- Publisher
- Wiley
- Published
- 2016-02-29
- Volume
- 11
- Issue
- 7
- Pages
- 473-480
- ISSN
- 1553-5592 (ISSN-L)
- 1553-5606
- PMC
- PMC5365027
- OpenAlex
- W2279842060
- Cited by
- 76
- References
- 39
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 3 · 2023: 4 · 2022: 2 · 2021: 12 · 2020: 10 · 2019: 8 · 2018: 13 · 2017: 11 · 2016: 4
- FWCI
- 9.86
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Hospital Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- Agency for Healthcare Research and Quality-funded UT Southwestern Center for Patient-Centered Outcomes Research (1R24HS022418-01)
- Commonwealth Foundation (#20100323)
- UT Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103)
- NIH/NCATS (U54 RFA-TR-12-006)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Emergency Medicine › Hospital Admissions and Outcomes
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- all-cause readmission, electronic health record data, hospital model, Clostridium difficile infection, hyponatremia, length of hospital stay, model discrimination, net reclassification improvement, safety-net hospitals, teaching hospitals
- MeSH terms
- Emergency Service, Hospital, Female, Hospitalization, Humans, Length of Stay, Male, Middle Aged, Models, Theoretical, Patient Readmission, Risk Factors, Texas, Cohort Studies, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
