Prediction of 30-Day All-Cause Readmissions in Patients Hospitalized for Heart Failure: Comparison of Machine Learning and Other Statistical Approaches
Open accessAbstract
Importance: Several attempts have been made at developing models to predict 30-day readmissions in patients with heart failure, but none have sufficient discriminatory capacity for clinical use. Machine-learning (ML) algorithms represent a novel approach and may have potential advantages over traditional statistical modeling. Objective: To develop models using a ML approach to predict all-cause readmissions 30 days after discharge from a heart failure hospitalization and to compare ML model performance with models developed using "conventional" statistically based methods. Design, Setting, and Participants: Models were developed using ML algorithms, specifically, a tree-augmented naive Bayesian network, a random forest algorithm, and a gradient-boosted model and compared with traditional statistical methods using 2 independently derived logistic regression models (a de novo model and an a priori model developed using electronic health records) and a least absolute shrinkage and selection operator method. The study sample was randomly divided into training (70%) and validation (30%) sets to develop and test model performance. This was a registry-based study, and the study sample was obtained by linking patients from the Get With the Guidelines Heart Failure registry with Medicare data. After applying appropriate inclusion and exclusion criteria, 56 477 patients were included in our analysis. The study was conducted between January 4, 2005, and December 1, 2010, and analysis of the data was conducted between November 25, 2014, and June 30, 2016. Main Outcomes and Measures: C statistics were used for comparison of discriminatory capacity across models in the validation sample. Results: The overall 30-day rehospitalization rate was 21.2% (11 959 of 56 477 patients). For the tree-augmented naive Bayesian network, random forest, gradient-boosted, logistic regression, and least absolute shrinkage and selection operator models, C statistics for the validation sets were similar: 0.618, 0.607, 0.614, 0.624, and 0.618, respectively. Applying the previously validated electronic health records model to our study sample yielded a C statistic of 0.589 for the validation set. Conclusions and Relevance: Use of a number of ML algorithms did not improve prediction of 30-day heart failure readmissions compared with more traditional prediction models. Although there will likely be further applications of ML approaches in prognostic modeling, our study fits within the literature of limited predictive ability for heart failure readmissions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies have shown the promise of machine learning in predicting events like unexpected 30-day readmission and identifying high-need high-cost patients that could preempt avoidable health expenses.
“Also, studies have shown the promise of machine learning in predicting events like unexpected 30-day readmission8,23 and identifying high-need high-cost patients24 that could preempt avoidable health expenses.”
From Early prediction of diagnostic-related groups and estimation of hospital cost by processing clinical notes (2021) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
2013 ACCF/AHA Guideline for the Management of Heart Failure
Yancy, C. W., Jessup, M., Bozkurt, B., Butler, J., Casey, D. E., Drazner, M. H., Fonarow, G. C., Geraci, S. A., Horwich, T., Januzzi, J. L., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.05.019 · Cited by 12,536 · see citing works
Self-citationRisk Prediction Models for Hospital Readmission
Kansagara, D. L., Englander, H. L., Salanitro, A. H., Kagen, D. I., Theobald, C. N., Freeman, M., & Kripalani, S. B. (2011) · JAMA · doi:10.1001/jama.2011.1515 · Cited by 1,712 · see citing works
Dharmarajan, K., Hsieh, A., Lin, Z., Bueno, H., Ross, J. S., Horwitz, L. I., Barreto-Filho, J. A. S., Kim, N. V., Bernheim, S. M., Suter, L. G., et al. (2013) · JAMA · doi:10.1001/jama.2012.216476 · Cited by 1,059 · see citing works
Keenan, P. S., Normand, S. T., Lin, Z., Drye, E. E., Bhat, K. R., Ross, J. S., Schuur, J. D., Stauffer, B. D., Bernheim, S. M., Epstein, A. J., et al. (2008) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.108.802686 · Cited by 501 · see citing works
Predictors of readmission among elderly survivors of admission with heart failure
Krumholz, H. M., Chen, Y.-T., Wang, Y., Vaccarino, V., Radford, M. J., & Horwitz, R. I. (2000) · American Heart Journal · doi:10.1016/s0002-8703(00)90311-9 · Cited by 437 · see citing works
Amarasingham, R., Moore, B. J., Tabak, Y. P., Drazner, M. H., Clark, C. A., Zhang, S., Reed, W. G., Swanson, T. S., Ma, Y., & Halm, E. A. (2010) · Medical Care · doi:10.1097/mlr.0b013e3181ef60d9 · Cited by 437 · see citing works
Linking inpatient clinical registry data to Medicare claims data using indirect identifiers
Hammill, B. G., Hernandez, A. F., Peterson, E. D., Fonarow, G. C., Schulman, K. A., & Curtis, L. H. (2009) · American Heart Journal · doi:10.1016/j.ahj.2009.04.002 · Cited by 390 · see citing works
Self-citationThe Prevention of Hospital Readmissions in Heart Failure
Ziaeian, B., & Fonarow, G. C. (2015) · Progress in Cardiovascular Diseases · doi:10.1016/j.pcad.2015.09.004 · Cited by 261 · see citing works
Self-citationPhilbin, E. F., & DiSalvo, T. G. (1999) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(99)00059-5 · Cited by 254 · see citing works
Eapen, Z. J., Liang, L. Y., Fonarow, G. C., et al. (2013) · JACC Heart Failure · doi:10.1016/j.jchf.2013.01.008 · Cited by 125 · see citing works
Self-citationNational Survey of Hospital Strategies to Reduce Heart Failure Readmissions
Kociol, R. D., Peterson, E. D., Hammill, B. G., Flynn, K. E., Heidenreich, P. A., Piña, I. L., Lytle, B. L., Albert, N. M., Curtis, L. H., Fonarow, G. C., et al. (2012) · Circulation Heart Failure · doi:10.1161/circheartfailure.112.967406 · Cited by 116 · see citing works
Self-citationHammill, B. G., Curtis, L. H., Fonarow, G. C., et al. (2010) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.110.954693 · Cited by 114 · see citing works
Self-citationDo Non-Clinical Factors Improve Prediction of Readmission Risk?
Krumholz, H. M., Chaudhry, S. I., Spertus, J. A., Mattera, J. A., Hodshon, B., & Herrin, J. S. (2015) · JACC Heart Failure · doi:10.1016/j.jchf.2015.07.017 · Cited by 58 · see citing works
Risk stratification in heart failure using artificial neural networks.
Atienza, F., Martínez‐Alzamora, N., Velasco, J. A., Dreiseitl, S., & Lucila Ohno-machado, A. (2000) · PubMed Central · Cited by 29 · see citing works
Predictors of readmission among elderly survivors of admission with heart failure
Henrich, W. L., Krumholz, H. M., Chen, Y.-T. M., Wang, Y. M., Vaccarino, V., & Radford, M. J. (2000) · American Heart Journal · doi:10.1067/mhj.2000.100183 · Cited by 29 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- National Research Council (US) and Institute of Medicine. New opportunities, new challenges: the changing nature of biomedical science. In: (US) Committee on the Organizational Structure of the National Institutes of Health, eds. Enhancing the Vitality of the National Institutes of Health: Organizational Change to Meet New Challenges. Washington, DC: National Academies Press; 2003.
Cited by
Sorted by citations
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Artificial intelligence in cardiology
Johnson, K. W., Torres Soto, J., Glicksberg, B. S., Shameer, K., Miotto, R., Ali, M., Ashley, E., & Dudley, J. T. (2018) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.03.521 · Cited by 1,202 · see citing works
Hatib, F., Jian, Z., Buddi, S., Lee, C., Settels, J., Sibert, K., Rinehart, J., & Cannesson, M. (2018) · Anesthesiology · doi:10.1097/aln.0000000000002300 · Cited by 614 · see citing works
Rajula, H. S. R., Verlato, G., Manchia, M., Antonucci, N., & Fanos, V. (2020) · Medicina · doi:10.3390/medicina56090455 · Cited by 606 · see citing works
Machine learning in cardiovascular medicine: are we there yet?
Shameer, K., Johnson, K. W., Glicksberg, B. S., Dudley, J. T., & Sengupta, P. P. (2018) · Heart · doi:10.1136/heartjnl-2017-311198 · Cited by 555 · see citing works
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101167 · Cited by 400 · see citing works
Armoundas, A. A., Narayan, S. M., Arnett, D. K., Spector-Bagdady, K., Bennett, D. A., Celi, L. A., Friedman, P. A., Gollob, M. H., Hall, J. L., Kwitek, A. E., et al. (2024) · Circulation · doi:10.1161/cir.0000000000001201 · Cited by 347 · see citing works
Golas, S. B., Shibahara, T., Agboola, S., Otaki, H., Sato, J., Nakae, T., Hisamitsu, T., Kojima, G., Felsted, J., Kakarmath, S., et al. (2018) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-018-0620-z · Cited by 287 · see citing works
Lee, C. K., Hofer, I., Gabel, E., Baldi, P., & Cannesson, M. (2018) · Anesthesiology · doi:10.1097/aln.0000000000002186 · Cited by 185 · see citing works
Putting the data before the algorithm in big data addressing personalized healthcare
Cahan, E. M., Hernandez-Boussard, T., Thadaney-Israni, S., & Rubin, D. L. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0157-2 · Cited by 149 · see citing works
Lynam, A. L., Dennis, J. M., Owen, K. R., Oram, R. A., Jones, A. G., Shields, B. M., & Ferrat, L. A. (2020) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00075-2 · Cited by 142 · see citing works
Applications of Artificial Intelligence in Cardiology. The Future is Already Here
Dorado-Díaz, P. I., Sampedro-Gómez, J., Vicente-Palacios, V., & Sánchez, P. L. (2019) · Revista Española de Cardiología (English Edition) · doi:10.1016/j.rec.2019.05.014 · Cited by 111 · see citing works
Machine learning to assist clinical decision-making during the COVID-19 pandemic
Debnath, S., Barnaby, D. P., Coppa, K., Makhnevich, A., Kim, E. J., Chatterjee, S., Tóth, V., Levy, T. J., Paradis, M. D., Cohen, S. L., et al. (2020) · Bioelectronic Medicine · doi:10.1186/s42234-020-00050-8 · Cited by 104 · see citing works
Miller, P. E., Pawar, S., Vaccaro, B., McCullough, M., Rao, P., Ghosh, R., Warier, P., Desai, N. R., & Ahmad, T. (2019) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2019.01.018 · Cited by 81 · see citing works
Varma, N., Cygankiewicz, I., Turakhia, M. P., Heidbuchel, H., Hu, Y.-F., Chen, L. Y., Couderc, J.-P., Cronin, E. M., Estep, J. D., Grieten, L., et al. (2021) · Circulation: Arrhythmia and Electrophysiology · doi:10.1161/circep.120.009204 · Cited by 74 · see citing works
Liu, J., Capurro, D., Nguyen, A., & Verspoor, K. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00474-9 · Cited by 50 · see citing works
Chiu, Y. M., Courteau, J., Dufour, I., Vanasse, A., & Hudon, C. (2023) · Scientific Reports · doi:10.1038/s41598-023-27568-6 · Cited by 20 · see citing works
- Authors
- Frizzell, J. D., Liang, L., Schulte, P. J., Yancy, C. W., Heidenreich, P. A., Hernandez, A. F., Bhatt, D. L., Fonarow, G. C., & Laskey, W. K.
- Affiliations
- University of New Mexico (US)
- Duke University (US)
- Mayo Clinic (US)
- Northwestern University (US)
- VA Palo Alto Health Care System (US)
- Brigham and Women's Hospital (US)
- University of California, Los Angeles (US)
- Corresponding author
- Laskey, Warren K.
- ORCID
- Venue
- JAMA Cardiology
- Publisher
- American Medical Association (AMA)
- Published
- 2017-02-01
- Volume
- 2
- Issue
- 2
- Pages
- 204
- ISSN
- 2380-6583 (ISSN-L)
- OpenAlex
- W2542719835
- Cited by
- 361
- References
- 15
- Citations by year
- 2026: 18 · 2025: 35 · 2024: 31 · 2023: 44 · 2022: 32 · 2021: 100 · 2020: 41 · 2019: 38 · 2018: 18 · 2017: 4
- FWCI
- 16.37
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- JAMA Cardiology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- American Heart Association
- Amarin Corporation
- Topics
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Failure Treatment and Management
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- heart failure, 30-day readmission, machine learning, random forest, gradient boosting, logistic regression, LASSO, prognostic model, electronic health records, Medicare beneficiaries, model discrimination, concordance index, discharge planning
- MeSH terms
- Machine Learning, Algorithms, Follow-Up Studies, Heart Failure, Humans, Patient Readmission, Predictive Value of Tests, Prognosis, Registries, United States, Models, Statistical, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
