The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Prediction Models for Prolonged Intensive Care Unit Stay After Cardiac Surgery: Systematic Review and Validation Study
Open accessAbstract
Background— Several models have been developed to predict prolonged stay in the intensive care unit (ICU) after cardiac surgery. However, no extensive quantitative validation of these models has yet been conducted. This study sought to identify and validate existing prediction models for prolonged ICU length of stay after cardiac surgery. Methods and Results— After a systematic review of the literature, the identified models were applied on a large registry database comprising 11 395 cardiac surgical interventions. The probabilities of prolonged ICU length of stay based on the models were compared with the actual outcome to assess the discrimination and calibration performance of the models. Literature review identified 20 models, of which 14 could be included. Of the 6 models for the general cardiac surgery population, the Parsonnet model showed the best discrimination (area under the receiver operating characteristic curve=0.75 [95% confidence interval, 0.73 to 0.76]), followed by the European system for cardiac operative risk evaluation (EuroSCORE) (0.71 [0.70 to 0.72]) and a model by Huijskes and colleagues (0.71 [0.70 to 0.73]). Most of the models showed good calibration. Conclusions— In this validation of prediction models for prolonged ICU length of stay, 2 widely implemented models (Parsonnet, EuroSCORE), although originally designed for prediction of mortality, were superior in identifying patients with prolonged ICU length of stay.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many reviews have showed that the quality of reporting in published articles describing the development or validation of multivariable prediction models in medicine is poor [23–27,29].”
From Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (2015) · cites this paper as [29]
In the citing paper
“However, the overwhelming evidence shows that the quality of reporting of prediction model studies is poor.”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [29]
In the citing paper
“Many reviews have showed that the quality of reporting in published articles describing the development or validation of multivariable prediction models in medicine is poor.23–27,29”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
European system for cardiac operative risk evaluation (EuroSCORE)
Nashef, S. A. M., Roques, F., Michel, P., Gauducheau, E., Lemeshow, S., & Salamon, R. (1999) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(99)00134-7 · Cited by 3,362
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Review: a gentle introduction to imputation of missing values.
Donders, A. R. T., van der Heijden, G. J. M. G., Stijnen, T., & Moons, K. G. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.014 · Cited by 2,582
Self-citationUse and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,156
Roques, F. (1999) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(99)00106-2 · Cited by 1,749
Prognosis and prognostic research: what, why, and how?
Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337
Self-citationAssessing the Generalizability of Prognostic Information
Justice, A. C., Covinsky, K. E., & Berlin, J. A. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00016 · Cited by 1,160
Flagged first, then by citations
1 of the 52 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- 10.1161/circ.99.16.2098 · DOI
Cited by
Sorted by citations
Items 101–129 of 129
Predicting and preventing postoperative decline in older cardiac surgery patients
Ettema, R. G. A. (2014) · Utrecht University Repository (Utrecht University) · Cited by 3
Self-citationPreoperative prediction of intensive care unit stay following cardiac surgery
Polat, A., & Polat, E. B. (2011) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/j.ejcts.2011.03.025 · Cited by 3
Kesumarini, D., Widyastuti, Y., Boom, C. E., & Dinarti, L. K. (2023) · World Journal for Pediatric and Congenital Heart Surgery · doi:10.1177/21501351231191456 · Cited by 2
Predicting Intensive Care Unit Length of Stay After Cardiac Surgery
Litton, E., McCann, M., & van Haren, F. M. P. (2018) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2018.04.010 · Cited by 2
Panza, G. A., McKay, R. G., Collazo, S., Loya, D., Burke‐Martindale, C. H., Mather, J. F., & Hashim, S. W. (2025) · Journal of Cardiac Surgery · doi:10.1155/jocs/6654088 · Cited by 1
Chaudhry, S., Jadran, A., Khan, H. A. R., Chaudhry, R., Chaudhry, D., Laleka, Z., Bashir, S., Fahadan, P. T., Altaweel, L. R., Shuaib, A. M., et al. (2025) · Neurological Research · doi:10.1080/01616412.2025.2559308 · Cited by 1
Critical care management of patients following transcatheter aortic valve replacement
Raiten, J. M., Gutsche, J. T., Horák, J., & Augoustides, J. G. (2013) · F1000Research · doi:10.3410/f1000research.2-62.v1 · Cited by 1
Predicting prolonged intensive care unit stay following surgery in adults with Tetralogy of Fallot
Kuwahara, Y., Saji, M., Yazaki, S., Kishiki, K., Yoshikawa, T., Komori, Y., Wada, N., Shimizu, J., & Isobe, M. (2022) · International Journal of Cardiology Congenital Heart Disease · doi:10.1016/j.ijcchd.2022.100421 · Cited by 1
Moons, K. G. M., Altman, D. G., Reitsma, J. B., loannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, P. G. S. (2023) · Вопросы современной педиатрии · doi:10.15690/vsp.v22i2.2557 · Cited by 1
Self-citationMeta-analysis of clinical prediction models
Debray, T. P. A. (2013) · Utrecht University Repository (Utrecht University) · doi:10.7490/f1000research.445.1
Salameh, B. S., Darwazah, A. K., Verhaaren, H. A., & Nooten, G. V. (2013) · World Journal of Cardiovascular Diseases · doi:10.4236/wjcd.2013.34a008
P-49 Early postoperative support profile and mortality in a cardiothoracic-vascular surgery ICU
Paino, R., Nonini, S., Milazzo, F., Mondino, M. G., Zannoli, S., & Cannata, A. (2011) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2011.03.142
Englberger, L. (2014) · International Journal of Physical Medicine & Rehabilitation · doi:10.4172/2329-9096.s5-007
Karmacharya, R. M., Xi-gang, G., Jianjie, G., Haichen, W., Yang, Y., & Yong-xin, L. (2013) · Nepalese Heart Journal · doi:10.3126/njh.v9i1.8341
Creagh-Brown, B. C. (2011) · Spiral (Imperial College London) · doi:10.25560/7057
Improving the Value of Clinical Quality Registries Through Data Linkage
Litton, E., & Lim, J. (2018) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2018.02.036
Jiang, D., & Shen, Y. (2019) · Expert Review of Molecular Diagnostics · doi:10.1080/14737159.2019.1692656
Commentary: Cutoffs and Tradeoffs: Predicting Prolonged Length of Stay After Routine Cardiac Surgery
Cerullo, M. (2021) · Seminars in Thoracic and Cardiovascular Surgery · doi:10.1053/j.semtcvs.2021.03.019
Editors, T. (2011) · Circulation · doi:10.1161/circulationaha.111.059758
Otokiti, A. U., Shih, H.-J., Ozoude, M. M., Siguachi, I., Warsame, L., Williams, K. S., & Akinloye, S. J. (2023) · doi:10.2196/preprints.55618
Agámez, D. M. A., García, N. Y. L., Quintín, M. J. N., Rodriguez, B. F., Rodríguez, E. E., & Lima, D. R. R. (2025) · Research Square · doi:10.21203/rs.3.rs-5903495/v1
Liu, C., Qi, Y., Liu, X., Chen, M., Xiong, Y., Huang, S., Zou, K., Tan, J., & Sun, X. (2023) · Figshare · doi:10.6084/m9.figshare.22599989
Zhang, H., Chen, W., Zhao, Y., Guan, L., Yu, M., Wang, R., & Chen, X. (2021) · Figshare · doi:10.6084/m9.figshare.14532146
Liu, C., Qi, Y., Liu, X., Chen, M., Xiong, Y., Huang, S., Zou, K., Tan, J., & Sun, X. (2023) · Figshare · doi:10.6084/m9.figshare.22599986
Liu, C., Qi, Y., Liu, X., Chen, M., Xiong, Y., Huang, S., Zou, K., Tan, J., & Sun, X. (2023) · Figshare · doi:10.6084/m9.figshare.22599992
İştar, H., Harmandar, B., Toker, M. K., & Türkiş, F. C. (2025) · Muğla Sıtkı Koçman Üniversitesi Tıp Dergisi · doi:10.47572/muskutd.1682439
Predictors of Prolonged Intensive Care Unit Stay After Heart Valve Surgery: A Case–Control Study
Lee, J.-H., & Choi, H.-R. (2026) · The Journal of Cardiovascular Nursing · doi:10.1097/jcn.0000000000001340
Anesthésie-réanimation en chirurgie cardiaque
Provenchère, S., & Iung, B. (2014) · Arnette eBooks · doi:10.3917/arne.fella.2014.01.0703
Gao, F., Cheng, X., Lu, W., & Wang, L. (2026) · Research Square · doi:10.21203/rs.3.rs-10411663/v1
Metadata
- Authors
- Ettema, R. G. A., Peelen, L. M., Schuurmans, M. J., Nierich, A. P., Kalkman, C. J., & Moons, K. G. M.
- Affiliations
- University of Applied Sciences Utrecht (NL)
- ORCID
- Venue
- Circulation
- Published
- 2010-08-17
- Volume
- 122
- Issue
- 7
- Pages
- 682-689
- ISSN
- 0009-7322 (ISSN-L)
- 1524-4539
- OpenAlex
- W1998435896
- Cited by
- 130
- References
- 52
- Citations by year
- 2026: 2 · 2025: 6 · 2023: 9 · 2022: 6 · 2021: 11 · 2020: 6 · 2019: 4 · 2018: 8 · 2017: 7 · 2016: 8 · 2015: 14 · 2014: 15 · 2013: 15 · 2012: 9
- FWCI
- 7.38
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Circulation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac Valve Diseases and Treatments
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac, Anesthesia and Surgical Outcomes
- Health Sciences › Medicine › Surgery › Cardiac and Coronary Surgery Techniques
- Keywords
- cardiac surgery, prediction model, model validation, EuroSCORE, model discrimination, model calibration, postoperative outcomes
- MeSH terms
- Aged, Female, Cardiac Surgical Procedures, Humans, Intensive Care Units, Length of Stay, Male, Middle Aged, Postoperative Period, Predictive Value of Tests, Risk Factors, Time Factors, Models, Statistical, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
