The DOI matches its Crossref record; 51 of the 52 references it cites were found in a registry or an index.
Prediction Models for Prolonged Intensive Care Unit Stay After Cardiac Surgery: Systematic Review and Validation Study
Open accessAbstract
Background— Several models have been developed to predict prolonged stay in the intensive care unit (ICU) after cardiac surgery. However, no extensive quantitative validation of these models has yet been conducted. This study sought to identify and validate existing prediction models for prolonged ICU length of stay after cardiac surgery. Methods and Results— After a systematic review of the literature, the identified models were applied on a large registry database comprising 11 395 cardiac surgical interventions. The probabilities of prolonged ICU length of stay based on the models were compared with the actual outcome to assess the discrimination and calibration performance of the models. Literature review identified 20 models, of which 14 could be included. Of the 6 models for the general cardiac surgery population, the Parsonnet model showed the best discrimination (area under the receiver operating characteristic curve=0.75 [95% confidence interval, 0.73 to 0.76]), followed by the European system for cardiac operative risk evaluation (EuroSCORE) (0.71 [0.70 to 0.72]) and a model by Huijskes and colleagues (0.71 [0.70 to 0.73]). Most of the models showed good calibration. Conclusions— In this validation of prediction models for prolonged ICU length of stay, 2 widely implemented models (Parsonnet, EuroSCORE), although originally designed for prediction of mortality, were superior in identifying patients with prolonged ICU length of stay.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many reviews have showed that the quality of reporting in published articles describing the development or validation of multivariable prediction models in medicine is poor [23–27,29].”
From Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement (2015) · cites this paper as [29]
In the citing paper
“However, the overwhelming evidence shows that the quality of reporting of prediction model studies is poor.”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [29]
In the citing paper
“Many reviews have showed that the quality of reporting in published articles describing the development or validation of multivariable prediction models in medicine is poor.23–27,29”
From Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
European system for cardiac operative risk evaluation (EuroSCORE)
Nashef, S. A. M., Roques, F., Michel, P., Gauducheau, E., Lemeshow, S., & Salamon, R. (1999) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(99)00134-7 · Cited by 3,362
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Review: a gentle introduction to imputation of missing values.
Donders, A. R. T., van der Heijden, G. J. M. G., Stijnen, T., & Moons, K. G. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.014 · Cited by 2,582
Self-citationUse and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,156
Roques, F. (1999) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(99)00106-2 · Cited by 1,749
Prognosis and prognostic research: what, why, and how?
Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337
Self-citationAssessing the Generalizability of Prognostic Information
Justice, A. C., Covinsky, K. E., & Berlin, J. A. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00016 · Cited by 1,160
Parsonnet, V., DEAN, D. L., & Bernstein, A. D. (1989) · PubMed · Cited by 1,112
Reilly, B. M., & Evans, A. T. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-3-200602070-00009 · Cited by 794
Laupacis, A. (1997) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1997.03540300056034 · Cited by 793
Missing covariate data in medical research: To impute is better than to ignore
Janssen, K. J. M., Donders, A. R. T., Harrell, F. E., Vergouwe, Y., Chen, Q., Grobbee, D. E., & Moons, K. G. M. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.12.008 · Cited by 593
Self-citationUpdating methods improved the performance of a clinical prediction model in new patients
Janssen, K. J. M., Moons, K. G. M., Kalkman, C. J., Grobbee, D. E., & Vergouwe, Y. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.04.018 · Cited by 395
Self-citationProbabilistic prediction in patient management and clinical trials
Spiegelhalter, D. J. (1986) · Statistics in Medicine · doi:10.1002/sim.4780050506 · Cited by 362
Risk stratification in heart surgery: comparison of six score systems
Geissler, H. J. (2000) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(00)00385-7 · Cited by 334
Morbidity and Duration of ICU Stay after Cardiac Surgery
Turnan, K. J., McCarthy, R. J., MARCH, R. J., Najafi, H., & Ivankovich, A. D. (1992) · CHEST Journal · doi:10.1378/chest.102.1.36 · Cited by 234
Wong, D. T., Cheng, D., Kustra, R., Tibshirani, R. J., Karski, J. M., Carroll-Munro, J., & Sandler, A. N. (1999) · Anesthesiology · doi:10.1097/00000542-199910000-00012 · Cited by 195
Prolonged Intensive Care Unit Stay in Cardiac Surgery: Risk Factors and Long-Term-Survival
Hein, O. V., Birnbaum, J., Wernecke, K. D., England, M. R., Konertz, W. F., & Spies, C. (2006) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2005.09.077 · Cited by 182
Family History of Premature Coronary Heart Disease and Coronary Artery Calcification
Nasir, K., Budoff, M. J., Wong, N. D., Scheuner, M. T., Herrington, D. M., Arnett, D. K., Szklo, M., Greenland, P., & Blumenthal, R. S. (2007) · Circulation · doi:10.1161/circulationaha.107.688739 · Cited by 174
Ready-Made, Recalibrated, or Remodeled?
Ivanov, J., Tu, J. V., & Naylor, C. D. (1999) · Circulation · doi:10.1161/01.cir.99.16.2098 · Cited by 158
Scott, B. H., Seifert, F. C., GRIMSON, R. C., & Glass, P. S. A. (2005) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2005.03.030 · Cited by 128
EuroSCORE Predicts Intensive Care Unit Stay and Costs of Open Heart Surgery
Nilsson, J., Algotsson, L., Höglund, P., Lührs, C., & Brandt, J. (2004) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2004.04.060 · Cited by 119
Fast-Track Anesthesia and Cardiac Surgery: A Retrospective Cohort Study of 7989 Patients
Svircevic, V., Nierich, A. P., Moons, K. G. M., Bruinsma, G. J. B. B., Kalkman, C. J., & van Dijk, D. (2009) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e318193c423 · Cited by 106
Self-citationGhotkar, S. V., Grayson, A. D., Fabri, B. M., Dihmis, W. C., & Pullan, D. M. (2006) · Journal of Cardiothoracic Surgery · doi:10.1186/1749-8090-1-14 · Cited by 100
Prognostic models based on literature and individual patient data in logistic regression analysis
Steyerberg, E. W., Eijkemans, M. J. C., van Houwelingen, H. C., Lee, K. L., & Habbema, J. D. F. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000130)19:2 · Cited by 90
Long-Term Survival of the Very Elderly Undergoing Coronary Artery Bypass Grafting
Likosky, D. S., Dacey, L. J., Baribeau, Y. R., Leavitt, B. J., Clough, R. A., Cochran, R. P., Quinn, R. D., Sisto, D., Charlesworth, D. C., Malenka, D. J., et al. (2008) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2007.12.066 · Cited by 88
Pätilä, T., Kukkonen, S., Vento, A. E., Pettilä, V., & Suojaranta-Ylinen, R. T. (2006) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2006.06.025 · Cited by 79
A Predictive Index for Length of Stay in the Intensive Care Unit Following Cardiac Surgery
Tu, J. V., Mazer, C. D., Levinton, C. M., Armstrong, P. W., & Naylor, C. D. (1995) · Survey of Anesthesiology · doi:10.1097/00132586-199504000-00013 · Cited by 76
Lawrence, D., Valencia, O., Smith, E. E. J., Murday, A., & Treasure, T. (2000) · Heart · doi:10.1136/heart.83.4.429 · Cited by 76
Predictors of prolonged ICU stay after on-pump versus off-pump coronary artery bypass grafting
Bucerius, J. A., Gummert, J. F., Walther, T., Doll, N., Falk, V., Schmitt, D. V., & Mohr, F. W. (2004) · Intensive Care Medicine · doi:10.1007/s00134-003-1950-5 · Cited by 75
Pitkänen, O., Niskanen, M. M., Rehnberg, S., Hippeläinen, M., & Hynynen, M. J. (2000) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(00)00579-0 · Cited by 73
Parolari, A., Pesce, L. L., Trezzi, M., Loardi, C. M., Kassem, S., Brambillasca, C., Miguel, B., Tremoli, E., Biglioli, P., & Alamanni, F. (2008) · European Heart Journal · doi:10.1093/eurheartj/ehn581 · Cited by 72
Risk Factors for Prolonged Stay in the Intensive Care Unit and on the Ward After Cardiac Surgery
Atoui, R., Ma, F., Langlois, Y., & Morin, J. (2008) · Journal of Cardiac Surgery · doi:10.1111/j.1540-8191.2007.00564.x · Cited by 72
Preoperative prediction of prolonged stay in the intensive care unit for coronary bypass surgery
Janssen, D. P. B. (2003) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/j.ejcts.2003.11.005 · Cited by 70
Christakis, G. T. (1996) · Cardiovascular Surgery · doi:10.1016/0967-2109(96)83780-x · Cited by 56
Determinants of duration of ICU stay after coronary artery bypass graft surgery
Michalopoulos, A., Tzelepis, G., Pavlides, G., Kriaras, J., Dafni, U., & Geroulanos, S. (1996) · British Journal of Anaesthesia · doi:10.1093/bja/77.2.208 · Cited by 55
Can EuroSCORE predict direct costs of cardiac surgery?
Pintor, P. P., Bobbio, M., Colangelo, S., Veglia, F., Marras, R., & Diena, M. (2003) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(02)00868-0 · Cited by 54
Rosenfeld, R., Smith, J. M., Woods, S. E., & Engel, A. M. (2006) · Journal of Cardiac Surgery · doi:10.1111/j.1540-8191.2006.00196.x · Cited by 49
Messaoudi, N., Cocker, J. D., Stockman, B. A., Bossaert, L. L., & Rodrigus, I. E. R. (2008) · Journal of Cardiac Surgery · doi:10.1111/j.1540-8191.2008.00716.x · Cited by 47
Opposite trends in coronary artery and valve surgery in a large multisurgeon practice, 1979–1999
Northrup, W. F., Emery, R. W., Nicoloff, D. M., Lillehei, T. J., Holter, A. R., & Blake, D. P. (2004) · The Annals of Thoracic Surgery · doi:10.1016/s0003-4975(03)01359-6 · Cited by 44
Preoperative risk prediction and intraoperative events in cardiac surgery
Stoica, S. C. (2002) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(01)01077-6 · Cited by 43
Huijskes, R. V. H. P., Rosseel, P. M. J., & Tijssen, J. G. P. (2003) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(03)00471-8 · Cited by 38
Ivanov, J., Borger, M. A., David, T. E., Cohen, G. N., Walton, N. A., & Naylor, C. D. (2000) · The Annals of Thoracic Surgery · doi:10.1016/s0003-4975(00)01387-4 · Cited by 31
Determinants of Morbidity and Intensive Care Unit Stay after Coronary Surgery
Abrahamyan, L., Demirchyan, A., Thompson, M. E., & Hovaguimian, H. (2006) · Asian Cardiovascular and Thoracic Annals · doi:10.1177/021849230601400207 · Cited by 27
Off-pump aortic valve replacement with catheter-mounted valved stents. Is the future already here?
Antunes, M. J. (2006) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/j.ejcts.2006.10.001 · Cited by 16
Ouattara, A., Niculescu, M., Boccara, G., Landi, M., Vaissier, E., Léger, P., Riou, B., Gandjbackch, I., & Coriat, P. (2003) · Annales Françaises d Anesthésie et de Réanimation · doi:10.1016/s0750-7658(03)00058-3 · Cited by 14
Prediction for Major Adverse Outcomes in Cardiac Surgery: Comparison of Three Prediction Models
Hsieh, C.-H., Peng, S.-K., Tsai, T.-C., Shih, Y.-R., & Peng, S.-Y. (2007) · Journal of the Formosan Medical Association · doi:10.1016/s0929-6646(08)60037-6 · Cited by 12
Can ICU stay be predicted accurately enough for fasttracking cardiac surgical patients?
Salamonsen, M., Bailey, M., & Salamonsen, R. F. (2008) · Critical Care and Resuscitation · doi:10.1016/s1441-2772(23)01019-0 · Cited by 6
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4
Research works printed but not linked: 1
- 10.1161/circ.99.16.2098 · DOI
Cited by
Sorted by citations
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationCollins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,319
Self-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citationMoons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Self-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. (2015) · BMC Medicine · doi:10.1186/s12916-014-0241-z · Cited by 1,927
Self-citation
Metadata
- Authors
- Ettema, R. G. A., Peelen, L. M., Schuurmans, M. J., Nierich, A. P., Kalkman, C. J., & Moons, K. G. M.
- Affiliations
- University of Applied Sciences Utrecht (NL)
- ORCID
- Venue
- Circulation
- Published
- 2010-08-17
- Volume
- 122
- Issue
- 7
- Pages
- 682-689
- ISSN
- 0009-7322 (ISSN-L)
- 1524-4539
- OpenAlex
- W1998435896
- Cited by
- 130
- References
- 52
- Citations by year
- 2026: 2 · 2025: 6 · 2023: 9 · 2022: 6 · 2021: 11 · 2020: 6 · 2019: 4 · 2018: 8 · 2017: 7 · 2016: 8 · 2015: 14 · 2014: 15 · 2013: 15 · 2012: 9
- FWCI
- 7.38
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Circulation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac Valve Diseases and Treatments
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiac, Anesthesia and Surgical Outcomes
- Health Sciences › Medicine › Surgery › Cardiac and Coronary Surgery Techniques
- Keywords
- cardiac surgery, prediction model, model validation, EuroSCORE, model discrimination, model calibration, postoperative outcomes
- MeSH terms
- Aged, Female, Cardiac Surgical Procedures, Humans, Intensive Care Units, Length of Stay, Male, Middle Aged, Postoperative Period, Predictive Value of Tests, Risk Factors, Time Factors, Models, Statistical, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
