The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
A comparison of imputation techniques for handling missing predictor values in a risk model with a binary outcome
Abstract
Risk models that aim to predict the future course and outcome of disease processes are increasingly used in health research, and it is important that they are accurate and reliable. Most of these risk models are fitted using routinely collected data in hospitals or general practices. Clinical outcomes such as short-term mortality will be near-complete, but many of the predictors may have missing values. A common approach to dealing with this is to perform a complete-case analysis. However, this may lead to overfitted models and biased estimates if entire patient subgroups are excluded. The aim of this paper is to investigate a number of methods for imputing missing data to evaluate their effect on risk model estimation and the reliability of the predictions. Multiple imputation methods, including hotdecking and multiple imputation by chained equations (MICE), were investigated along with several single imputation methods. A large national cardiac surgery database was used to create simulated yet realistic datasets. The results suggest that complete case analysis may produce unreliable risk predictions and should be avoided. Conditional mean imputation performed well in our scenario, but may not be appropriate if using variable selection methods. MICE was amongst the best performing multiple imputation methods with regards to the quality of the predictions. Additionally, it produced the least biased estimates, with good coverage, and hence is recommended for use in practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We examined missing data bias using multiple imputation (37) and whether correcting random measurement error in health behaviors increased their capacity to explain observed SES/ personality risks, using simulation extrapolation (38).”
From Personality, Socioeconomic Status, and All-Cause Mortality in the United States (2010) · cites this paper as [37]
Citations
Cites
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289
The APACHE III Prognostic System
Knaus, W. A., Wagner, D. P., Draper, E. A., Zimmerman, J. E., Bergner, M., BASTOS, P. G., Sirio, C., Murphy, D. J., Lotring, T., Damiano, A., et al. (1991) · CHEST Journal · doi:10.1378/chest.100.6.1619 · Cited by 3,539
European system for cardiac operative risk evaluation (EuroSCORE)
Nashef, S. A. M., Roques, F., Michel, P., Gauducheau, E., Lemeshow, S., & Salamon, R. (1999) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(99)00134-7 · Cited by 3,362
The APACHE III Prognostic System
Teres, D., & Lemeshow, S. A. (1992) · CHEST Journal · doi:10.1378/chest.102.6.1919b · Cited by 2,896
Statistical Analysis with Missing Data, Third Edition
Little, R. J. A., & Rubin, D. B. (2019) · Wiley series in probability and statistics · doi:10.1002/9781119482260 · Cited by 2,089
Acock, A. (2005) · Journal of Marriage and the Family · doi:10.1111/j.1741-3737.2005.00191.x · Cited by 1,955
Multiple Imputation of Missing Values: Update of Ice
Royston, P. J. (2005) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0500500404 · Cited by 1,038
Regression with Missing X 's: A Review
Little, R. J. A. (1992) · Journal of the American Statistical Association · doi:10.1080/01621459.1992.10476282 · Cited by 968
Risk Adjustment for Measuring Healthcare Outcomes
Iezzoni, L. I. (1994) · Cited by 950
Multiple Imputation for Interval Estimation From Simple Random Samples With Ignorable Nonresponse
Rubin, D. B., & Nathaniel Schenker, A. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478280 · Cited by 768
Ambulatory care groups: a categorization of diagnoses for research and management.
Starfield, B. H., Weiner, J. P., Mumford, L. M., & Steinwachs, D. M. (1991) · Europe PMC (PubMed Central) · Cited by 520
O'Connor, G. T., Plume, S. K., Olmstead, E. M., Coffin, L. H., Morton, J. R., Maloney, C. T., Nowicki, E. R., Levy, D. G., Tryzelaar, J. F., & Hernandez, F. (1992) · Circulation · doi:10.1161/01.cir.85.6.2110 · Cited by 403
Validation techniques for logistic regression models
Miller, M. E., Hui, S. L., & Tierney, W. M. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100805 · Cited by 322
Generic, Simple Risk Stratification Model for Heart Valve Surgery
Ambler, G., Omar, R. Z., Royston, P., Kinsman, R., Keogh, B. E., & Taylor, K. M. (2005) · Circulation · doi:10.1161/circulationaha.104.515049 · Cited by 279
Self-citationBarzi, F. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh175 · Cited by 263
(1992) · Psychiatric Bulletin · doi:10.1192/pb.16.5.303 · Cited by 248
The Society of Thoracic Surgeons National Cardiac Surgery Database: Current Risk Assessment
Edwards, F. H., Grover, F. L., Shroyer, A. L., Schwartz, M., & Bero, J. W. (1997) · The Annals of Thoracic Surgery · doi:10.1016/s0003-4975(97)00017-9 · Cited by 220
Burton, A., & Altman, D. G. (2004) · British Journal of Cancer · doi:10.1038/sj.bjc.6601907 · Cited by 219
Developing a prognostic model in the presence of missing data
Clark, T. G., & Altman, D. G. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(02)00539-5 · Cited by 214
Berrisford, R. G., Brunelli, A., Rocco, G., Treasure, T., Utley, M., & guide, O. B. O. T. A. A. (2005) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/j.ejcts.2005.03.047 · Cited by 174
Can one assess whether missing data are missing at random in medical studies?
Potthoff, R. F., Tudor, G. E., Pieper, K. S., & Hasselblad, V. (2006) · Statistical Methods in Medical Research · doi:10.1191/0962280206sm448oa · Cited by 141
The relative importance of prognostic factors in studies of survival
SCHEMPER, M. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122413 · Cited by 99
Pérez, A., Dennis, R., Gil, F., Rondón, M., & López, A. (2002) · Statistics in Medicine · doi:10.1002/sim.1391 · Cited by 74
Case-mix and variation in specialist referrals in general practice.
Sullivan, C. O., Omar, R., Ambler, G., & Majeed, A. (2005) · Europe PMC (PubMed Central) · Cited by 71
Predicting the risk of repetition after self harm: cohort study
Kapur, N., Cooper, J., Rodway, C., Kelly, J., Guthrie, E., & Mackway-Jones, K. C. (2005) · BMJ · doi:10.1136/bmj.38337.584225.82 · Cited by 69
Methods for estimating the AIDS incubation time distribution when date of seroconversion is censored
Geskus, R. B. (2001) · Statistics in Medicine · doi:10.1002/sim.700 · Cited by 62
Omar, R., Ambler, G., Royston, P. J., Eliahoo, J., & Taylor, K. M. (2004) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2003.10.032 · Cited by 58
A toolkit in SAS for the evaluation of multiple imputation methods
Brand, J., van Buuren, S., Groothuis‐Oudshoorn, C. G. M., & Gelsema, E. S. (2003) · Statistica Neerlandica · doi:10.1111/1467-9574.00219 · Cited by 38
Mander, A., & Clayton, D. G. (2000) · Stata technical bulletin · Cited by 18
Omar, R., Morton, L., Murad, S., & Taylor, K. M. (2003) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/s0022-5223(03)00362-3 · Cited by 2
Multiple imputation of missing blood pressure covariates in survival analysis
van Buuren, S., Boshuizen, H. C., & Knook, D. L. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990330)18:6
Research works printed but not linked: 7
- Rosen AK, 2003, Insights, 4, 1
- Mander A., 2000, Stata Technical Bulletin, 9, 199
- Allison PD, 2001, Missing data
- Fay RE, Proceedings of the Survey Research Methods Sections
- Keogh B., 2002, Systems
- Berrisford R., 2005, Surgery, 28, 306
- O'Sullivan C., 2005, British Journal of General Practice, 55, 529
Cited by
Sorted by citations
Jerez, J. M., Molina, I., García-Laencina, P. J., Alba, E., Ribelles, N., Martín, M., & Franco, L. (2010) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2010.05.002 · Cited by 608
The Effect of Multidisciplinary Care Teams on Intensive Care Unit Mortality
Kim, M. M., Barnato, A. E., Angus, D. C., Fleisher, L., & Kahn, J. M. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.521 · Cited by 531
Inflammatory Markers and Incident Heart Failure Risk in Older Adults
Kalogeropoulos, A., Georgiopoulou, V., Psaty, B. M., Rodondi, N., Smith, A. L., Harrison, D. G., Liu, Y., Hoffmann, U., Bauer, D. C., Newman, A. B., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.12.045 · Cited by 478
Imputation of missing data in life‐history trait datasets: which approach performs the best?
Penone, C., Davidson, A. D., Shoemaker, K. T., Marco, M. D., Rondinini, C., Brooks, T. M., Young, B. E., Graham, C. H., & Costa, G. C. (2014) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12232 · Cited by 452
Gerry, S., Bonnici, T., Birks, J., Kirtley, S., Virdee, P. S., Watkinson, P., & Collins, P. G. S. (2020) · BMJ · doi:10.1136/bmj.m1501 · Cited by 415
Metadata
- Authors
- Ambler, G., Omar, R. Z., & Royston, P.
- Affiliations
- UCL Biomedical Research Centre (GB)
- MRC Clinical Trials Unit at UCL (GB)
- Corresponding author
- Ambler, Gareth
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2007-06
- Volume
- 16
- Issue
- 3
- Pages
- 277-298
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W1965973862
- Cited by
- 233
- References
- 39
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 9 · 2023: 10 · 2022: 6 · 2021: 15 · 2020: 12 · 2019: 13 · 2018: 11 · 2017: 8 · 2016: 11 · 2015: 14 · 2014: 14 · 2013: 12 · 2012: 19
- FWCI
- 15.72
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- missing data imputation, risk prediction model, binary outcomes, multiple imputation, multiple imputation by chained equations, single imputation, complete case analysis, cardiac surgery, mortality prediction, forecast reliability, estimation bias
- MeSH terms
- Aged, Female, United Kingdom, Humans, Male, Middle Aged, Thoracic Surgery, Models, Statistical, Bias, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review