A comparison of imputation techniques for handling missing predictor values in a risk model with a binary outcome
Abstract
Risk models that aim to predict the future course and outcome of disease processes are increasingly used in health research, and it is important that they are accurate and reliable. Most of these risk models are fitted using routinely collected data in hospitals or general practices. Clinical outcomes such as short-term mortality will be near-complete, but many of the predictors may have missing values. A common approach to dealing with this is to perform a complete-case analysis. However, this may lead to overfitted models and biased estimates if entire patient subgroups are excluded. The aim of this paper is to investigate a number of methods for imputing missing data to evaluate their effect on risk model estimation and the reliability of the predictions. Multiple imputation methods, including hotdecking and multiple imputation by chained equations (MICE), were investigated along with several single imputation methods. A large national cardiac surgery database was used to create simulated yet realistic datasets. The results suggest that complete case analysis may produce unreliable risk predictions and should be avoided. Conditional mean imputation performed well in our scenario, but may not be appropriate if using variable selection methods. MICE was amongst the best performing multiple imputation methods with regards to the quality of the predictions. Additionally, it produced the least biased estimates, with good coverage, and hence is recommended for use in practice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Missing data bias was examined using multiple imputation, and whether correcting random measurement error in health behaviors increased their capacity to explain observed SES/personality risks was examined using simulation extrapolation.
“We examined missing data bias using multiple imputation (37) and whether correcting random measurement error in health behaviors increased their capacity to explain observed SES/ personality risks, using simulation extrapolation (38).”
From Personality, Socioeconomic Status, and All-Cause Mortality in the United States (2010) · cites this paper as [37]
Cites
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701 · see citing works
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · see citing works
Prediction of coronary heart disease using risk factor categories
Wilson, P. W. F., D’Agostino, R. B., Levy, D., Belanger, A. M., Silbershatz, H., & Kannel, W. B. (1998) · Circulation · doi:10.1161/01.cir.97.18.1837 · Cited by 9,581 · see citing works
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270 · see citing works
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473 · see citing works
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878 · see citing works
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289 · see citing works
The APACHE III Prognostic System
Knaus, W. A., Wagner, D. P., Draper, E. A., Zimmerman, J. E., Bergner, M., BASTOS, P. G., Sirio, C., Murphy, D. J., Lotring, T., Damiano, A., et al. (1991) · CHEST Journal · doi:10.1378/chest.100.6.1619 · Cited by 3,524 · see citing works
European system for cardiac operative risk evaluation (EuroSCORE)
Nashef, S. A. M., Roques, F., Michel, P., Gauducheau, E., Lemeshow, S., & Salamon, R. (1999) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/s1010-7940(99)00134-7 · Cited by 3,362 · see citing works
The APACHE III Prognostic System
Teres, D., & Lemeshow, S. A. (1992) · CHEST Journal · doi:10.1378/chest.102.6.1919b · Cited by 2,880 · see citing works
Statistical Analysis with Missing Data, Third Edition
Little, R. J. A., & Rubin, D. B. (2019) · Wiley series in probability and statistics · doi:10.1002/9781119482260 · Cited by 2,089 · see citing works
Acock, A. (2005) · Journal of Marriage and the Family · doi:10.1111/j.1741-3737.2005.00191.x · Cited by 1,955 · see citing works
Multiple Imputation of Missing Values: Update of Ice
Royston, P. J. (2005) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0500500404 · Cited by 1,038 · see citing works
Regression with Missing X 's: A Review
Little, R. J. A. (1992) · Journal of the American Statistical Association · doi:10.1080/01621459.1992.10476282 · Cited by 968 · see citing works
Risk Adjustment for Measuring Healthcare Outcomes
Iezzoni, L. I. (1994) · Cited by 950 · see citing works
Multiple Imputation for Interval Estimation From Simple Random Samples With Ignorable Nonresponse
Rubin, D. B., & Nathaniel Schenker, A. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478280 · Cited by 768 · see citing works
Ambulatory care groups: a categorization of diagnoses for research and management.
Starfield, B. H., Weiner, J. P., Mumford, L. M., & Steinwachs, D. M. (1991) · Europe PMC (PubMed Central) · Cited by 520 · see citing works
O'Connor, G. T., Plume, S. K., Olmstead, E. M., Coffin, L. H., Morton, J. R., Maloney, C. T., Nowicki, E. R., Levy, D. G., Tryzelaar, J. F., & Hernandez, F. (1992) · Circulation · doi:10.1161/01.cir.85.6.2110 · Cited by 406 · see citing works
Validation techniques for logistic regression models
Miller, M. E., Hui, S. L., & Tierney, W. M. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100805 · Cited by 322 · see citing works
Generic, Simple Risk Stratification Model for Heart Valve Surgery
Ambler, G., Omar, R. Z., Royston, P., Kinsman, R., Keogh, B. E., & Taylor, K. M. (2005) · Circulation · doi:10.1161/circulationaha.104.515049 · Cited by 279 · see citing works
Self-citationBarzi, F. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh175 · Cited by 263 · see citing works
(1992) · Psychiatric Bulletin · doi:10.1192/pb.16.5.303 · Cited by 248 · see citing works
The Society of Thoracic Surgeons National Cardiac Surgery Database: Current Risk Assessment
Edwards, F. H., Grover, F. L., Shroyer, A. L., Schwartz, M., & Bero, J. W. (1997) · The Annals of Thoracic Surgery · doi:10.1016/s0003-4975(97)00017-9 · Cited by 220 · see citing works
Burton, A., & Altman, D. G. (2004) · British Journal of Cancer · doi:10.1038/sj.bjc.6601907 · Cited by 219 · see citing works
Developing a prognostic model in the presence of missing data
Clark, T. G., & Altman, D. G. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(02)00539-5 · Cited by 214 · see citing works
Berrisford, R. G., Brunelli, A., Rocco, G., Treasure, T., Utley, M., & guide, O. B. O. T. A. A. (2005) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/j.ejcts.2005.03.047 · Cited by 174 · see citing works
Can one assess whether missing data are missing at random in medical studies?
Potthoff, R. F., Tudor, G. E., Pieper, K. S., & Hasselblad, V. (2006) · Statistical Methods in Medical Research · doi:10.1191/0962280206sm448oa · Cited by 141 · see citing works
The relative importance of prognostic factors in studies of survival
SCHEMPER, M. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122413 · Cited by 99 · see citing works
Pérez, A., Dennis, R., Gil, F., Rondón, M., & López, A. (2002) · Statistics in Medicine · doi:10.1002/sim.1391 · Cited by 74 · see citing works
Case-mix and variation in specialist referrals in general practice.
Sullivan, C. O., Omar, R., Ambler, G., & Majeed, A. (2005) · Europe PMC (PubMed Central) · Cited by 71 · see citing works
Predicting the risk of repetition after self harm: cohort study
Kapur, N., Cooper, J., Rodway, C., Kelly, J., Guthrie, E., & Mackway-Jones, K. C. (2005) · BMJ · doi:10.1136/bmj.38337.584225.82 · Cited by 69 · see citing works
Methods for estimating the AIDS incubation time distribution when date of seroconversion is censored
Geskus, R. B. (2001) · Statistics in Medicine · doi:10.1002/sim.700 · Cited by 62 · see citing works
Omar, R., Ambler, G., Royston, P. J., Eliahoo, J., & Taylor, K. M. (2004) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2003.10.032 · Cited by 58 · see citing works
A toolkit in SAS for the evaluation of multiple imputation methods
Brand, J., van Buuren, S., Groothuis‐Oudshoorn, C. G. M., & Gelsema, E. S. (2003) · Statistica Neerlandica · doi:10.1111/1467-9574.00219 · Cited by 38 · see citing works
Mander, A., & Clayton, D. G. (2000) · Stata technical bulletin · Cited by 18 · see citing works
Omar, R., Morton, L., Murad, S., & Taylor, K. M. (2003) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/s0022-5223(03)00362-3 · Cited by 2 · see citing works
Multiple imputation of missing blood pressure covariates in survival analysis
van Buuren, S., Boshuizen, H. C., & Knook, D. L. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990330)18:6
Flagged first, then by citations
Cited by
Sorted by citations
Jerez, J. M., Molina, I., García-Laencina, P. J., Alba, E., Ribelles, N., Martín, M., & Franco, L. (2010) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2010.05.002 · Cited by 608 · see citing works
The Effect of Multidisciplinary Care Teams on Intensive Care Unit Mortality
Kim, M. M., Barnato, A. E., Angus, D. C., Fleisher, L., & Kahn, J. M. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.521 · Cited by 531 · see citing works
Inflammatory Markers and Incident Heart Failure Risk in Older Adults
Kalogeropoulos, A., Georgiopoulou, V., Psaty, B. M., Rodondi, N., Smith, A. L., Harrison, D. G., Liu, Y., Hoffmann, U., Bauer, D. C., Newman, A. B., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.12.045 · Cited by 478 · see citing works
Imputation of missing data in life‐history trait datasets: which approach performs the best?
Penone, C., Davidson, A. D., Shoemaker, K. T., Marco, M. D., Rondinini, C., Brooks, T. M., Young, B. E., Graham, C. H., & Costa, G. C. (2014) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12232 · Cited by 452 · see citing works
Gerry, S., Bonnici, T., Birks, J., Kirtley, S., Virdee, P. S., Watkinson, P., & Collins, P. G. S. (2020) · BMJ · doi:10.1136/bmj.m1501 · Cited by 398 · see citing works
A predictive model for local recurrence after transanal endoscopic microsurgery for rectal cancer
Bach, S. P., Hill, J., Monson, J. R. T., Simson, J. N. L., Lane, L., Merrie, A. E. H., Warren, B. F., & Mortensen, N. J. M. (2009) · British journal of surgery · doi:10.1002/bjs.6456 · Cited by 371 · see citing works
Risk factors for relapse of antineutrophil cytoplasmic antibody–associated vasculitis
Walsh, M. W., Floßmann, O., Berden, A. E., Westman, K. W. A., Höglund, P., Stegeman, C. A., Jayne, D. R. W., & European Vasculitis Study Group (2011) · Arthritis & Rheumatism · doi:10.1002/art.33361 · Cited by 365 · see citing works
Zhang, X., Bullard, K. M., Cotch, M. F., Wilson, M. R., Rovner, B. W., McGwin, G., Owsley, C. J., Barker, L. E., Crews, J. E., & SAADDINE, J. B. (2013) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2013.2597 · Cited by 294 · see citing works
Hematologic Manifestations and Predictors of Lymphoma Development in Primary Sjögren Syndrome
Baimpa, E., Dahabreh, I. J., Voulgarelis, M., & Moutsopoulos, H. Μ. (2009) · Medicine · doi:10.1097/md.0b013e3181b76ab5 · Cited by 274 · see citing works
Mental Health Difficulties in Children With Developmental Coordination Disorder
Lingam, R., Jongmans, M. J., Ellis, M., Hunt, L. P., Golding, J., & Emond, A. M. (2012) · PEDIATRICS · doi:10.1542/peds.2011-1556 · Cited by 214 · see citing works
Frailty and risk for heart failure in older adults: The health, aging, and body composition study
Khan, H., Kalogeropoulos, A. P., Georgiopoulou, V. V., Newman, A. B., Harris, T. B., Rodondi, N., Bauer, D. C., Kritchevsky, S. B., & Butler, J. (2013) · American Heart Journal · doi:10.1016/j.ahj.2013.07.032 · Cited by 207 · see citing works
Personality, Socioeconomic Status, and All-Cause Mortality in the United States
Chapman, B. P., Fiscella, K., Kawachi, I., & Duberstein, P. R. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwp323 · Cited by 198 · see citing works
Missing Data and Multiple Imputation
Cummings, P. (2013) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2013.1329 · Cited by 189 · see citing works
Missing data methods for arbitrary missingness with small samples
McNeish, D. (2017) · Journal of Applied Statistics · doi:10.1080/02664763.2016.1158246 · Cited by 189 · see citing works
Glance, L. G., Wissler, R., Mukamel, D., Li, Y., Diachun, C., Salloum, R., Fleming, F., & Dick, A. (2010) · Anesthesiology · doi:10.1097/aln.0b013e3181eff32e · Cited by 172 · see citing works
Surgeon Specialty and Long-Term Survival After Pulmonary Resection for Lung Cancer
Farjah, F., Flum, D. R., Varghese, T. K., Symons, R. G., & Wood, D. E. (2009) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2008.12.030 · Cited by 175 · see citing works
Hardt, J., Herke, M., & Leonhart, R. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-184 · Cited by 169 · see citing works
Multiple imputation in a large-scale complex survey: a practical guide
He, Y., Zaslavsky, A. M., Landrum, M., Harrington, D. P., & Catalano, P. J. (2009) · Statistical Methods in Medical Research · doi:10.1177/0962280208101273 · Cited by 168 · see citing works
Population prevalence of edentulism and its association with depression and self-rated health
Tyrovolas, S., Koyanagi, A., Panagiotakos, D. B., Haro, J. M., Kassebaum, N. J., Chrepa, V., & Kotsakis, G. A. (2016) · Scientific Reports · doi:10.1038/srep37083 · Cited by 166 · see citing works
Peters, S. A. E., Bots, M. L., den Ruijter, H. M., Palmer, M. K., Grobbee, D. E., Crouse, J. R., O’Leary, D. H., Evans, G. W., Raichlen, J. S., Moons, K. G. M., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.012 · Cited by 165 · see citing works
Return to Work in Out-of-Hospital Cardiac Arrest Survivors
Kragholm, K. H., Wissenberg, M., Mortensen, R. N., Fonager, K., Jensen, S. E., Rajan, S., Lippert, F. K., Christensen, E. F., Hansen, P. A., Lang-Jensen, T., et al. (2015) · Circulation · doi:10.1161/circulationaha.114.011366 · Cited by 161 · see citing works
The Association Between Developmental Coordination Disorder and Other Developmental Traits
Lingam, R., Golding, J., Jongmans, M. J., Hunt, L. P., Ellis, M., & Emond, A. M. (2010) · PEDIATRICS · doi:10.1542/peds.2009-2789 · Cited by 162 · see citing works
Lemaignen, A., Birgand, G., Ghodhbane, W., Alkhoder, S., Lolom, I., Belorgey, S., Lescure, F.-X., Armand-Lefèvre, L., Raffoul, R., Dilly, M., et al. (2015) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2015.03.025 · Cited by 156 · see citing works
THE GOLDEN AGE : MASA EFEKTIF MERANCANG KUALITAS ANAK
Uce, L. (2017) · Bunayya Jurnal Pendidikan Anak · doi:10.22373/bunayya.v1i2.1322 · Cited by 156 · see citing works
Exploratory Factor Analysis With Small Samples and Missing Data
McNeish, D. (2016) · Journal of Personality Assessment · doi:10.1080/00223891.2016.1252382 · Cited by 152 · see citing works
Gittelsohn, J., Kim, E. M., He, S., & Pardilla, M. (2013) · Journal of Nutrition · doi:10.3945/jn.112.165266 · Cited by 122 · see citing works
Davies, R. R., Russo, M. J., Morgan, J. A., Sorabella, R. A., Naka, Y., & Chen, J. M. (2010) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2010.04.029 · Cited by 116 · see citing works
Survival after out-of-hospital cardiac arrest in relation to sex: A nationwide registry-based study
Wissenberg, M., Hansen, C. M., Folke, F., Lippert, F. K., Weeke, P. E., Karlsson, L., Rajan, S., Søndergaard, K. B., Kragholm, K. H., Christensen, E. F., et al. (2014) · Resuscitation · doi:10.1016/j.resuscitation.2014.06.008 · Cited by 115 · see citing works
MICE vs PPCA: Missing data imputation in healthcare
Hegde, H., Shimpi, N. A., Panny, A., Glurich, I., Christie, P., & Acharya, A. (2019) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2019.100275 · Cited by 100 · see citing works
Sekhon, R. S., Joyner, C. N., Ackerman, A. J., McMahan, C. S., Cook, D. D., & Robertson, D. J. (2020) · Field Crops Research · doi:10.1016/j.fcr.2020.107737 · Cited by 98 · see citing works
Key steps and common pitfalls in developing and validating risk models
Wynants, L., Collins, P. G. S., & CALSTER, B. V. (2016) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.14170 · Cited by 95 · see citing works
Nicotinic acetylcholine receptor variation and response to smoking cessation therapies
Bergen, A. W., Javitz, H. S., Krasnow, R. E., Nishita, D., Michel, M., Conti, D. V., Liu, J., Lee, W., Edlund, C. K., Hall, S. M., et al. (2012) · Pharmacogenetics and Genomics · doi:10.1097/fpc.0b013e32835cdabd · Cited by 90 · see citing works
Association of water temperature and submersion duration and drowning outcome
Quan, L., Mack, C. D., & Schiff, M. A. (2014) · Resuscitation · doi:10.1016/j.resuscitation.2014.02.024 · Cited by 89 · see citing works
Butler, J., Kalogeropoulos, A. P., Georgiopoulou, V. V., Bibbins-Domingo, K. B., Najjar, S. S., Sutton‐Tyrrell, K., Harris, T. B., Kritchevsky, S. B., Lloyd‐Jones, D. M., Newman, A. B., et al. (2011) · Heart · doi:10.1136/hrt.2011.225482 · Cited by 89 · see citing works
Risk factors for childhood enteric infection in urban Maputo, Mozambique: A cross-sectional study
Knee, J., Sumner, T., Adriano, Z., Berendes, D. M., de Bruijn, E., Schmidt, W., Nalá, R., Cumming, O., & Brown, J. M. (2018) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0006956 · Cited by 87 · see citing works
Serum Resistin Concentrations and Risk of New Onset Heart Failure in Older Persons
Butler, J., Kalogeropoulos, A. P., Georgiopoulou, V. V., de Rekeneire, N., Rodondi, N., Smith, A. L., Hoffmann, U., Kanaya, A. M., Newman, A. B., Kritchevsky, S. B., et al. (2009) · Arteriosclerosis Thrombosis and Vascular Biology · doi:10.1161/atvbaha.109.186783 · Cited by 87 · see citing works
Goyal, A., Ngufor, C. G., Kerezoudis, P., McCutcheon, B. A., Storlie, C. B., & Bydon, M. (2019) · Journal of Neurosurgery Spine · doi:10.3171/2019.3.spine181367 · Cited by 80 · see citing works
Understanding Patterns of Service Utilization Among Informal Caregivers of Community Older Adults
Hong, S.-I. (2009) · The Gerontologist · doi:10.1093/geront/gnp105 · Cited by 80 · see citing works
Wood, A. M., Royston, P., & White, I. R. (2015) · Biometrical Journal · doi:10.1002/bimj.201400004 · Cited by 78 · see citing works
Ali, A. M., Dawson, S.-J., Blows, F. M., Provenzano, E., Ellis, I. O., Baglietto, L., Huntsman, D. D., Caldas, C., & Pharoah, P. D. P. (2011) · British Journal of Cancer · doi:10.1038/sj.bjc.6606078 · Cited by 75 · see citing works
Knudsen, H. K., Studts, J. L., Boyd, S., & Roman, P. M. (2010) · Journal of Addictive Diseases · doi:10.1080/10550887.2010.489446 · Cited by 73 · see citing works
Impact of Statistical Approaches for Handling Missing Data on Trauma Center Quality
Glance, L. G., Osler, T., Mukamel, D., Meredith, W., & Dick, A. (2009) · Annals of Surgery · doi:10.1097/sla.0b013e31818e544b · Cited by 70 · see citing works
Vedder, M. M., Márquez, M., de Bekker-Grob, E. W., Calle, M. L., Dyrskjøt, L., Kogevinas, M., Segersten, U., Malmström, P., Ferran Algaba, B., Beukers, W., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096849 · Cited by 67 · see citing works
Khan, H., Greene, S. J., Fonarow, G. C., Kalogeropoulos, A. P., Ambrosy, A. P., Maggioni, A. P., Zannad, F. M., Konstam, M. A., Swedberg, K. B., Yancy, C. W., et al. (2015) · European Journal of Heart Failure · doi:10.1002/ejhf.282 · Cited by 66 · see citing works
Parekh, J. R., Bostrom, A., & Feng, S. (2010) · American Journal of Transplantation · doi:10.1111/j.1600-6143.2009.02936.x · Cited by 66 · see citing works
Clinical and Laboratory Predictors of Distinct Histopathogical Features of Lupus Nephritis
Mavragani, C. P., Fragoulis, G. E., Somarakis, G., Drosos, A. A., Tzioufas, A. G., & Moutsopoulos, H. Μ. (2015) · Medicine · doi:10.1097/md.0000000000000829 · Cited by 63 · see citing works
Health of Elderly Mexican American Adults and Family Caregiver Distress
Rote, S. M., Angel, J. L., & Markides, K. S. (2014) · Research on Aging · doi:10.1177/0164027514531028 · Cited by 60 · see citing works
MacLean, C. D., Gagnon, M., CALLAS, P. W., & Littenberg, B. (2009) · Journal of General Internal Medicine · doi:10.1007/s11606-009-1147-x · Cited by 58 · see citing works
Internal Migration and Life Satisfaction: Well-Being Paths of Young Adult Migrants
Switek, M. (2014) · Social Indicators Research · doi:10.1007/s11205-014-0829-x · Cited by 57 · see citing works
Stewart, Z. A., Locke, J. E., Montgomery, R. A., Singer, A. L., Cameron, A. M., & Segev, D. L. (2009) · Liver Transplantation · doi:10.1002/lt.21723 · Cited by 57 · see citing works
Religious Attendance and Biological Functioning
Hill, T. D., Rote, S. M., Ellison, C. G., & Burdette, A. M. (2014) · Journal of Aging and Health · doi:10.1177/0898264314529333 · Cited by 58 · see citing works
Carder, M., McNamee, R., Beverland, I. J., Elton, R. A., van Tongeren, M., Cohen, G. R., Boyd, J., MacNee, W., & Agius, R. M. (2007) · Occupational and Environmental Medicine · doi:10.1136/oem.2007.032896 · Cited by 56 · see citing works
A cautionary case study of approaches to the treatment of missing data
Paul, C., Mason, W., McCaffrey, D., & Fox, S. (2008) · Stat. Methods Appl. · doi:10.1007/s10260-007-0090-4 · Cited by 48 · see citing works
Asymptomatic Rotavirus Infections in England: Prevalence, Characteristics, and Risk Factors
Phillips, G., Lopman, B. A., Rodrigues, L. C., & Tam, C. C. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq050 · Cited by 53 · see citing works
Zonies, D. H., Mack, C. D., Kramer, B. R., Rivara, F. P., & Klein, M. B. (2010) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2009.11.008 · Cited by 52 · see citing works
Davies, R. R., Haldeman, S., McCulloch, M. A., & Pizarro, C. (2014) · The Journal of Heart and Lung Transplantation · doi:10.1016/j.healun.2014.02.010 · Cited by 52 · see citing works
Modeling major lung resection outcomes using classification trees and multiple imputation techniques
Ferguson, M. K., Siddique, J., & Karrison, T. G. (2008) · European Journal of Cardio-Thoracic Surgery · doi:10.1016/j.ejcts.2008.07.037 · Cited by 51 · see citing works
Risk-Adjusted Pathologic Margin Positivity Rate As a Quality Indicator in Rectal Cancer Surgery
Massarweh, N. N., Hu, C., You, Y. N., Bednarski, B. K., Rodrı́guez-Bigas, M. A., Skibber, J. M., Cantor, S. B., Cormier, J. N., Feig, B. W., & Chang, G. J. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2014.55.5334 · Cited by 49 · see citing works
Bethge, M., Radoschewski, F., & Müller-Fahrnow, W. (2009) · Disability and Rehabilitation · doi:10.1080/09638280902751949 · Cited by 54 · see citing works
Knudsen, H. K., & Studts, J. (2010) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2009.12.002 · Cited by 50 · see citing works
Transcending data gaps: a framework to reduce inferential errors in ecological analyses
Kim, S. W., Blomberg, S. P., & Pandolfi, J. M. (2018) · Ecology Letters · doi:10.1111/ele.13089 · Cited by 46 · see citing works
Low Somatosensory Cortex Excitability in the Acute Stage of Low Back Pain Causes Chronic Pain
Jenkins, L. C., Chang, W., Buscemi, V., Liston, M. B., Skippen, P., Cashin, A. G., McAuley, J. H., & Schabrun, S. M. (2021) · Journal of Pain · doi:10.1016/j.jpain.2021.08.003 · Cited by 44 · see citing works
Uso da imputação múltipla de dados faltantes: uma simulação utilizando dados epidemiológicos
Nunes, L. N., Klück, M. M., & Fachel, J. M. G. (2009) · Cadernos de Saúde Pública · doi:10.1590/s0102-311x2009000200005 · Cited by 44 · see citing works
Donor organ turn-downs and outcomes after listing for pediatric heart transplant
Davies, R. R., Bano, M., Butts, R. J., Jaquiss, R. D. B., & Kirk, R. (2018) · The Journal of Heart and Lung Transplantation · doi:10.1016/j.healun.2018.09.026 · Cited by 43 · see citing works
Fukuhara, S., Takeda, K., Polanco, A. R., Takayama, H., & Naka, Y. (2015) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2015.10.024 · Cited by 40 · see citing works
Prognostic Value of BNP Reduction During Hospitalization in Patients With Acute Heart Failure
Kagiyama, N., Kitai, T., Hayashida, A., Yamaguchi, T., Okumura, T., Kida, K., Mizuno, A., Oishi, S., Inuzuka, Y., Akiyama, E., et al. (2019) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2019.04.004 · Cited by 40 · see citing works
Multiple Imputation of Missing Data: A Simulation Study on a Binary Response
Hardt, J., Herke, M., Brian, T., & Laubach, W. (2013) · Open Journal of Statistics · doi:10.4236/ojs.2013.35043 · Cited by 48 · see citing works
Bertolini, R., Finch, S. J., & Nehm, R. H. (2021) · Journal of Science Education and Technology · doi:10.1007/s10956-020-09888-8 · Cited by 44 · see citing works
Screening Stress Myocardial Perfusion Imaging and Eligibility for Liver Transplantation
Bradley, S. M., Soine, L. A., Caldwell, J. H., & Goldberg, S. L. (2010) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2009.11.023 · Cited by 39 · see citing works
Biccler, J. L., Eloranta, S., Brown, P. D. N., Frederiksen, H., Jerkeman, M., Jørgensen, J. M., Jakobsen, L. H., Smedby, K. E. E., Bøgsted, M., & El‐Galaly, T. C. (2018) · JCO Clinical Cancer Informatics · doi:10.1200/cci.18.00025 · Cited by 38 · see citing works
Albert‐Daviaud, A., Buerki, S., Onjalalaina, G. E., Perillo, S., Rabarijaona, R. N., Razafindratsima, O. H., Sato, H., Valenta, K., Wright, P. C., & Stuppy, W. (2020) · Biological Conservation · doi:10.1016/j.biocon.2020.108438 · Cited by 37 · see citing works
Giorgi, R., Belot, A., Gaudart, J., & Launoy, G. D. (2008) · Statistics in Medicine · doi:10.1002/sim.3476 · Cited by 37 · see citing works
Hattori, M., Naruse, Y., Oginosawa, Y., Matsue, Y., Hanaki, Y., Kowase, S., Kurosaki, K., Mizukami, A., Kohno, R., Abe, H., et al. (2019) · Heart Rhythm · doi:10.1016/j.hrthm.2019.01.008 · Cited by 35 · see citing works
Rendleman, M. C., Buatti, J. M., Braun, T. A., Smith, B. J., Nwakama, C., Beichel, R., Brown, B., & Casavant, T. L. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-2929-8 · Cited by 34 · see citing works
The search for stable prognostic models in multiple imputed data sets
Vergouw, D., Heymans, M. W., Peat, G., Kuijpers, T., Croft, P. R., de Vet, H. C. W., van der Horst, H. E., & van der Windt, D. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-81 · Cited by 34 · see citing works
Sustainment of smoking cessation programs in substance use disorder treatment organizations.
Knudsen, H. K., Muilenburg, J., & Eby, L. (2013) · Nicotine & Tobacco Research · doi:10.1093/ntr/nts242 · Cited by 34 · see citing works
Prognostic factors in adults with knee pain in general practice
Belo, J. N., Berger, M. Y., Koes, B. W., & Bierma-Zeinstra, S. (2009) · Arthritis Care & Research · doi:10.1002/art.24419 · Cited by 32 · see citing works
Health Care Utilization Among Surgically Treated Medicare Beneficiaries With Lung Cancer
Farjah, F., Wood, D. E., Varghese, T. K., Massarweh, N. N., Symons, R. G., & Flum, D. R. (2009) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2009.08.006 · Cited by 31 · see citing works
Knudsen, H. K., & Abraham, A. J. (2012) · Psychiatric Services · doi:10.1176/appi.ps.201100034 · Cited by 30 · see citing works
Jones, R. M., van den Bree, M. B. M., Ferriter, M., & Taylor, P. J. (2009) · Behavioral Sciences & the Law · doi:10.1002/bsl.885 · Cited by 30 · see citing works
Karlsson, L., Wissenberg, M., Fosbøl, E. L., Hansen, C. M., Lippert, F. K., Bagai, A., McNally, B., Granger, C. B., Christensen, E. F., Folke, F., et al. (2014) · Resuscitation · doi:10.1016/j.resuscitation.2014.06.012 · Cited by 30 · see citing works
Barry, S., Tammemägi, M. C., Penek, S., Kassan, E. C., Dorfman, C. S., Riley, T. L., Commin, J., & Taylor, K. L. (2012) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs398 · Cited by 29 · see citing works
CORR Synthesis: When Should We Be Skeptical of Clinical Prediction Models?
Karhade, A. V., & Schwab, J. H. (2020) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001367 · Cited by 29 · see citing works
Perceived Accessibility of Cigarettes Among Youth
Doubeni, C. A., Li, W., Fouayzi, H., & DiFranza, J. R. (2009) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.11.006 · Cited by 29 · see citing works
Seymour, C. W., Cooke, C. R., Hebert, P. L., & Rea, T. D. (2011) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2011.07.021 · Cited by 29 · see citing works
Seed dispersal syndromes in the Madagascan flora: the unusual importance of primates
Albert‐Daviaud, A., Perillo, S., & Stuppy, W. (2018) · Oryx · doi:10.1017/s0030605317001600 · Cited by 29 · see citing works
Predicting Mortality of Ruptured Abdominal Aortic Aneurysms in the Era of Endovascular Repair
Healey, C., Neilson, M., Clark, D. E., Schanzer, A. S., & Robinson, W. P. (2016) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2016.09.006 · Cited by 28 · see citing works
Very Early Diuretic Response After Admission for Acute Heart Failure
Kuroda, S., Damman, K., ter Maaten, J. M., Voors, A. A., Okumura, T., Kida, K., Oishi, S., Akiyama, E., Suzuki, S., Yamamoto, M., et al. (2018) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2018.09.004 · Cited by 28 · see citing works
Manusov, E. G., Diego, V. P., Smith, J., Garza, J. R., Lowdermilk, J., Blangero, J., Williams‐Blangero, S., & Fernandez, F. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00215 · Cited by 28 · see citing works
Lau, S., Günther, M. P., Kling, S., & Kirchebner, J. (2019) · European Psychiatry · doi:10.1016/j.eurpsy.2019.05.005 · Cited by 27 · see citing works
Nagarajan, G., & Babu, L. D. D. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102214 · Cited by 26 · see citing works
Organic Cation Transporter Variation and Response to Smoking Cessation Therapies
Bergen, A. W., Javitz, H. S., Krasnow, R. E., Michel, M., Nishita, D., Conti, D. V., Edlund, C. K., Kwok, P., McClure, J. B., Kim, R. B., et al. (2014) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntu161 · Cited by 27 · see citing works
Charlton, C. E., Poon, M. T. C., Brennan, P. M., & Fleuriot, J. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107482 · Cited by 25 · see citing works
He, Y., & Raghunathan, T. E. (2009) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610910802677191 · Cited by 25 · see citing works
Davies, R. R., McCulloch, M. A., Haldeman, S., Gidding, S. S., & Pizarro, C. (2016) · The Journal of Heart and Lung Transplantation · doi:10.1016/j.healun.2016.09.007 · Cited by 25 · see citing works
Gade, G. V., Jørgensen, M. G., Ryg, J., Masud, T., Jakobsen, L. H., & Andersen, S. (2021) · BMC Geriatrics · doi:10.1186/s12877-021-02346-z · Cited by 24 · see citing works
Davies, R. R., Haldeman, S., McCulloch, M. A., Gidding, S. S., & Pizarro, C. (2015) · The Journal of Heart and Lung Transplantation · doi:10.1016/j.healun.2015.05.023 · Cited by 24 · see citing works
Lavanya, K., V, Reddy, L. S. S., & Reddy, B. E. (2019) · International Journal of Ambient Computing and Intelligence · doi:10.4018/ijaci.2019040105 · Cited by 23 · see citing works
The impact of missing trauma data on predicting massive transfusion
Trickey, A. W., Fox, E. E., del Junco, D. J., Ning, J., Holcomb, J. B., Brasel, K. J., Cohen, M. J. M., Schreiber, M. A., Bulger, E. M., Phelan, H. A., et al. (2013) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/ta.0b013e3182914530 · Cited by 23 · see citing works
The Quality Metric Prolonged Length of Stay Misses Clinically Important Adverse Events
Farjah, F., Lou, F., Rusch, V. W., & Rizk, N. P. (2012) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2012.04.082 · Cited by 23 · see citing works
- Authors
- Ambler, G., Omar, R. Z., & Royston, P.
- Affiliations
- UCL Biomedical Research Centre (GB)
- MRC Clinical Trials Unit at UCL (GB)
- Corresponding author
- Ambler, Gareth
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2007-06
- Volume
- 16
- Issue
- 3
- Pages
- 277-298
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W1965973862
- Cited by
- 233
- References
- 39
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 9 · 2023: 10 · 2022: 6 · 2021: 15 · 2020: 12 · 2019: 13 · 2018: 11 · 2017: 8 · 2016: 11 · 2015: 14 · 2014: 14 · 2013: 12 · 2012: 19
- FWCI
- 15.72
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- missing data imputation, risk prediction model, binary outcomes, multiple imputation, multiple imputation by chained equations, single imputation, complete case analysis, cardiac surgery, mortality prediction, forecast reliability, estimation bias
- MeSH terms
- Aged, Female, United Kingdom, Humans, Male, Middle Aged, Thoracic Surgery, Models, Statistical, Bias, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
