Cited by 892Statistical Methods and Bayesian InferenceStatistical Methods in EpidemiologyAdvanced Causal Inference Techniques

The DOI matches its Crossref record.

A critical look at methods for handling missing covariates in epidemiologic regression analyses.

Greenland, Sander, Finkle, William D.·1995·American Journal of Epidemiology

DOI 10.1093/oxfordjournals.aje.a117592

Open this paper in CoChat — free account

Abstract

Epidemiologic studies often encounter missing covariate values. While simple methods such as stratification on missing-data status, conditional-mean imputation, and complete-subject analysis are commonly employed for handling this problem, several studies have shown that these methods can be biased under reasonable circumstances. The authors review these results in the context of logistic regression and present simulation experiments showing the limitations of the methods. The method based on missing-data indicators can exhibit severe bias even when the data are missing completely at random, and regression (conditional-mean) imputation can be inordinately sensitive to model misspecification. Even complete-subject analysis can outperform these methods. More sophisticated methods, such as maximum likelihood, multiple imputation, and weighted estimating equations, have been given extensive attention in the statistics literature. While these methods are superior to simple methods, they are not commonly used in epidemiology, no doubt due to their complexity and the lack of packaged software to apply these methods. The authors contrast the results of multiple imputation to simple methods in the analysis of a case-control study of endometrial cancer, and they find a meaningful difference in results for age at menarche. In general, the authors recommend that epidemiologists avoid using the missing-indicator method and use more sophisticated methods whenever a large proportion of data are missing.

Integrity signals

Registry

Crossref · 10.1093/oxfordjournals.aje.a117592

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Sorted by citations

Items 301–400 of 892

Longitudinal Use of Complementary and Alternative Medicine Among Older Adults With Radiographic Knee Osteoarthritis
Yang, S., Dube, C. E., Eaton, C. B., McAlindon, T. E., & Lapane, K. L. (2013) · Clinical Therapeutics · doi:10.1016/j.clinthera.2013.09.022 · Cited by 48
Imputing Missing Genotypes with WeightedkNearest Neighbors
Schwender, H. (2012) · Journal of Toxicology and Environmental Health · doi:10.1080/15287394.2012.674910 · Cited by 48
“Moreover, this approach might bias the results of further analysis (Greenland and Finkle 1995).”
Result
Glucose and haemoglobin in the assessment of prognosis after first hospitalisation for heart failure
Newton, J. D., & Squire, I. B. (2006) · Heart · doi:10.1136/hrt.2005.080895 · Cited by 47
“To minimise loss of power and introduction of bias by excluding patients with incomplete data,”
Background
Diagnostic Accuracy and User-Friendliness of 5 Point-of-Care D-Dimer Tests for the Exclusion of Deep Vein Thrombosis
Geersing, G., Toll, D. B., Janssen, K. J. M., Oudega, R., Blikman, M. J., Wijland, R., de Vooght, K. M. K., Hoes, A. W., & Moons, K. G. M. (2010) · Clinical Chemistry · doi:10.1373/clinchem.2010.147892 · Cited by 48
Differences in problem behaviour among ethnic minority and majority preschoolers in the Netherlands and the role of family functioning and parenting factors as mediators: the Generation R Study
Flink, I. J. E., Jansen, P. W., Beirens, T. M. J., Tiemeier, H. W., van IJzendoorn, M. H., Jaddoe, V. W. V., Albert Hofman, D., & Raat, H. (2012) · BMC Public Health · doi:10.1186/1471-2458-12-1092 · Cited by 47
“confounders and potential mediators), multiple imputation was applied [29].”
Method
Predictors of development and progression of microvascular complications in a cohort of Brazilian type 2 diabetic patients
Cardoso, C. R. L., & Salles, G. F. (2008) · Journal of Diabetes and its Complications · doi:10.1016/j.jdiacomp.2007.02.004 · Cited by 49
“It has been shown that deleting subjects with a missing value on one of the predictor variables included in the multivariate models (the so-called complete cases analysis) commonly leads to biased results and, surely, to loss of power (Greenland & Finkle, 1995).”
Background
Predicting missing biomarker data in a longitudinal study of Alzheimer disease
Lo, R. Y., Jagust, W. J., Aisen, P. S. S., Jack, C. R., Toga, A. W., Beckett, L. A., Gamst, A. C., Soares, H. D., Green, R. C., Montine, T., et al. (2012) · Neurology · doi:10.1212/wnl.0b013e318253d5b3 · Cited by 46
Selection of embryos for transfer in IVF: ranking embryos based on their implantation potential using morphological scoring
van Loendersloot, L. L., van Wely, M., van der Veen, F., et al. (2014) · Reproductive BioMedicine Online · doi:10.1016/j.rbmo.2014.04.016 · Cited by 46
“Simple exclusion of couples with missing values on one or more variables commonly causes biased results and decreases statistical efficiency (Greenland and Finkle, 1995).”
Background
Identifying patients with cellulitis who are likely to require inpatient admission after a stay in an ED observation unit
Volz, K. A., Canham, L. S., Kaplan, E., et al. (2012) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2012.09.005 · Cited by 46
Immunohistochemical analysis of the expression of cdk4 and p16INK4 in human endometrioid-type endometrial carcinoma
Shiozawa, T., Nikaido, T., 清水, 元., Zhai, Y., & Fujii, S. (1997) · Cancer · doi:10.1002/(sici)1097-0142(19971215)80:12 · Cited by 54
Father’s Education: An Independent Marker of Risk for Preterm Birth
Blumenshine, P., Egerter, S. A., Libet, M., & Braveman, P. (2010) · Maternal and Child Health Journal · doi:10.1007/s10995-009-0559-x · Cited by 48
“While previous studies have also been limited by their handling of missing data, the present analysis uses multiple imputation, a technique that reduces the biases introduced by other simpler methods for handling missing data, such as complete case analysis, overall mean imputation, and the indicator method [19, 32, 3…”
Method
Gender-specific nonrandom association between the α1-antichymotrypsin and apolipoprotein E polymorphisms in the general population and its implication for the risk of Alzheimer's disease
Kamboh, M. I., Sanghera, D. K., Aston, C. E., Bunker, C. H., Hamman, R. F., Ferrell, R. E., & DeKosky, S. T. (1997) · Genetic Epidemiology · doi:10.1002/(sici)1098-2272(1997)14:2 · Cited by 16
Patients evaluate accessibility and nurse telephone consultations in out-of-hours GP care: Determinants of a negative evaluation
Giesen, P. H. J., van Charante, E. P. M., Mokkink, H. G. A., Bindels, P., van den Bosch, W., & Grol, R. P. T. M. (2006) · Patient Education and Counseling · doi:10.1016/j.pec.2006.06.021 · Cited by 46
Gunshot Injuries in Children Served by Emergency Services
Newgard, C. D., Kuppermann, N., Holmes, J. F., Haukoos, J. S., Wetzel, B., Hsia, R. Y.-J., Wang, N. E., Bulger, E. M., Staudenmayer, K., Mann, N. C., et al. (2013) · PEDIATRICS · doi:10.1542/peds.2013-1350 · Cited by 46
A randomised clinical trial on cardiotocography plus fetal blood sampling versus cardiotocography plus ST-analysis of the fetal electrocardiogram (STAN®) for intrapartum monitoring
Westerhuis, M. E. M. H., Moons, K. G. M., van Beek, E., et al. (2007) · BMC Pregnancy and Childbirth · doi:10.1186/1471-2393-7-13 · Cited by 46
“This will be done using single and multiple imputation methods [28-30].”
The incidence of second primary invasive melanoma in Queensland, 1982–2003
McCaul, K. A., Fritschi, L., Baade, P. D., & Coory, M. D. (2007) · Cancer Causes & Control · doi:10.1007/s10552-007-9106-5 · Cited by 45
“Since this is a category where the morphology is missing, this approach can potentially bias regression models [17].”
Primary risk factors influence risk of recurrent myocardial infarction/death from coronary heart disease: results from the Stockholm Heart Epidemiology Program (SHEEP)
Leander, K., Wiman, B., Hallqvist, J., Andersson, T., Ahlbom, A., & Fairé, U. D. (2007) · European Journal of Cardiovascular Prevention & Rehabilitation · doi:10.1097/hjr.0b013e328012e3cc · Cited by 45
An Assessment of Bone Fluoride and Osteosarcoma
Kim, F. M., Hayes, C., Williams, P. L., Whitford, G. M., JOSHIPURA, K. J., Hoover, R. N., Douglass, C. W., the National Osteosarcoma Etiology Group, Gebhardt, M. C., Scarborough, M. T., et al. (2011) · Journal of Dental Research · doi:10.1177/0022034511418828 · Cited by 46
Artificial Intelligence–Based Multimodal Risk Assessment Model for Surgical Site Infection (AMRAMS): Development and Validation Study
Chen, W., Lü, Z., You, L., Zhou, L., Xu, J., & Chen, K. (2020) · JMIR Medical Informatics · doi:10.2196/18186 · Cited by 46
“For missing data, we generated missing-value indicators [25] (ie, binary dummy features indicating whether the values of the original features were missing) and conducted mean imputation for continuous features and mode imputation for binary or categorical features.”
Breast Cancer Risk is not Associated with Polymorphic Forms of Xeroderma Pigmentosum Genes in a Cohort of Women from Washington County, Maryland
Jorgensen, T. J., Visvanathan, K., Ruczinski, I., Thuita, L. W., Hoffman, S. C., & Helzlsouer, K. J. (2006) · Breast Cancer Research and Treatment · doi:10.1007/s10549-006-9263-3 · Cited by 45
“and, therefore, can support the validity of a finding where some observations have missing data [11].”
Background
Influence of commercial information on prescription quantity in primary care
Caamaño‐Isorna, F. (2002) · European Journal of Public Health · doi:10.1093/eurpub/12.3.187 · Cited by 45
Early antiretroviral therapy in HIV-1-infected infants, 1996–2008
The European Pregnancy and Paediatric HIV Cohort Collaboration (EPPICC) study group in EuroCoord (2011) · AIDS · doi:10.1097/qad.0b013e32834d614c · Cited by 44
Associations of birth weight, linear growth and relative weight gain throughout life with abdominal fat depots in adulthood: the 1982 Pelotas (Brazil) birth cohort study
de França, G. V. A., Rolfe, E. D. L., Horta, B. L., et al. (2015) · International Journal of Obesity · doi:10.1038/ijo.2015.192 · Cited by 44
Prenatal exposure to maternal and paternal depressive symptoms and white matter microstructure in children
El Marroun, H., Zou, R., Muetzel, R. L., Jaddoe, V. W., Verhulst, F. C., White, T., & Tiemeier, H. (2018) · Depression and Anxiety · doi:10.1002/da.22722 · Cited by 45
Common Genetic Variation, Residential Proximity to Traffic Exposure, and Left Ventricular Mass: The Multi-Ethnic Study of Atherosclerosis
Hee, V. C. V., Adar, S. D., Szpiro, A. A., Barr, R. G., Roux, A. V. D., Bluemke, D. A., Sheppard, L. A., Gill, E. A., Bahrami, H., Wassel, C. L., et al. (2010) · Environmental Health Perspectives · doi:10.1289/ehp.0901535 · Cited by 43
Prevalence, Characteristics, and Risk Factors of Moderate or High Hyperopia among Multiethnic Children 6 to 72 Months of Age
Jiang, X., Tarczy‐Hornoch, K., Stram, D. A., Katz, J., Friedman, D. S., Tielsch, J. M., Matsumura, S., Saw, S., Mitchell, P., Rose, K. A., et al. (2019) · Ophthalmology · doi:10.1016/j.ophtha.2019.02.021 · Cited by 43
To Adjust or not to Adjust? Estimating the Average Treatment Effect in Randomized Experiments with Missing Covariates
Zhao, A., & Ding, P. (2022) · Journal of the American Statistical Association · doi:10.1080/01621459.2022.2123814 · Cited by 45
“Specifically, Greenland and Finkle (1995) evaluated this method in the context of logistic regression for observational studies and reported severe bias even when the data are missing completely at random; Donders et al. (2006) offered a similar discussion.”
Method
Integrated Supply Chain Model for Sustainable Manufacturing: A System Dynamics Approach
Shamsuddoha, M. (2015) · Emerald Group Publishing Limited eBooks · doi:10.1108/s1069-09642015000022b003 · Cited by 52
Estimating treatment effects with partially observed covariates using outcome regression with missing indicators
Blake, H. A., Leyrat, C., Mansfield, K. E., Tomlinson, L. A., Carpenter, J. R., & Williamson, E. (2020) · Biometrical Journal · doi:10.1002/bimj.201900041 · Cited by 39
“The use of missing indicators to handle missing covariates in outcome regression has been criticized in the literature for being “ad hoc” (Greenland & Finkle, 1995; Vach & Blettner, 1991), and for giving biased results (Groenwold et al., 2012; Jones, 1996).”
MethodInfluential
Prediction rule for cardiovascular events and mortality in peripheral arterial disease patients: Data from the prospective Second Manifestations of ARTerial disease (SMART) cohort study
Sprengers, R. W., Janssen, K. J. M., Moll, F. L., Verhaar, M. C., & van der Graaf, Y. (2009) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2009.07.095 · Cited by 43
Comparing the Diagnostic Performance of 2 Clinical Decision Rules to Rule Out Deep Vein Thrombosis in Primary Care Patients
van der Velde, E. F., Toll, D. B., ten Cate‐Hoek, A. J., et al. (2011) · The Annals of Family Medicine · doi:10.1370/afm.1198 · Cited by 42
Factors associated with intersucking in Swiss dairy heifers
Keil, N. M., Audigé, L., & Langhans, W. (2000) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(00)00127-6 · Cited by 42
“Although the replacement procedure can lead to an underestimation of the standard error of the final regression coefficients (Greenland and Finkle, 1995), we considered it more appropriate to impute missing values than to exclude the corresponding farms.”
Background
Database Comparison of the Adult‐to‐Adult Living Donor Liver Transplantation Cohort Study (A2ALL) and the SRTR U.S. Transplant Registry ,
Gillespie, B. W., Merion, R. M., Ortiz-Rios, E., et al. (2010) · American Journal of Transplantation · doi:10.1111/j.1600-6143.2010.03039.x · Cited by 41
Comparison of Methods of Handling Missing Data in Individual Patient Data Meta-analyses: An Empirical Example on Antibiotics in Children with Acute Otitis Media
Koopman, L., van der Heijden, G. J. M. G., Grobbee, D. E., & Rovers, M. M. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwm341 · Cited by 41
“Conventional complete case analyses, that is, removing subjects with a missing value from the analyses, may not only reduce precision because only part of the data is used (7–9) but also produce biased results because the results may improve when missing data are imputed (7, 10).”
BackgroundMethod
Infant feeding and child fussy eating: The Generation R Study
de Barse, L. M., Jansen, P. W., Edelson, L. R., Jaddoe, V. W. V., Franco, O. H., Tiemeier, H. W., & Graaff, J. S. (2017) · Appetite · doi:10.1016/j.appet.2017.04.006 · Cited by 41
Lost working days, productivity, and restraint use among occupants of motor vehicles that crashed in the United States
Ebel, B. E., Mack, C. D., Diehr, P., & Rivara, F. P. (2004) · Injury Prevention · doi:10.1136/ip.2004.005850 · Cited by 41
Does ovarian hyperstimulation in intrauterine insemination for cervical factor subfertility improve pregnancy rates?
Steures, P., van der Steeg, J. W., Verhoeve, H. R., et al. (2004) · Human Reproduction · doi:10.1093/humrep/deh435 · Cited by 41
“Since this item was missing in all cycles regardless of the result, pregnancy or not, data were missing at random and therefore not subject to bias (Greenland and Finkle, 1995).”
Background
Deep vein thrombosis
Yablon, S. A., Rock, W., Nick, T. G., et al. (2004) · Neurology · doi:10.1212/01.wnl.0000133009.24727.9f · Cited by 41
Digoxin use and risk of invasive breast cancer: evidence from the Nurses’ Health Study and meta-analysis
Ahern, T. P., Tamimi, R. M., ROSNER, B. A., & Hankinson, S. E. (2014) · Breast Cancer Research and Treatment · doi:10.1007/s10549-014-2886-x · Cited by 40
“The missing indicator method can lead to substantial bias if the variable in question is strongly related to both the exposure and outcome [51].”
Gene-covariate interaction between dysplastic nevi and the CDKN2A gene in American melanoma-prone families.
Goldstein, A. M., Martinez, M. M., Tucker, M. A., & Démenais, F. (2000) · PubMed · Cited by 40
Effects of Glucosamine and Chondroitin Supplementation on Knee Osteoarthritis: An Analysis With Marginal Structural Models
Yang, S., Eaton, C. B., McAlindon, T. E., & Lapane, K. L. (2014) · Arthritis & Rheumatology · doi:10.1002/art.38932 · Cited by 39
“For participants with missing information on the time-varying variables, we imputed missing values with the last observation carried forward method (29).”
Method
A multilevel analysis of the association between social networks and support on leisure time physical activity: Evidence from 40 disadvantaged areas in London
Yu, G., Renton, A. M., Schmidt, E., Tobi, P., Bertotti, M., Watts, P. N., & Lais, S. (2011) · Health & Place · doi:10.1016/j.healthplace.2011.07.002 · Cited by 38
“Based on simulation studies and theoretical reasoning, it is widely advocated that imputation of missing data is better than ignoring missing data (Greenland and Finkle, 1995; Little, 1992; Rubin and Schenker, 1991; Toutenburg, 1990).”
Background
Sedentary behaviors, physical activity behaviors, and body fat in 6-year-old children: the Generation R Study
Wijtzes, A. I., Bouthoorn, S. H., Jansen, W., Franco, O. H., Albert Hofman, D., Jaddoe, V. W. V., & Raat, H. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0096-x · Cited by 44
“To handle missing data, multiple imputation was applied [34].”
Method
Viral suppression and viral rebound among young adults living with HIV in Canada
Palmer, A. K., Gabler, K., Rachlis, B., Ding, E., Chia, J., Bacani, N., Bayoumi, A. M., Closson, K., Klein, M. B., Cooper, C., et al. (2018) · Medicine · doi:10.1097/md.0000000000010562 · Cited by 39
Multiple imputation with missing indicators as proxies for unmeasured variables: simulation study
Sperrin, M., & Martin, G. P. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01068-x · Cited by 41
Orthogeriatric Trauma Unit Improves Patient Outcomes in Geriatric Hip Fracture Patients
Schuijt, H. J., Kusen, J. Q., van Hernen, J. J., et al. (2020) · Geriatric Orthopaedic Surgery & Rehabilitation · doi:10.1177/2151459320949476 · Cited by 39
“001* 19% Time to surgery in hours; median (IQR) 53 (7) 20 (15-25) 21 (16-25) 0.”
Lifestyle variables and the risk of myocardial infarction in the General Practice Research Database
Delaney, J. A., Daskalopoulou, S. S., Brophy, J. M., Steele, R. J., Opatrny, L., & Suissa, S. (2007) · BMC Cardiovascular Disorders · doi:10.1186/1471-2261-7-38 · Cited by 40
“Furthermore, we are not able to verify the assumption that the missing data were ignorable (an assumption of multiple imputation in that the missing data can be completely predicted from the observed data) [4-6,12].”
BackgroundMethodInfluential
Quantifying the potential role of unmeasured confounders: the example of influenza vaccination
Groenwold, R. H. H., Hoes, A. W., Nichol, K. L., & Hak, E. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn173 · Cited by 39
“Vaccination 2006 53% (194) 41% (54) 0.019 Vaccination 2005 64% (232) 52% (68) 0.022 Cardiovascular co-morbidity 53% (193) 46% (60) 0.165 Pulmonary co-morbidity 9% (33) 8% (10) 0.623 Diabetes mellitus 11% (39) 10% (13) 0.807 Oncologic co-morbidity 4% (13) 3% (4) 0.784 Cardiovascular drug use 50% (184) 41% (54) 0.071 Pu…”
Influential
IS MALARIAL PLACENTAL INFECTION RELATED TO PERIPHERAL INFECTION AT ANY TIME OF PREGNANCY?
Cottrell, G., Mary, J., Barro, D., & Cot, M. (2005) · American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.2005.73.1112 · Cited by 39
Early prediction of neurological sequelae or death after bacterial meningitis
Oostenbrink, R., Moons, K. G. M., Derksen‐Lubsen, G., Grobbee, D. E., & Moll, H. (2002) · Acta Paediatrica · doi:10.1080/080352502317371616 · Cited by 44
“To reduce bias and increase statistical ef ciency multiple imputation was used for missing values in the data (26, 27).”
Method
Quality and performance of validated prognostic models for survival after resection of intrahepatic cholangiocarcinoma: a systematic review and meta-analysis
Buettner, S., Galjart, B., Beumer, B. R., van Vugt, J. L. A., van Eijck, C. H. J., Polak, W. G., de Jonge, J., Homs, M. Y. V., van Driel, L. M. J. W., Pawlik, T. M., et al. (2020) · HPB · doi:10.1016/j.hpb.2020.07.007 · Cited by 39
Symptoms and Suffering at End of Life for Children With Complex Chronic Conditions
Marcus, K. L., Kao, P., Ma, C., Wolfe, J., & DeCourcey, D. D. (2021) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2021.07.010 · Cited by 39
Information for decision making from imperfect national data: tracking major changes in health care use in Kenya using geostatistics
Gething, P. W., Noor, A. M., Goodman, C. A., Gikandi, P. W., Hay, S. I., Sharif, S. K., Atkinson, P. M., & Snow, R. W. (2007) · BMC Medicine · doi:10.1186/1741-7015-5-37 · Cited by 38
“Whilst multiple imputation approaches have been extended to address datasets constructed over time and/or across space [32-35], they remain reliant on the availability of covariates to predict missing values.”
Multiple Informants: A New Method to Assess Breast Cancer Patients' Comorbidity
Lash, T. L. (2003) · American Journal of Epidemiology · doi:10.1093/aje/kwf193 · Cited by 38
Distance Impacts Mortality in Trauma Patients with an Intubation Attempt
Cudnik, M. T., Newgard, C. D., Wang, H. E., Bangs, C. A., & Herrington, R. R. (2008) · Prehospital Emergency Care · doi:10.1080/10903120802290745 · Cited by 38
Children With Meningeal Signs
Oostenbrink, R., Moons, K. G. M., Twijnstra, M. J., Grobbee, D. E., & Moll, H. A. (2002) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/archpedi.156.12.1189 · Cited by 38
“This allowed all patients to be included in the analysis (increased statistical efficiency) and reduced bias since missing data may not occur at random (eg, a record of a seriously sick patient may be more complete than that of a less sick patient), such that the complete cases would reflect a selected sample of child…”
Background
Social Inequalities in Young Children’s Meal Skipping Behaviors: The Generation R Study
Wijtzes, A. I., Jansen, W., Jaddoe, V. W. V., Franco, O. H., Albert Hofman, D., van Lenthe, F. J., & Raat, H. (2015) · PLoS ONE · doi:10.1371/journal.pone.0134487 · Cited by 37
“A multiple imputation procedure was applied to handle missing data in the family SEP variables [29].”
Method
Prenatal exposure to selective serotonin reuptake inhibitors and non-verbal cognitive functioning in childhood
Marroun, H. E., White, T. J. H., Fernández, G., Jaddoe, V. W. V., Verhulst, F. C., Stricker, B. H. C., & Tiemeier, H. W. (2016) · Journal of Psychopharmacology · doi:10.1177/0269881116665335 · Cited by 39
“Imputations were based on the relations between all variables in the study (Greenland and Finkle, 1995).”
Method
An observational study on socio-economic and ethnic differences in indicators of sedentary behavior and physical activity in preschool children
van Rossem, L., Vogel, I., Moll, H. A., Jaddoe, V. W. V., Albert Hofman, D., Mackenbach, J. P., & Raat, H. (2011) · Preventive Medicine · doi:10.1016/j.ypmed.2011.10.016 · Cited by 36
Supplementing claims data with outpatient laboratory test results to improve confounding adjustment in effectiveness studies of lipid-lowering treatments
Schneeweiß, S., Rassen, J. A., Glynn, R. J., et al. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-180 · Cited by 36
“Even if all the missingness were due to the third case, in which data are most plausibly missing completely at random, the use of a missing indicator term could still cause bias [4,5].”
Background
Missing Outcome Data in Epidemiologic Studies
Cole, S. R., Zivich, P. N., Edwards, J. K., Ross, R. K., Shook‐Sa, B. E., Price, J. T., & Stringer, J. S. A. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac179 · Cited by 37
Prediction of serious complications in patients admitted to a surgical ward
Veltkamp, S. H. C., Kemmeren, J. M., van der Graaf, Y., et al. (2002) · British journal of surgery · doi:10.1046/j.0007-1323.2001.01963.x · Cited by 36
Maternal anthropometric risk factors for caesarean delivery before or after onset of labour
Sherrard, A., Platt, R. W., Vallerand, D., Usher, R. H., Zhang, X., & Ms, K. (2007) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2007.01275.x · Cited by 36
Increased risk of exceeding entertainment-media guidelines in preschool children from low socioeconomic background: The Generation R Study
Wijtzes, A. I., Jansen, W., Kamphuis, C. B. M., Jaddoe, V. W. V., Moll, H. A., Tiemeier, H. W., Verhulst, F. C., Albert Hofman, D., Mackenbach, J. P., & Raat, H. (2012) · Preventive Medicine · doi:10.1016/j.ypmed.2012.07.023 · Cited by 36
The Danish Youth Cohort: Characteristics of participants and non-participants and determinants of attrition
Vinther‐Larsen, M., Riegels, M., Rod, M. H., Schiøtz, M. L., Curtis, T., & Grønbæk, M. (2010) · Scandinavian Journal of Public Health · doi:10.1177/1403494810374222 · Cited by 36
“Methods to account for low participation rates in cohort studies have been introduced [26].”
Background
Missing Data in a Long Food Frequency Questionnaire
Fraser, G. E., Yan, R., Butler, T. L., et al. (2009) · Epidemiology · doi:10.1097/ede.0b013e31819642c4 · Cited by 36
“Other approaches are to impute the means of nonmissing values,(3) or to use the indicator variable method,(4,5) which is unsatisfactory in multivariate analyses.(6) Multiple imputation is a more attractive solution,(7) particularly if steps are taken to ensure that data are approximately missing at random.”
Method
Antibiotic Therapy to Prevent the Development of Asymptomatic Middle Ear Effusion in Children With Acute Otitis Media
Koopman, L., Hoes, A. W., Glasziou, P., Appelman, C. L., Burke, P., McCormick, D. P., Damoiseaux, R. A. M. J., Saux, N. L., & Rovers, M. M. (2008) · Archives of Otolaryngology - Head and Neck Surgery · doi:10.1001/archoto.2007.3 · Cited by 36
Survival in Danish patients with breast cancer and inflammatory bowel disease: A nationwide cohort study
Søgaard, K. K., Cronin-Fenton, D. P., Pedersen, L. H., Sørensen, H. T., & Lash, T. L. (2007) · Inflammatory Bowel Diseases · doi:10.1002/ibd.20341 · Cited by 36
Matching Methods for Causal Inference
King, G. (2011) · Cited by 35
Feeding styles, parenting styles and snacking behaviour in children attending primary schools in multiethnic neighbourhoods: a cross-sectional study
Wang, L., van de Gaar, V. M., Jansen, W., Mieloo, C. L., van Grieken, A., & Raat, H. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015495 · Cited by 40
“Assuming a random missing pattern of data, complete subject analyses were chosen to handle the missing values.(45) All analyses were conducted using the statistical software SAS V.”
Method
Alcohol involvement among young female drivers in US fatal crashes: unfavourable trends
Tsai, V. W., Anderson, C. L., & Vaca, F. E. (2010) · Injury Prevention · doi:10.1136/ip.2009.022301 · Cited by 35
Psychotic‐like experiences in pre‐adolescence: what precedes the antecedent symptoms of severe mental illness?
Bolhuis, K., Koopman‐Verhoeff, M. E., Blanken, L. M. E., Cibrev, D., Jaddoe, V. W. V., Verhulst, F. C., Hillegers, M. H. J., Kushner, S. A., & Tiemeier, H. W. (2018) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12891 · Cited by 37
“25 (41, 42) using 100 imputed data sets, which were pooled for analyses.”
Method
Injuries and death of children in rollover motor vehicle crashes in the United States
Rivara, F. P., Cummings, P., & Mock, C. (2003) · Injury Prevention · doi:10.1136/ip.9.1.76 · Cited by 35
Consumption of vegetables and fruits and risk of subtypes of head–neck cancer in the N etherlands C ohort S tudy
Maasland, D. H. E., van den Brandt, P. A., Kremer, B., Goldbohm, R. A., & Schouten, L. J. (2014) · International Journal of Cancer · doi:10.1002/ijc.29219 · Cited by 33
“This statistical approach resulted in only a minor loss of power when compared with the use of multiple imputation, but might have led to bias as a result of removing data with missing values.(38) Our study lacks information on HPV infection, which has been associated with HNC risk.”
Result
Occupational dust exposure and head and neck squamous cell carcinoma risk in a population‐based case–control study conducted in the greater B oston area
Langevin, S. M., McClean, M. D., Michaud, D. S., Eliot, M., Nelson, H. H., & Kelsey, K. T. (2013) · Cancer Medicine · doi:10.1002/cam4.155 · Cited by 34
“To compensate for the missing values in the logistic regression models, multiple imputation was employed using multivariate normal regression, based on age, sex, and smoking data; multiple imputation results in less biased findings when dealing with missing covariate data [13].”
Method
Vitamin or mineral supplement intake and the risk of head and neck cancer: pooled analysis in the INHANCE consortium
Li, Q., Chuang, S., Eluf‐Neto, J., Menezes, A. M. B., Matos, E., Koifman, S., Wünsch‐Filho, V., Fernández, L., Daudt, A. W., Curado, M. P., et al. (2011) · International Journal of Cancer · doi:10.1002/ijc.27405 · Cited by 34
“, whether or not education was missing did not depend on any other unobserved or missing values.(37) We used the logistic regression model(38) to predict education level using age, sex, race/ethnicity, study center and case/control status as the covariates within each of the geographic region.”
Patients Enrolled in HIV Care in Mozambique
Lahuerta, M., de Lima, J. N., Elul, B., et al. (2011) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0b013e31822ac0a9 · Cited by 34
Metabolic syndrome and incidence of type 2 diabetes in patients with manifest vascular disease
Wassink, A. M. J., van der Graaf, Y., Soedamah‐Muthu, S. S., Spiering, W., & Visseren, F. L. J. (2008) · Diabetes and Vascular Disease Research · doi:10.3132/dvdr.2008.019 · Cited by 34
“Abdominal obesity, n (%) 667 (64) 273 (15) 766 (59) 174 (11) 1,421 (100) 435 (29)”
Racial/ethnic differences in emergency care for joint dislocation in 53 US EDs
Tsai, C., Sullivan, A. F., Gordon, J. A., Kaushal, R., Magid, D. J., Blumenthal, D., & Camargo, C. A. (2012) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2012.04.023 · Cited by 34
“There were 9% of patients missing pain assessment at ED presentation, and we used the missing indicator method for this potential confounder in multivariable models [37].”
Method
Differences in mail and telephone responses to self‐rated health: use of multiple imputation in correcting for response bias
Powers, J. R., Mishra, G. D., & Young, A. F. (2005) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1467-842x.2005.tb00065.x · Cited by 34
Young Female Drivers in Fatal Crashes: Recent Trends, 1995–2004
Tsai, V. W., Anderson, C. L., & Vaca, F. E. (2008) · Traffic Injury Prevention · doi:10.1080/15389580701729881 · Cited by 34
“This approach avoids bias that may result from excluding cases with incomplete data (Greenland & Finkle, 1995).”
Background
The child survival impact of the Ghana Essential Health Interventions Program: A health systems strengthening plausibility trial in Northern Ghana
Bawah, A. A., Awoonor‐Williams, J. K., Asuming, P. O., Jackson, E. F., Boyer, C., Kanmiki, E. W., Achana, F. S., Akazili, J., & Phillips, J. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0218025 · Cited by 37
“A regression extension of the DiD concept is estimated for the present analysis that is based on individual observation [50].”
Method
Left Ventricular Mass in Dialysis Patients, Determinants and Relation with Outcome. Results from the COnvective TRansport STudy (CONTRAST)
Mostovaya, I. M., Bots, M. L., van den Dorpel, M. A., Goldschmeding, R., den Hoedt, C. H., Otto Kamp, B., Levesque, R. L., Mazairac, A. H. A., Penne, E. L., Swinkels, D. W., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0084587 · Cited by 34
“Imputation was performed to prevent bias in reported estimates and to improve statistical power [29].”
BackgroundMethod
Analyzing partially missing confounder information in comparative effectiveness and safety research of therapeutics
Toh, S., Rodríguez, L. A. G., & Hernán, M. Á. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3248 · Cited by 33
Reweighted Mahalanobis distance matching for cluster‐randomized trials with missing data
Greevy, R. A., Grijalva, C. G., Roumie, C. L., Beck, C., Hung, A. M., Murff, H. J., Liu, X., & Griffin, M. R. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3260 · Cited by 32
“To avoid potential bias, we discourage including the indicators for missingness that are created by RMD. 18 In addition, A total of 12 sites were selected via simple random samples for the simple randomization, 12 sites optimally selected using phantoms for MD and RMD_race++, and 16 sites selected using chameleons wit…”
MethodInfluential
Variation in Inpatient Rehabilitation Utilization After Hospitalization for Burn Injury in the United States
Greene, N. H., Pham, T. N., Esselman, P. C., et al. (2014) · Journal of Burn Care & Research · doi:10.1097/bcr.0000000000000200 · Cited by 33
Pediatric Obesity Management: Variation by Specialty and Awareness of Guidelines
Rhodes, E., Ebbeling, C., Meyers, A., Bayerl, C., Ooi, W. L., Bettencourt, M., & Ludwig, D. S. (2007) · La Clinica pediatrica · doi:10.1177/0009922806298704 · Cited by 36
Socioeconomic differences in children’s television viewing trajectory: A population-based prospective cohort study
Yang‐Huang, J., van Grieken, A., Moll, H. A., Jaddoe, V. W. V., Wijtzes, A. I., & Raat, H. (2017) · PLoS ONE · doi:10.1371/journal.pone.0188363 · Cited by 33
Heart rate, beta-blocker use, and outcomes of heart failure with reduced ejection fraction
Ibrahim, N. E., Gaggin, H. K., Turchin, A., Patel, H., Song, Y., Trebnick, A., Doros, G. D., Maya, J. F., Cannon, C. P., & Januzzi, J. L. (2018) · European Heart Journal - Cardiovascular Pharmacotherapy · doi:10.1093/ehjcvp/pvy011 · Cited by 32
Handling Missing Data in Nursing Research With Multiple Imputation
Kneipp, S. M., & McIntosh, M. (2001) · Nursing Research · doi:10.1097/00006199-200111000-00010 · Cited by 33
Air Bags and Passenger Fatality According to Passenger Age and Restraint Use
Cummings, P., Koepsell, T. D., Rivara, F. P., et al. (2002) · Epidemiology · doi:10.1097/00001648-200209000-00007 · Cited by 33
Positive affect is not associated with incidence of cardiovascular disease: A population-based study of older persons
Freak‐Poli, R., Mirza, S. S., Franco, O. H., Ikram, M. A., Albert Hofman, D., & Tiemeier, H. W. (2015) · Preventive Medicine · doi:10.1016/j.ypmed.2015.01.032 · Cited by 32
“Additional strengths were the validity of our outcome variable (reviewed medical records for CVD) and the use of multiple imputation for missing covariates (Greenland and Finkle, 1995).”
Method
Association of measured physical performance and demographic and health characteristics with self-reported physical function: implications for the interpretation of self-reported limitations
Louie, G. H., & Ward, M. M. (2010) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-8-84 · Cited by 31
“This allowed us to retain all participants in the analyses, and provides estimates that are less biased than those of a complete-case analysis [35].”
Result
Gap Filling and Quality Control Applied to Meteorological Variables Measured in the Northeast Region of Brazil
Costa, R. L., Gomes, H. B., Pinto, D. D. C., Júnior, R. L. D. R., Silva, F. D. D. S., Gomes, H. B., da Silva, M. C. L., & Herdies, D. L. (2021) · Atmosphere · doi:10.3390/atmos12101278 · Cited by 32
“Several forecasts are thus created for each missing value, and the one that produces the least uncertainty and fewest errors, when compared with the observed data, is adopted [34].”
Result
Doing Data Analysis with the Multilevel Model for Change
Singer, J. D., & Willett, J. B. (2003) · doi:10.1093/acprof:oso/9780195152968.003.0004 · Cited by 33
L’imputation multiple des données manquantes aléatoirement : concepts généraux et présentation d’une méthode Monte-Carlo
Cottrell, G., Cot, M., & Mary, J. (2009) · Revue d Épidémiologie et de Santé Publique · doi:10.1016/j.respe.2009.04.011 · Cited by 33
Data quality considerations for evaluating COVID-19 treatments using real world data: learnings from the National COVID Cohort Collaborative (N3C)
Sidky, H., Young, J. C., Girvin, A. T., Lee, E., Shao, Y. R., Hotaling, N. A., Michael, S., Wilkins, K. J., Setoguchi, S., Funk, M. J., et al. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01839-2 · Cited by 31
“Finally, although complete-case analysis is used by some practitioners in these settings, omitting records with any missing data among variables associated with exposures/ confounders or outcomes is known to bias effects estimates and, at best, reduce precision [32, 38].”
BackgroundMethod
Ethnic differences in frailty: a cross-sectional study of pooled data from community-dwelling older persons in the Netherlands
Franse, C. B., van Grieken, A., Li, Q., Melis, R. J. F., Rietjens, J. A. C., & Raat, H. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-022241 · Cited by 31
“Five imputed datasets were created using a fully conditional specified model, thus taking into account the uncertainty of imputed values.(39) We used pooled estimates from these five imputed datasets to report regression coefficients and their 95% CIs.”
Method
How to apply variable selection machine learning algorithms with multiply imputed data: A missing discussion.
Gunn, H., Rezvan, P. H., Fernandez, M. I., & Comulada, W. S. (2022) · Psychological Methods · doi:10.1037/met0000478 · Cited by 37
Predictors of improved quality of life 1 year after pacemaker implantation
van Eck, J. W. M., van Hemel, N. M., van den Bos, A., Taks, W., Grobbee, D. E., & Moons, K. G. M. (2008) · American Heart Journal · doi:10.1016/j.ahj.2008.04.029 · Cited by 31

Authors: Greenland, S., & Finkle, W. D.
Affiliations: University of California, Los Angeles (US)
Corresponding author: Greenland, Sander
ORCID: Greenland, Sander
Venue: American Journal of Epidemiology
Publisher: Oxford University Press (OUP)
Published: 1995-12-15
Volume: 142
Issue: 12
Pages: 1255-1264
ISSN: 0002-9262 (ISSN-L)
1476-6256
DOI: 10.1093/oxfordjournals.aje.a117592
OpenAlex: W2166561686
Cited by: 891
References: 0
Citations by year: 2026: 13 · 2025: 18 · 2024: 13 · 2023: 24 · 2022: 26 · 2021: 21 · 2020: 33 · 2019: 29 · 2018: 34 · 2017: 32 · 2016: 27 · 2015: 40 · 2014: 57 · 2013: 53 · 2012: 67
FWCI: 6.57
Type: journal article
Language: en
Open access: closed
Locations: American Journal of Epidemiology (publishedVersion)
PubMed (publishedVersion)
American Journal of Epidemiology
Funders: National Institutes of Health (n01cp095680-001)
National Cancer Institute (N01 CP 95680)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords: missing covariates, missing data, logistic regression, bias, model misspecification, multiple imputation, maximum likelihood estimation, weighted estimating equations, endometrial cancer, age at menarche, case-control study
MeSH terms: Epidemiologic Methods, Female, Humans, Menarche, Research Design, Logistic Models, Odds Ratio, Case-Control Studies, Endometrial Neoplasms

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A critical look at methods for handling missing covariates in epidemiologic regression analyses.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cited by

Related

Metadata

Record

About these checks