The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
The proportion of missing data should not be used to guide decisions on multiple imputation
Open accessAbstract
OBJECTIVES: Researchers are concerned whether multiple imputation (MI) or complete case analysis should be used when a large proportion of data are missing. We aimed to provide guidance for drawing conclusions from data with a large proportion of missingness. STUDY DESIGN AND SETTING: Via simulations, we investigated how the proportion of missing data, the fraction of missing information (FMI), and availability of auxiliary variables affected MI performance. Outcome data were missing completely at random or missing at random (MAR). RESULTS: Provided sufficient auxiliary information was available; MI was beneficial in terms of bias and never detrimental in terms of efficiency. Models with similar FMI values, but differing proportions of missing data, also had similar precision for effect estimates. In the absence of bias, the FMI was a better guide to the efficiency gains using MI than the proportion of missing data. CONCLUSION: We provide evidence that for MAR data, valid MI reduces bias even when the proportion of missingness is large. We advise researchers to use FMI to guide choice of auxiliary variables for efficiency gain in imputation analyses, and that sensitivity analyses including different imputation models may be needed if the number of complete cases is small.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,265
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · cited 2×
“These are included to make the MAR assumption (required in the standard implementation of MI to produce unbiased estimates) more plausible and to provide information about the missing values [11].”
BackgroundSterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009) · BMJ · doi:10.1136/bmj.b2393 · Cited by 7,299
Missing data analysis: making it work in the real world
Graham, J. W. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085530 · Cited by 6,163
Statistical Analysis With Missing Data
Ghosh, S. (1988) · Technometrics · doi:10.1080/00401706.1988.10488446 · Cited by 4,576
Multiple imputation after 18+ years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476908 · Cited by 4,072
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949 · cited 1×
“Varying guidance exists; in the literature, 5% missingness has been suggested as a lower threshold below which MI provides negligible benefit [16].”
BackgroundBoyd, A., Golding, J., Macleod, J., Lawlor, D. A., Fraser, A., Henderson, J., Molloy, L., Ness, A., Ring, S., & Davey Smith, G. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys064 · Cited by 2,996
Jakobsen, J. C., Gluud, C., Wetterslev, J., & Winkel, P. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0442-1 · Cited by 2,747 · cited 2×
“These findings provide an example of valid estimates from properly specified MI at much larger proportions of missing data than current guidance [19] advises.”
BackgroundReview: a gentle introduction to imputation of missing values.
Donders, A. R. T., van der Heijden, G. J. M. G., Stijnen, T., & Moons, K. G. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.014 · Cited by 2,582
Cohort Profile: The Avon Longitudinal Study of Parents and Children: ALSPAC mothers cohort
Fraser, A., Macdonald-Wallis, C., Tilling, K., Boyd, A., Golding, J., Davey Smith, G., Henderson, J., Macleod, J., Molloy, L., Ness, A., et al. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dys066 · Cited by 2,494 · cited 7×
“The authors thank the midwives for their help in recruiting the families and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists, and nurses.”
MethodInfluentialSelf-citationA comparison of inclusive and restrictive strategies in modern missing data procedures.
Collins, L. M., Schafer, J. L., & Kam, C.-M. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.330 · Cited by 2,389
Multiple Imputation for Nonresponse in Surveys.
Lynn, P., & Rubin, D. B. (1988) · The Statistician · doi:10.2307/2348774 · Cited by 2,329
Principled missing data methods for researchers
Dong, Y., & Peng, C.-Y. J. (2013) · SpringerPlus · doi:10.1186/2193-1801-2-222 · Cited by 2,208 · cited 1×
“Statistical guidance articles have stated that bias is likely in analyses with more than 10% missingness and that if more than 40% data are missing in important variables then results should only be considered as hypothesis generating [18,19].”
BackgroundReview of inverse probability weighting for dealing with missing data
Seaman, S. R., & White, I. R. (2013) · Statistical Methods in Medical Research · doi:10.1177/0962280210395740 · Cited by 1,654
Using the outcome for imputation of missing predictor values was preferred
Moons, K. G. M., Donders, R. A. R. T., Stijnen, T., & Harrell, F. E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.009 · Cited by 994
Multiple Imputation for Missing Data: A Cautionary Tale
ALLISON, P. D. (2000) · Sociological Methods & Research · doi:10.1177/0049124100028003003 · Cited by 795
Bishop, D. V. M. (1998) · Journal of Child Psychology and Psychiatry · doi:10.1017/s0021963098002832 · Cited by 758
White, I. R., & Carlin, J. B. (2010) · Statistics in Medicine · doi:10.1002/sim.3944 · Cited by 744
What Improves with Increased Missing Data Imputations?
Bodner, T. E. (2008) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510802339072 · Cited by 606
Sensitivity analysis after multiple imputation under missing at random: a weighting approach
Carpenter, J. R., Kenward, M. G., & White, I. R. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206075303 · Cited by 233
Missing data methods for arbitrary missingness with small samples
McNeish, D. (2017) · Journal of Applied Statistics · doi:10.1080/02664763.2016.1158246 · Cited by 189
Hardt, J., Herke, M., & Leonhart, R. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-184 · Cited by 171
Multivariate Imputation by Chained Equations: Mice V1.0 User's manual
van Buuren, S., & Groothuis‐Oudshoorn, C. G. M. (2000) · Cited by 167
Asymptotically Unbiased Estimation of Exposure Odds Ratios in Complete Records Logistic Regression
Bartlett, J. W., Harel, O., & Carpenter, J. R. (2015) · American Journal of Epidemiology · doi:10.1093/aje/kwv114 · Cited by 161
Recovery of information from multiple imputation: a simulation study
Lee, K. J., & Carlin, J. B. (2012) · Emerging Themes in Epidemiology · doi:10.1186/1742-7622-9-3 · Cited by 113
Appropriate inclusion of interactions was needed to avoid bias in multiple imputation
Tilling, K. M., Williamson, E., Spratt, M., Sterne, J. A. C., & Carpenter, J. R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.07.004 · Cited by 109
Self-citationMissing Data Estimation in Morphometrics: How Much is Too Much?
Clavel, J., Merceron, G., & Escarguel, G. (2013) · Systematic Biology · doi:10.1093/sysbio/syt100 · Cited by 89
The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data
Wagner, J. R. (2010) · Public Opinion Quarterly · doi:10.1093/poq/nfq007 · Cited by 85
Simsum: Analyses of Simulation Studies Including Monte Carlo Error
White, I. R. (2010) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1001000305 · Cited by 82
Saklofske, D. H., Caravan, G., & Schwartz, C. (2000) · Canadian Journal of School Psychology · doi:10.1177/082957350001600106 · Cited by 55
Multiple Imputation of Missing Data: A Simulation Study on a Binary Response
Hardt, J., Herke, M., Brian, T., & Laubach, W. (2013) · Open Journal of Statistics · doi:10.4236/ojs.2013.35043 · Cited by 49
Mishra, S., & Khare, D. (2014) · Journal of Medical Statistics and Informatics · doi:10.7243/2053-7662-2-9 · Cited by 48
Zhang, Y., Flórez, I. D., Colunga‐Lozano, L. E., Aloweni, F. B. A. B., Kennedy, S. A., Li, A., Craigie, S., Zhang, S., Agarwal, A., Lopes, L. C., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.017 · Cited by 36
Jones, M., Mishra, G. D., & Dobson, A. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.011 · Cited by 28
Cited by
Accounting for missing data in statistical analyses: multiple imputation is not always the answer
Hughes, R. A., Heron, J., Sterne, J. A. C., & Tilling, K. (2019) · International Journal of Epidemiology · doi:10.1093/ije/dyz032 · Cited by 802
Self-citationCost, K. T., Crosbie, J., Anagnostou, E., Birken, C. S., Charach, A., Monga, S., Kelley, E., Nicolson, R., Maguire, J. L., Burton, C. L., et al. (2022) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-021-01744-3 · Cited by 463
Lee, K. J., Tilling, K. M., Cornish, R. P., Little, R. J. A., Bell, M. L., Goetghebeur, E., Hogan, J. W., & Carpenter, J. R. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.01.008 · Cited by 374
Self-citationSICE: an improved missing data imputation technique
Khan, S. I., & Hoque, A. S. M. L. (2020) · Journal of Big Data · doi:10.1186/s40537-020-00313-w · Cited by 279
The normative modeling framework for computational psychiatry
Rutherford, S., Kia, S. M., Wolfers, T., Fraza, C., Zabihi, M., Dinga, R., Berthet, P., Worker, A., Verdi, S., Ruhe, H. G., et al. (2022) · Nature Protocols · doi:10.1038/s41596-022-00696-5 · Cited by 274
Metadata
- Authors
- Madley-Dowd, P., Hughes, R., Tilling, K., & Heron, J.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2019-06
- Volume
- 110
- Pages
- 63-73
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2920529430
- Cited by
- 1,053
- References
- 38
- FWCI
- 73.01
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.