Missing data methods for arbitrary missingness with small samples
Open accessAbstract
Missing data are a prevalent and widespread data analytic issue and previous studies have performed simulations to compare the performance of missing data methods in various contexts and for various models; however, one such context that has yet to receive much attention in the literature is the handling of missing data with small samples, particularly when the missingness is arbitrary. Prior studies have either compared methods for small samples with monotone missingness commonly found in longitudinal studies or have investigated the performance of a single method to handle arbitrary missingness with small samples but studies have yet to compare the relative performance of commonly implemented missing data methods for small samples with arbitrary missingness. This study conducts a simulation study to compare and assess the small sample performance of maximum likelihood, listwise deletion, joint multiple imputation, and fully conditional specification multiple imputation for a single-level regression model with a continuous outcome. Results showed that, provided assumptions are met, joint multiple imputation unanimously performed best of the methods examined in the conditions under study.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Complete case analysis was conducted rather than intention to treat analysis because multiple imputation could not be implemented (the model would not converge).
“We conducted complete case analysis, rather than intention to treat analysis, as it was not possible to implement multiple imputation (the model would not converge; see McNeish, 2017).”
From Bringing light into darkness: A multiple baseline mixed methods case series evaluation of Augmented Depression Therapy (ADepT) (2019) · cites this paper as McNeish, 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · see citing works
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701 · see citing works
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638 · see citing works
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925 · see citing works
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · see citing works
Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009) · BMJ · doi:10.1136/bmj.b2393 · Cited by 7,299 · see citing works
(2010) · Cited by 7,034 · see citing works
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627 · see citing works
Enders, C., & Bandalos, D. (2001) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0803_5 · Cited by 4,978 · see citing works
Multiple imputation by chained equations: what is it and how does it work?
Azur, M. J., Stuart, E. A., Frangakis, C., & Leaf, P. J. (2011) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.329 · Cited by 3,542 · see citing works
Flora, D. B., & Curran, P. J. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.466 · Cited by 3,105 · see citing works
Multiple imputation of discrete and continuous data by fully conditional specification
van Buuren, S. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206074463 · Cited by 2,827 · see citing works
How Many Imputations are Really Needed? Some Practical Clarifications of Multiple Imputation Theory
Graham, J. W., Olchowski, A. E., & Gilreath, T. D. (2007) · Prevention Science · doi:10.1007/s11121-007-0070-9 · Cited by 2,795 · cited 1× · see citing works
“Additionally, for the conditions in this study, augmenting the number of imputations did not have a discernable effect of power, which has been similarly observed in small to moderate sample size conditions (though not extending as small as included in the present study) in studies focusing on the number of imputation…”
Result<b>Amelia</b> II: A Program for Missing Data
Honaker, J., King, G., & Blackwell, M. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i07 · Cited by 2,552 · see citing works
Statistical Strategies for Small Sample Research
Hoyle, R. H. (1999) · Cited by 1,896 · see citing works
(1996) · Dialnet (Universidad de la Rioja) · Cited by 1,733 · see citing works
Advanced Structural Equation Modeling: Issues and Techniques
Rigdon, E. E. (1998) · Applied Psychological Measurement · doi:10.1177/01466216980221008 · Cited by 1,631 · see citing works
Missing Data Techniques for Structural Equation Modeling.
Allison, P. D. (2003) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.112.4.545 · Cited by 1,335 · see citing works
Peugh, J., & Enders, C. K. (2004) · Review of Educational Research · doi:10.3102/00346543074004525 · Cited by 1,107 · see citing works
A Primer on Maximum Likelihood Algorithms Available for Use With Missing Data
Enders, C. K. (2001) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0801_7 · Cited by 1,027 · see citing works
Regression with Missing X 's: A Review
Little, R. J. A. (1992) · Journal of the American Statistical Association · doi:10.1080/01621459.1992.10476282 · Cited by 968 · see citing works
Miscellanea. Small-sample degrees of freedom with multiple imputation
Barnard, J. (1999) · Biometrika · doi:10.1093/biomet/86.4.948 · Cited by 841 · see citing works
Lee, K. J., & Carlin, J. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwp425 · Cited by 787 · see citing works
Bradley, J. V. (1978) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1978.tb00581.x · Cited by 743 · see citing works
Robustness Studies in Covariance Structure Modeling: An Overview and a Meta-Analysis
HOOGLAND, J. J., & BOOMSMA, A. (1998) · Sociological Methods & Research · doi:10.1177/0049124198026003003 · Cited by 734 · see citing works
The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration
McNeish, D., & Stapleton, L. M. (2014) · Educational Psychology Review · doi:10.1007/s10648-014-9287-x · Cited by 726 · see citing works
Self-citationOn the performance of multiple imputation for multivariate data with small sample size
Graham, J. A., Schafer, J. L., & Hoyle, R. H. (1999) · Cited by 235 · see citing works
Ambler, G., Omar, R. Z., & Royston, P. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206074466 · Cited by 233 · see citing works
Multiple Imputation for the Fatal Accident Reporting System
Heitjan, D. F., & Little, R. J. A. (1991) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347902 · Cited by 227 · see citing works
What Is Meant by “Missing at Random”?
Seaman, S. R., Galati, J. C., Jackson, D., & Carlin, J. B. (2013) · Statistical Science · doi:10.1214/13-sts415 · Cited by 226 · see citing works
Methods to Account for Attrition in Longitudinal Data: Do They Work? A Simulation Study
Kristman, V. L., Manno, M., & Côté, P. (2005) · European Journal of Epidemiology · doi:10.1007/s10654-005-7919-7 · Cited by 116 · see citing works
Multiple Imputation Strategies for Multiple Group Structural Equation Models
Enders, C. K., & Gottschall, A. C. (2011) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2011.532695 · Cited by 114 · see citing works
Multiple imputation techniques in small sample clinical trials
Barnes, S. A., Lindborg, S. R., & Seaman, J. W. (2005) · Statistics in Medicine · doi:10.1002/sim.2231 · Cited by 112 · see citing works
COMPARISON OF ALTERNATIVE STRATEGIES FOR ANALYSIS OF LONGITUDINAL TRIALS WITH DROPOUTS
Liu, G. F., & Gould, A. L. (2002) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120015744 · Cited by 101 · see citing works
Analysis of incomplete multivariate data
Schlittgen, R. (1999) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(99)90025-7 · Cited by 80 · see citing works
Improving upon the efficiency of complete case analysis when covariates are MNAR
Bartlett, J. W., Carpenter, J. R., Tilling, K. M., & Vansteelandt, S. (2014) · Biostatistics · doi:10.1093/biostatistics/kxu023 · Cited by 75 · see citing works
Savalei, V. (2010) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511003659375 · Cited by 68 · see citing works
Testing multivariate normality in incomplete data of small sample size
Tan, M. T., Fang, H., Tian, G., & Wei, G. (2004) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2004.02.014 · Cited by 24 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The registration continuum in clinical science: A guide toward transparent practices
Benning, S. D., Bachrach, R. L., Smith, E. A., Freeman, A. J., & Wright, A. G. C. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000451 · Cited by 77 · see citing works
Dunn, B. D., Widnall, E., Reed, N., Owens, C., Campbell, J., & Kuyken, W. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103418 · Cited by 70 · see citing works
“We conduct complete case analysis, rather than intention to treat analysis, as it was not possible to conduct multiple imputation (the model would not converge; see Mcneish, 2017).”
MethodArtom, M., Czuber-Dochan, W., Sturt, J., Proudfoot, H., Roberts, D., & Norton, C. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0538-y · Cited by 48 · see citing works
- Authors
- McNeish, D.
- Affiliations
- Utrecht University (NL)
- Corresponding author
- McNeish, Daniel
- ORCID
- McNeish, Daniel
- Venue
- Journal of Applied Statistics
- Publisher
- Informa UK Limited
- Published
- 2016-03-22
- Volume
- 44
- Issue
- 1
- Pages
- 24-39
- ISSN
- 0266-4763 (ISSN-L)
- 1360-0532
- OpenAlex
- W2320089545
- Cited by
- 189
- References
- 38
- Citations by year
- 2026: 18 · 2025: 24 · 2024: 41 · 2023: 16 · 2022: 18 · 2021: 27 · 2020: 20 · 2019: 15 · 2018: 4 · 2017: 4 · 2016: 1
- FWCI
- 8.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Applied Statistics (publishedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Keywords
- missing data, small sample size, maximum likelihood estimation, listwise deletion, multiple imputation, fully conditional specification, continuous outcomes, method comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
