The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Multiple imputation of missing values was not necessary before performing a longitudinal mixed-model analysis
Abstract
Background and objectives As a result of the development of sophisticated techniques, such as multiple imputation, the interest in handling missing data in longitudinal studies has increased enormously in past years. Within the field of longitudinal data analysis, there is a current debate on whether it is necessary to use multiple imputations before performing a mixed-model analysis to analyze the longitudinal data. In the current study this necessity is evaluated.Study design and setting The results of mixed-model analyses with and without multiple imputation were compared with each other. Four data sets with missing values were created-one data set with missing completely at random, two data sets with missing at random, and one data set with missing not at random). In all data sets, the relationship between a continuous outcome variable and two different covariates were analyzed: a time-independent dichotomous covariate and a time-dependent continuous covariate.Results Although for all types of missing data, the results of the mixed-model analysis with or without multiple imputations were slightly different, they were not in favor of one of the two approaches. In addition, repeating the multiple imputations 100 times showed that the results of the mixed-model analysis with multiple imputation were quite unstable.Conclusion It is not necessary to handle missing data using multiple imputations before performing a mixed-model analysis on longitudinal data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Multiple imputation when performing mixed models analyses can lead to unstable results [26]”
In the citing paper
“Linear mixed modelling has been demonstrated to control for missing data, and therefore, multiple imputation was not required (Peters et al., 2012;Twisk et al., 2013).”
From Exercise combined with Acceptance and Commitment Therapy for chronic pain: One-year follow-up from a randomized controlled trial (2024) · cites this paper as Twisk et al., 2013
In the citing paper
“Twisk et al 71 provided evidence that multiple imputation for missing data is not necessary before computing longitudinal mixed models.”
In the citing paper
“However, completely missing measurements will not be imputed as this is not necessary for multilevel analysis (63).”
From Timing of imagery rescripting during schema therapy for borderline personality disorder: the LUCY trial (2023) · cites this paper as [63]
In the citing paper
“Therefore, patterns of missingness were not found to be MNAR and the decision to continue the analysis with the available data was made [84,85].”
From Negative and positive affect regulation in a transdiagnostic internet-based protocol for emotional disorders: Randomized controlled trial (2021) · cites this paper as [84]
In the citing paper
“Mixed model analysis handles missing data; no imputation method was applied since there are reports that imputations before a mixed model analysis is not necessary and simulations indicated that imputations before mixed models could result in unstable results [30].”
From Is sleep disturbance in patients with chronic pain affected by physical exercise or ACT-based stress management? - A randomized controlled study (2018) · cites this paper as [30]
In the citing paper
“We also performed all analyses on the data for 100 imputations. Because there is a current debate whether it is necessary to perform multiple imputations in combination with mixed-model analyses in longitudinal studies [65], we also performed the LMM analyses without multiple imputations.”
From Long-Term Results of a Web-Based Guided Self-Help Intervention for Employees With Depressive Symptoms: Randomized Controlled Trial (2014) · cites this paper as [65]
In the citing paper
“Because of the current debate on the necessity of multiple imputations in combination with mixed-model analysis in longitudinal studies [65], we also performed mixed-model analysis without multiple imputations. The results were comparable, indicating that data were robust and multiple imputations may not have been needed.”
From Long-Term Results of a Web-Based Guided Self-Help Intervention for Employees With Depressive Symptoms: Randomized Controlled Trial (2014) · cites this paper as [65]
Citations
Cites
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
(2010) · Cited by 7,034
Missing data analysis: making it work in the real world
Graham, J. W. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085530 · Cited by 6,163
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949
Statistical Analysis With Missing Data
Lazar, N. A. (2003) · Technometrics · doi:10.1198/tech.2003.s167 · Cited by 2,477
Multiple Imputation of Missing Values
Royston, P. (2004) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0400400301 · Cited by 2,340 · cited 1×
“[28,29] discuss the impact of the number of imputations on the precision of the estimates and suggest ways of determining the required number of imputations by evaluating the sampling error of the multiple-imputation estimates.”
BackgroundMultiple Imputation for Nonresponse in Surveys.
Lynn, P. J., & Rubin, D. B. (1988) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348774 · Cited by 2,330
Hox, J. J. (1995) · Utrecht University Repository (Utrecht University) · Cited by 1,179
Pattern-Mixture Models for Multivariate Incomplete Data
Little, R. J. A. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594302 · Cited by 1,026
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1191/096228099671525676 · Cited by 920
Twisk, J. W. R. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511610806 · Cited by 861
Self-citationThe design of simulation studies in medical statistics
Burton, A., Altman, D. G., Royston, P. J., & HOLDER, R. L. (2006) · Statistics in Medicine · doi:10.1002/sim.2673 · Cited by 755
Multiple imputation: current perspectives
Kenward, M., & Carpenter, J. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206075304 · Cited by 491
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/5.3.445 · Cited by 463
Royston, P. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900308 · Cited by 428
Handling drop‐out in longitudinal studies
Hogan, J. W., Roy, J., & Korkontzelou, C. (2004) · Statistics in Medicine · doi:10.1002/sim.1728 · Cited by 404
A Class of Pattern-Mixture Models for Normal Incomplete Data
Little, R. J. A. (1994) · Biometrika · doi:10.1093/biomet/81.3.471 · Cited by 403
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/kxh001 · Cited by 378
Applications of multiple imputation in medical studies: from AIDS to NHANES
Barnard, J., & Meng, X.-L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800103 · Cited by 340
Stata multiple-imputation reference manual
StataCorp (2011) · Stata Press/StataCorp eBooks · Cited by 284
Inference for imputation estimators
Robins, J. M. (2000) · Biometrika · doi:10.1093/biomet/87.1.113 · Cited by 278
Multiple Imputation With Large Data Sets: A Case Study of the Children's Mental Health Initiative
Stuart, E. A., Azur, M. J., Frangakis, C., & Leaf, P. J. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp026 · Cited by 267
Multiple Imputation of Missing Values: New Features for Mim
Royston, P. J., Carlin, J. B., & White, I. R. (2009) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900205 · Cited by 217
Robustness of a multivariate normal approximation for imputation of incomplete binary data
Bernaards, C. A., Belin, T. R., & Schafer, J. L. (2006) · Statistics in Medicine · doi:10.1002/sim.2619 · Cited by 191
Gadbury, G. L., Coffey, C. S., & Allison, D. B. (2003) · Obesity Reviews · doi:10.1046/j.1467-789x.2003.00109.x · Cited by 181
On the performance of random‐coefficient pattern‐mixture models for non‐ignorable drop‐out
Demirtaş, H., & Schafer, J. L. (2003) · Statistics in Medicine · doi:10.1002/sim.1475 · Cited by 177
Peters, S. A. E., Bots, M. L., den Ruijter, H. M., Palmer, M. K., Grobbee, D. E., Crouse, J. R., O’Leary, D. H., Evans, G. W., Raichlen, J. S., Moons, K. G. M., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.012 · Cited by 165
Demirtaş, H., Freels, S. A., & Yucel, R. M. (2008) · Journal of Statistical Computation and Simulation · doi:10.1080/10629360600903866 · Cited by 153
Beunckens, C., Molenberghs, G., & Kenward, M. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn119oa · Cited by 153
Can one assess whether missing data are missing at random in medical studies?
Potthoff, R. F., Tudor, G. E., Pieper, K. S., & Hasselblad, V. (2006) · Statistical Methods in Medical Research · doi:10.1191/0962280206sm448oa · Cited by 141
Methods to Account for Attrition in Longitudinal Data: Do They Work? A Simulation Study
Kristman, V. L., Manno, M., & Côté, P. (2005) · European Journal of Epidemiology · doi:10.1007/s10654-005-7919-7 · Cited by 116
Nevalainen, J., Kenward, M. G., & Virtanen, S. (2009) · Statistics in Medicine · doi:10.1002/sim.3731 · Cited by 93
Applications of multiple imputation in medical studies: from AIDS to NHANES
Barnard, J., & Meng, X.-L. (1999) · Statistical Methods in Medical Research · doi:10.1191/096228099666230705 · Cited by 84
Multiple Imputation and Posterior Simulation for Multivariate Missing Data in Longitudinal Studies
Liu, M., Taylor, J. M. G., & Belin, T. R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.01157.x · Cited by 69
Statistical analysis of longitudinal psychiatric data with dropouts
Mazumdar, S., Tang, G., Houck, P. R., Dew, M. A., Begley, A. E., Scott, J., Mulsant, B. H., & Reynolds, C. F. (2006) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2006.09.007 · Cited by 50
Using Calibration to Improve Rounding in Imputation
Yucel, R. M., He, Y., & Zaslavsky, A. M. (2008) · The American Statistician · doi:10.1198/000313008x300912 · Cited by 49
Predicting Presidential and Other Multistage Election Outcomes Using State-Level Pre-Election Polls
Christensen, W. F., & Florence, L. W. (2008) · The American Statistician · doi:10.1198/000313008x267820 · Cited by 38
Imputation‐based strategies for clinical trial longitudinal data with nonignorable missing values
Yang, X., Li, J., & SHOPTAW, S. J. (2008) · Statistics in Medicine · doi:10.1002/sim.3111 · Cited by 35
A multiple imputation strategy for incomplete longitudinal data
Landrum, M. B., & Becker, M. P. (2001) · Statistics in Medicine · doi:10.1002/sim.740 · Cited by 26
A hot‐deck multiple imputation procedure for gaps in longitudinal data on recurrent events
Little, R. J. A., Yosef, M., Cain, K. C., Nan, B., & Harlow, S. D. (2007) · Statistics in Medicine · doi:10.1002/sim.2939 · Cited by 22
Handling missing quality of life data in HIV clinical trials: what is practical?
Fairclough, D. L., Thijs, H., Huang, I., Finnern, H. W., & Wu, A. W. (2007) · Quality of Life Research · doi:10.1007/s11136-007-9284-3 · Cited by 21
Barzi, F., Woodward, M., Marfisi, R. M., Tognoni, G., Marchioli, R., & on behalf of GISSI-Prevenzione Investigators (2006) · European Journal of Epidemiology · doi:10.1007/s10654-005-5086-5 · Cited by 14
Cited by
Sorted by citations
Handling missing data in clinical research
Heymans, M. W., & Twisk, J. W. R. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.08.016 · Cited by 399
“Furthermore, in longitudinal data analyses, when outcome data are missing in some of the repeated measures, an analysis on the available data will also provide valid results [10].”
BackgroundSelf-citationDementia and caregiver burden: A three‐year longitudinal study
Connors, M. H., Seeher, K., Teixeira‐Pinto, A., Woodward, M., Ames, D., & Brodaty, H. (2020) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.5244 · Cited by 265
“We used linear mixed models to analyze the data; these have the advantage of being able to handle missing data, including attrition, without the need for imputation.(31) Based on previous research, we expected that a range of patient characteristics—such as higher levels of neuropsychiatric symptoms, greater cognitive…”
MethodAllam, A., Kostova, Z., Nakamoto, K., & Schulz, P. J. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3510 · Cited by 274
“Recent research [70] showed that for all types of missing data (missing completely at random, missing at random, and missing not at random), multiple imputation is not necessary before performing longitudinal mixed model analysis.”
Gottenberg, J., Brocq, O., Perdriger, A., Lassoued, S., Berthelot, J., Wendling, D., Euller‐Ziegler, L., Soubrier, M., Richez, C., Fautrel, B., et al. (2016) · JAMA · doi:10.1001/jama.2016.13512 · Cited by 190
Nerland, U. S., Jakola, A. S., Solheim, O. S., Weber, C., Rao, V., Lønne, G., Solberg, T. K., Salvesen, Ø. O., Carlsen, S. M., Nygaard, O. P., et al. (2015) · BMJ · doi:10.1136/bmj.h1603 · Cited by 179
Metadata
- Authors
- Twisk, J., de Boer, M., de Vente, W., & Heymans, M.
- Affiliations
- Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Twisk, Jos
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2013-09
- Volume
- 66
- Issue
- 9
- Pages
- 1022-1028
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W1974992958
- Cited by
- 568
- References
- 43
- Citations by year
- 2026: 58 · 2025: 68 · 2024: 74 · 2023: 57 · 2022: 61 · 2021: 52 · 2020: 46 · 2019: 45 · 2018: 27 · 2017: 26 · 2016: 21 · 2015: 19 · 2014: 10 · 2013: 4
- FWCI
- 15.69
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- VU Research Portal (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- multiple imputation, missing data, longitudinal analysis, mixed models, missing completely at random, missing at random, missing not at random, time-varying covariates
- MeSH terms
- Data Interpretation, Statistical, Humans, Longitudinal Studies, Patient Dropouts, Research Design, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
