Multiple imputation of missing values was not necessary before performing a longitudinal mixed-model analysis

Twisk, Jos; de Boer, Michiel; de Vente, Wieke; Heymans, Martijn

doi:10.1016/j.jclinepi.2013.03.017

Cited by 568Statistical Methods and Bayesian InferenceStatistical Methods in Clinical TrialsStatistical Methods in Epidemiology

Multiple imputation of missing values was not necessary before performing a longitudinal mixed-model analysis

Twisk, Jos, de Boer, Michiel, de Vente, Wieke, Heymans, Martijn·2013·Journal of Clinical Epidemiology

DOI 10.1016/j.jclinepi.2013.03.017

Open this paper in CoChat — free account

Abstract

Background and objectives As a result of the development of sophisticated techniques, such as multiple imputation, the interest in handling missing data in longitudinal studies has increased enormously in past years. Within the field of longitudinal data analysis, there is a current debate on whether it is necessary to use multiple imputations before performing a mixed-model analysis to analyze the longitudinal data. In the current study this necessity is evaluated.Study design and setting The results of mixed-model analyses with and without multiple imputation were compared with each other. Four data sets with missing values were created-one data set with missing completely at random, two data sets with missing at random, and one data set with missing not at random). In all data sets, the relationship between a continuous outcome variable and two different covariates were analyzed: a time-independent dichotomous covariate and a time-dependent continuous covariate.Results Although for all types of missing data, the results of the mixed-model analysis with or without multiple imputations were slightly different, they were not in favor of one of the two approaches. In addition, repeating the multiple imputations 100 times showed that the results of the mixed-model analysis with multiple imputation were quite unstable.Conclusion It is not necessary to handle missing data using multiple imputations before performing a mixed-model analysis on longitudinal data.

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2013.03.017

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Multiple imputation when performing mixed models analyses can lead to unstable results.
“Multiple imputation when performing mixed models analyses can lead to unstable results [26]”
From Effects of a school-based physical activity intervention on academic performance in 14-year old adolescents: a cluster randomized controlled trial – the School in Motion study (2021) · cites this paper as [26]
Linear mixed modelling has been demonstrated to control for missing data, so multiple imputation was not required.
“Linear mixed modelling has been demonstrated to control for missing data, and therefore, multiple imputation was not required (Peters et al., 2012;Twisk et al., 2013).”
From Exercise combined with Acceptance and Commitment Therapy for chronic pain: One-year follow-up from a randomized controlled trial (2024) · cites this paper as Twisk et al., 2013
Multiple imputation for missing data is not necessary before computing longitudinal mixed models.
“Twisk et al 71 provided evidence that multiple imputation for missing data is not necessary before computing longitudinal mixed models.”
From Efficacy of Videoconference Group Acceptance and Commitment Therapy (ACT) and Behavioral Activation Therapy for Depression (BATD) for Chronic Low Back Pain (CLBP) Plus Comorbid Depressive Symptoms: A Randomized Controlled Trial (IMPACT Study) (2023) · cites this paper as [71]
Completely missing measurements will not be imputed as this is not necessary for multilevel analysis.
“However, completely missing measurements will not be imputed as this is not necessary for multilevel analysis (63).”
From Timing of imagery rescripting during schema therapy for borderline personality disorder: the LUCY trial (2023) · cites this paper as [63]
Patterns of missingness were not found to be missing not at random (MNAR), and the decision to continue the analysis with the available data was made.
“Therefore, patterns of missingness were not found to be MNAR and the decision to continue the analysis with the available data was made [84,85].”
From Negative and positive affect regulation in a transdiagnostic internet-based protocol for emotional disorders: Randomized controlled trial (2021) · cites this paper as [84]

Mixed model analysis handles missing data, and no imputation method was applied because reports and simulations indicate that imputation before mixed model analysis is not necessary and could result in unstable results.
“Mixed model analysis handles missing data; no imputation method was applied since there are reports that imputations before a mixed model analysis is not necessary and simulations indicated that imputations before mixed models could result in unstable results [30].”
From Is sleep disturbance in patients with chronic pain affected by physical exercise or ACT-based stress management? - A randomized controlled study (2018) · cites this paper as [30]
We also performed all analyses on the data for 100 imputations and performed the LMM analyses without multiple imputations.
“We also performed all analyses on the data for 100 imputations. Because there is a current debate whether it is necessary to perform multiple imputations in combination with mixed-model analyses in longitudinal studies [65], we also performed the LMM analyses without multiple imputations.”
From Long-Term Results of a Web-Based Guided Self-Help Intervention for Employees With Depressive Symptoms: Randomized Controlled Trial (2014) · cites this paper as [65]
Mixed-model analysis without multiple imputations was also performed, and the results were comparable, indicating that data were robust and multiple imputations may not have been needed.
“Because of the current debate on the necessity of multiple imputations in combination with mixed-model analysis in longitudinal studies [65], we also performed mixed-model analysis without multiple imputations. The results were comparable, indicating that data were robust and multiple imputations may not have been needed.”
From Long-Term Results of a Web-Based Guided Self-Help Intervention for Employees With Depressive Symptoms: Randomized Controlled Trial (2014) · cites this paper as [65]

Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701 · see citing works
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
Untitled work
(2010) · Cited by 7,034 · see citing works
Missing data analysis: making it work in the real world
Graham, J. W. (2009) · Annual Review of Psychology · doi:10.1146/annurev.psych.58.110405.085530 · Cited by 6,163 · see citing works
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012 · see citing works
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627 · see citing works
Multiple imputation: a primer
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949 · see citing works
Statistical Analysis With Missing Data
Lazar, N. A. (2003) · Technometrics · doi:10.1198/tech.2003.s167 · Cited by 2,479 · see citing works
Multiple Imputation of Missing Values
Royston, P. (2004) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0400400301 · Cited by 2,340 · cited 1× · see citing works
“[28,29] discuss the impact of the number of imputations on the precision of the estimates and suggest ways of determining the required number of imputations by evaluating the sampling error of the multiple-imputation estimates.”
Background
Multiple Imputation for Nonresponse in Surveys.
Lynn, P. J., & Rubin, D. B. (1988) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348774 · Cited by 2,330 · see citing works
Applied multilevel analysis
Hox, J. J. (1995) · Utrecht University Repository (Utrecht University) · Cited by 1,179 · see citing works
Pattern-Mixture Models for Multivariate Incomplete Data
Little, R. J. A. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594302 · Cited by 1,026 · see citing works
Multiple imputation: a primer
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1191/096228099671525676 · Cited by 920 · see citing works
The design of simulation studies in medical statistics
Burton, A., Altman, D. G., Royston, P. J., & HOLDER, R. L. (2006) · Statistics in Medicine · doi:10.1002/sim.2673 · Cited by 755 · see citing works
Applied Multilevel Analysis
Twisk, J. W. R. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511610806 · Cited by 692 · see citing works
Self-citation
Multiple imputation: current perspectives
Kenward, M., & Carpenter, J. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206075304 · Cited by 491 · see citing works
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/5.3.445 · Cited by 463 · see citing works
Multiple Imputation of Missing Values: Further Update of Ice, with an Emphasis on Categorical Variables
Royston, P. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900308 · Cited by 428 · see citing works
Handling drop‐out in longitudinal studies
Hogan, J. W., Roy, J., & Korkontzelou, C. (2004) · Statistics in Medicine · doi:10.1002/sim.1728 · Cited by 404 · see citing works
A Class of Pattern-Mixture Models for Normal Incomplete Data
Little, R. J. A. (1994) · Biometrika · doi:10.1093/biomet/81.3.471 · Cited by 403 · see citing works
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/kxh001 · Cited by 378 · see citing works
Applications of multiple imputation in medical studies: from AIDS to NHANES
Barnard, J., & Meng, X.-L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800103 · Cited by 340 · see citing works
Stata multiple-imputation reference manual
StataCorp (2011) · Stata Press/StataCorp eBooks · Cited by 285 · see citing works
Inference for imputation estimators
Robins, J. M. (2000) · Biometrika · doi:10.1093/biomet/87.1.113 · Cited by 278 · see citing works
Multiple Imputation With Large Data Sets: A Case Study of the Children's Mental Health Initiative
Stuart, E. A., Azur, M. J., Frangakis, C., & Leaf, P. J. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp026 · Cited by 267 · see citing works
Multiple Imputation of Missing Values: New Features for Mim
Royston, P. J., Carlin, J. B., & White, I. R. (2009) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900205 · Cited by 217 · see citing works
Robustness of a multivariate normal approximation for imputation of incomplete binary data
Bernaards, C. A., Belin, T. R., & Schafer, J. L. (2006) · Statistics in Medicine · doi:10.1002/sim.2619 · Cited by 191 · see citing works
Modern statistical methods for handling missing repeated measurements in obesity trial data: beyond LOCF
Gadbury, G. L., Coffey, C. S., & Allison, D. B. (2003) · Obesity Reviews · doi:10.1046/j.1467-789x.2003.00109.x · Cited by 181 · see citing works
On the performance of random‐coefficient pattern‐mixture models for non‐ignorable drop‐out
Demirtaş, H., & Schafer, J. L. (2003) · Statistics in Medicine · doi:10.1002/sim.1475 · Cited by 177 · see citing works
Multiple imputation of missing repeated outcome measurements did not add to linear mixed-effects models
Peters, S. A. E., Bots, M. L., den Ruijter, H. M., Palmer, M. K., Grobbee, D. E., Crouse, J. R., O’Leary, D. H., Evans, G. W., Raichlen, J. S., Moons, K. G. M., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.012 · Cited by 165 · see citing works
Plausibility of multivariate normality assumption when multiply imputing non-Gaussian continuous outcomes: a simulation assessment
Demirtaş, H., Freels, S. A., & Yucel, R. M. (2008) · Journal of Statistical Computation and Simulation · doi:10.1080/10629360600903866 · Cited by 153 · see citing works
Direct likelihood analysis versus simple forms of imputation for missing data in randomized clinical trials
Beunckens, C., Molenberghs, G., & Kenward, M. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn119oa · Cited by 152 · see citing works
Can one assess whether missing data are missing at random in medical studies?
Potthoff, R. F., Tudor, G. E., Pieper, K. S., & Hasselblad, V. (2006) · Statistical Methods in Medical Research · doi:10.1191/0962280206sm448oa · Cited by 141 · see citing works
Methods to Account for Attrition in Longitudinal Data: Do They Work? A Simulation Study
Kristman, V. L., Manno, M., & Côté, P. (2005) · European Journal of Epidemiology · doi:10.1007/s10654-005-7919-7 · Cited by 116 · see citing works
Missing values in longitudinal dietary data: A multiple imputation approach based on a fully conditional specification
Nevalainen, J., Kenward, M. G., & Virtanen, S. (2009) · Statistics in Medicine · doi:10.1002/sim.3731 · Cited by 93 · see citing works
Applications of multiple imputation in medical studies: from AIDS to NHANES
Barnard, J., & Meng, X.-L. (1999) · Statistical Methods in Medical Research · doi:10.1191/096228099666230705 · Cited by 84 · see citing works
Multiple Imputation and Posterior Simulation for Multivariate Missing Data in Longitudinal Studies
Liu, M., Taylor, J. M. G., & Belin, T. R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.01157.x · Cited by 69 · see citing works
Statistical analysis of longitudinal psychiatric data with dropouts
Mazumdar, S., Tang, G., Houck, P. R., Dew, M. A., Begley, A. E., Scott, J., Mulsant, B. H., & Reynolds, C. F. (2006) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2006.09.007 · Cited by 50 · see citing works
Using Calibration to Improve Rounding in Imputation
Yucel, R. M., He, Y., & Zaslavsky, A. M. (2008) · The American Statistician · doi:10.1198/000313008x300912 · Cited by 49 · see citing works
Predicting Presidential and Other Multistage Election Outcomes Using State-Level Pre-Election Polls
Christensen, W. F., & Florence, L. W. (2008) · The American Statistician · doi:10.1198/000313008x267820 · Cited by 38 · see citing works
Imputation‐based strategies for clinical trial longitudinal data with nonignorable missing values
Yang, X., Li, J., & SHOPTAW, S. J. (2008) · Statistics in Medicine · doi:10.1002/sim.3111 · Cited by 35 · see citing works
A multiple imputation strategy for incomplete longitudinal data
Landrum, M. B., & Becker, M. P. (2001) · Statistics in Medicine · doi:10.1002/sim.740 · Cited by 26 · see citing works
A hot‐deck multiple imputation procedure for gaps in longitudinal data on recurrent events
Little, R. J. A., Yosef, M., Cain, K. C., Nan, B., & Harlow, S. D. (2007) · Statistics in Medicine · doi:10.1002/sim.2939 · Cited by 22 · see citing works
Handling missing quality of life data in HIV clinical trials: what is practical?
Fairclough, D. L., Thijs, H., Huang, I., Finnern, H. W., & Wu, A. W. (2007) · Quality of Life Research · doi:10.1007/s11136-007-9284-3 · Cited by 21 · see citing works
Analysis of the Benefits of a Mediterranean Diet in the GISSI-Prevenzione Study: A Case Study in Imputation of Missing Values from Repeated Measurements
Barzi, F., Woodward, M., Marfisi, R. M., Tognoni, G., Marchioli, R., & on behalf of GISSI-Prevenzione Investigators (2006) · European Journal of Epidemiology · doi:10.1007/s10654-005-5086-5 · Cited by 14 · see citing works
Untitled work
(1999) · doi:10.1002/(sici)1097-0258(19990915/30)18:17/18

Flagged first, then by citations

Sorted by citations

Handling missing data in clinical research
Heymans, M. W., & Twisk, J. W. R. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.08.016 · Cited by 399 · see citing works
“Furthermore, in longitudinal data analyses, when outcome data are missing in some of the repeated measures, an analysis on the available data will also provide valid results [10].”
BackgroundSelf-citation
Dementia and caregiver burden: A three‐year longitudinal study
Connors, M. H., Seeher, K., Teixeira‐Pinto, A., Woodward, M., Ames, D., & Brodaty, H. (2020) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.5244 · Cited by 265 · see citing works
“We used linear mixed models to analyze the data; these have the advantage of being able to handle missing data, including attrition, without the need for imputation.(31) Based on previous research, we expected that a range of patient characteristics—such as higher levels of neuropsychiatric symptoms, greater cognitive…”
Method
The effect of social support features and gamification on a Web-based intervention for rheumatoid arthritis patients: randomized controlled trial
Allam, A., Kostova, Z., Nakamoto, K., & Schulz, P. J. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3510 · Cited by 274 · see citing works
“Recent research [70] showed that for all types of missing data (missing completely at random, missing at random, and missing not at random), multiple imputation is not necessary before performing longitudinal mixed model analysis.”
Non–TNF-Targeted Biologic vs a Second Anti-TNF Drug to Treat Rheumatoid Arthritis in Patients With Insufficient Response to a First Anti-TNF Drug
Gottenberg, J., Brocq, O., Perdriger, A., Lassoued, S., Berthelot, J., Wendling, D., Euller‐Ziegler, L., Soubrier, M., Richez, C., Fautrel, B., et al. (2016) · JAMA · doi:10.1001/jama.2016.13512 · Cited by 190 · see citing works
Minimally invasive decompression versus open laminectomy for central stenosis of the lumbar spine: pragmatic comparative effectiveness study
Nerland, U. S., Jakola, A. S., Solheim, O. S., Weber, C., Rao, V., Lønne, G., Solberg, T. K., Salvesen, Ø. O., Carlsen, S. M., Nygaard, O. P., et al. (2015) · BMJ · doi:10.1136/bmj.h1603 · Cited by 179 · see citing works
Effectiveness of a Blended Physical Therapist Intervention in People With Hip Osteoarthritis, Knee Osteoarthritis, or Both: A Cluster-Randomized Controlled Trial
Kloek, C. J. J., Bossen, D., Spreeuwenberg, P. M. M., Dekker, J., de Bakker, D. H., & Veenhof, C. (2018) · Physical Therapy · doi:10.1093/ptj/pzy045 · Cited by 141 · see citing works
“28 According to the recommendations of Twisk et al 29,30 about handling missing data in longitudinal mixed-model analyses, no imputation techniques were used.”
Sex Differences in Age-Related Loss of Kidney Function
Melsom, T., Norvik, J. V., Enoksen, I. T., Stefansson, V. T. N., Mathisen, U. D., Fuskevåg, O., Jenssen, T. G., Solbu, M. D., & Eriksen, B. O. (2022) · Journal of the American Society of Nephrology · doi:10.1681/asn.2022030323 · Cited by 146 · see citing works
“All study participants (n51837) were included in the linear mixed model and GAMM analyses because mixed models allow for missing observations at one or more points in time.(25,26) The Akaike information criterion (AIC) was used to compare the fit of the different models.”
Method
Effect of Fractional Carbon Dioxide Laser vs Sham Treatment on Symptom Severity in Women With Postmenopausal Vaginal Symptoms
Li, F. G., Maheux‐Lacroix, S., Deans, R., Nesbitt‐Hawes, E. M., Budden, A., Nguyen, K., Lim, C. Y., Song, S., McCormack, L., Lyons, S. D., et al. (2021) · JAMA · doi:10.1001/jama.2021.14892 · Cited by 129 · see citing works
The incidence and predictors of depressive and anxiety symptoms in older adults with vision impairment: a longitudinal prospective cohort study
Heesterbeek, T. J., van der Aa, H. P. A., van Rens, G. H. M. B., Twisk, J. W. R., & van Nispen, R. M. A. (2017) · Ophthalmic and Physiological Optics · doi:10.1111/opo.12388 · Cited by 127 · see citing works
“42,43 Therefore, we chose to use zero imputation for the few items that were missing in the CES-D (n T0 = 26, n T1 = 29, n T2 = 26, n T3 = 24, n T4 = 40), HADS-A (n T0 = 0, n T1 = 5, n T2 = 10, n T3 = 4, n T4 = 6), LVQOL ( n = 61) and AVL ( n = 19) if there was at least an 80% completion.”
Self-citation
Effect of Spinal Cord Burst Stimulation vs Placebo Stimulation on Disability in Patients With Chronic Radicular Pain After Lumbar Spine Surgery
Hara, S., Andresen, H., Solheim, O. S., Carlsen, S. M., Sundstrøm, T., Lønne, G., Vangen-Lønne, V., Taraldsen, K., Tronvik, E., Øie, L. R., et al. (2022) · JAMA · doi:10.1001/jama.2022.18231 · Cited by 126 · see citing works
Effect on HBs antigen clearance of addition of pegylated interferon alfa-2a to nucleos(t)ide analogue therapy versus nucleos(t)ide analogue therapy alone in patients with HBe antigen-negative chronic hepatitis B and sustained undetectable plasma hepatitis B virus DNA: a randomised, controlled, open-label trial
Bourlière, M., Rabiéga, P., Ganne‐Carrié, N., Serfaty, L., Marcellin, P., Barthe, Y., Thabut, D., Guyader, D., Hézode, C., Picon, M., et al. (2017) · The Lancet. Gastroenterology & hepatology · doi:10.1016/s2468-1253(16)30189-3 · Cited by 121 · see citing works
“PEGIFN Control p-value Fisher's test N= 90 N= 93 AEs N (%) AEs N (%) All Adverse events 1018 85 (94) 195 74 (80) 0·0038 grade 1 613 4 (4) 130 35 (38) grade 2 319 36 (40) 54 30 (32) <0·0001 grade 3 61 26 (29) 5 3 (3) grade 4 25 19 (21) 6 6 (6) Events imputed to PEGIFN 562 76 (84) grade 1 352 15 (17) grade 2 163 31 (34)…”
Background
Effects of Physical Activity in Nursing Home Residents with Dementia: A Randomized Controlled Trial
Henskens, M., Nauta, I. M., van Eekeren, M. C. A., & Scherder, E. (2018) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000491818 · Cited by 114 · see citing works
Effectiveness of transcranial direct current stimulation preceding cognitive behavioural management for chronic low back pain: sham controlled double blinded randomised controlled trial
Luedtke, K., Rushton, A. B., Wright, C. C., Jurgens, T., Polzer, A., Mueller, G., & May, A. (2015) · BMJ · doi:10.1136/bmj.h1640 · Cited by 110 · see citing works
Early Hormonal Treatment Affects Body Composition and Body Shape in Young Transgender Adolescents
Klaver, M., de Mutsert, R., Wiepjes, C. M., Twisk, J. W. R., den Heijer, M., Rotteveel, J., & Klink, D. T. (2018) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2017.12.009 · Cited by 110 · see citing works
Self-citation
Higher Doses Improve Walking Recovery During Stroke Inpatient Rehabilitation
Klassen, T. D., Dukelow, S. P., Bayley, M. T., Benavente, O., Hill, M. D., Krassioukov, A. V., Liu-Ambrose, T. Y. L., Pooyania, S., Poulin, M. J., Schneeberg, A., et al. (2020) · Stroke · doi:10.1161/strokeaha.120.029245 · Cited by 109 · see citing works
Immunological effects of behavioral activation with exercise in major depression: an exploratory randomized controlled trial
Euteneuer, F., Dannehl, K., del Rey, A., Engler, H., Schedlowski, M., & Rief, W. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.76 · Cited by 106 · see citing works
A randomized controlled efficacy trial of mindfulness-based stress reduction compared with an active control group and usual care for fibromyalgia: The EUDAIMON study
Pérez-Aranda, A., Feliu-Soler, A., Montero-Marín, J., García-Campayo, J., Andrés-Rodríguez, L., Borràs, X., Rozadilla-Sacanell, A., Peñarrubia-Maria, M. T., Angarita-Osorio, N., McCracken, L. M., et al. (2019) · Pain · doi:10.1097/j.pain.0000000000001655 · Cited by 98 · see citing works
“A study clearly showed that for all types of missing data (missing completely at random, missing at random, and missing not at random), multiple imputation is not necessary before computing longitudinal mixed model analysis.(82) Regression coefficients (B) and 95% confidence intervals (95% CIs) were calculated for the…”
Method
Breast Development in Transwomen After 1 Year of Cross-Sex Hormone Therapy: Results of a Prospective Multicenter Study
de Blok, C. J. M., Klaver, M., Wiepjes, C. M., Nota, N. M., Heijboer, A. C., Fisher, A. D., Schreiner, T., T'Sjoen, G. G., & den Heijer, M. (2017) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2017-01927 · Cited by 95 · see citing works
Internet based vestibular rehabilitation with and without physiotherapy support for adults aged 50 and older with a chronic vestibular syndrome in general practice: three armed randomised controlled trial
van Vugt, V. A., van der Wouden, J. C., Essery, R., Yardley, L., Twisk, J. W. R., van der Horst, H. E., & Maarsingh, O. R. (2019) · BMJ · doi:10.1136/bmj.l5922 · Cited by 96 · see citing works
Self-citation
Long-term efficacy of a printed or a Web-based tailored physical activity intervention among older adults
Peels, D. A., Bolman, C. A. W., Golsteijn, R. H. J., de Vries, H., Mudde, A. N., van Stralen, M. M., & Lechner, L. (2013) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-10-104 · Cited by 89 · see citing works
“Several studies concluded that it was not necessary using multiple imputations before performing a multilevel analysis on longitudinal data [19,25].”
BackgroundMethod
Long-Term Results of a Web-Based Guided Self-Help Intervention for Employees With Depressive Symptoms: Randomized Controlled Trial
Geraedts, A. S., Kleiboer, A. M., Twisk, J., Wiezer, N. M., van Mechelen, W., & Cuijpers, P. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3539 · Cited by 89 · see citing works
“Because there is a current debate whether it is necessary to perform multiple imputations in combination with mixed-model analyses in longitudinal studies [65], we also performed the LMM analyses without multiple imputations.”
Self-citation
Effectiveness of a Multicomponent Treatment for Fibromyalgia Based on Pain Neuroscience Education, Exercise Therapy, Psychological Support, and Nature Exposure (NAT-FM): A Pragmatic Randomized Controlled Trial
Serrat, M., Almirall, M., Musté, M., Sanabria‐Mazo, J. P., Feliu‐Soler, A., Méndez-Ulrich, J. L., Luciano, J. V., & Sanz, A. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9103348 · Cited by 86 · see citing works
“Regression coefficients (B) of bias-corrected bootstrapped indirect effects were calculated, as well as the corresponding SEs and 95% CIs [68].”
Method
Using linear and natural cubic splines, SITAR, and latent trajectory models to characterise nonlinear longitudinal growth trajectories in cohort studies
Elhakeem, A., Hughes, R. A., Tilling, K. M., Cousminer, D. L., Jackowski, S. A., Cole, T. J., Kwong, A. S. F., Li, Z., Grant, S. F. A., Baxter-Jones, A. D. G., et al. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01542-8 · Cited by 85 · see citing works
Effect of vitamin D3 supplementation on vascular and metabolic health of vitamin D–deficient overweight and obese children: a randomized clinical trial
Rajakumar, K., Moore, C. G., Khalid, A. T., Vallejo, A. N., Virji, M. A., Holick, M. F., Greenspan, S. L., Arslanian, S. A., & Reís, S. E. (2019) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqz340 · Cited by 82 · see citing works
Multimodal physical therapy versus topical lidocaine for provoked vestibulodynia: a multicenter, randomized trial
Morin, M., Dumoulin, C. L., Bergeron, S., Mayrand, M., Khalifé, S., Waddell, G., Dubois, M., Girard, I., Bureau, Y.-A., Ouellet, S., et al. (2020) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2020.08.038 · Cited by 82 · see citing works
Positive and negative side effects of androgen abuse. The HAARLEM study: A one‐year prospective cohort study in 100 men
Smit, D. L., Buijs, M. M., de Hon, O. M., den Heijer, M., & de Ronde, W. (2020) · Scandinavian Journal of Medicine and Science in Sports · doi:10.1111/sms.13843 · Cited by 78 · see citing works
The impact of an 8-year mass drug administration programme on prevalence, intensity and co-infections of soil-transmitted helminthiases in Burundi
Ortu, G., Assoum, M., Wittmann, U., Knowles, S. C. L., Clements, M., Ndayishimiye, O., Basáñez, M.-G., Lau, C. L., Clements, A., Fenwick, A., et al. (2016) · Parasites & Vectors · doi:10.1186/s13071-016-1794-9 · Cited by 76 · see citing works
Effects of an arm‐support exoskeleton on perceived work intensity and musculoskeletal discomfort: An 18‐month field study in automotive assembly
Kim, S., Nussbaum, M. A., Smets, M., & Ranganathan, S. (2021) · American Journal of Industrial Medicine · doi:10.1002/ajim.23282 · Cited by 76 · see citing works
Long-Term Immune Reconstitution and Infection Burden after Mismatched Hematopoietic Stem Cell Transplantation
Servais, S., Lengliné, É., Porcher, R., Carmagnat, M., de Latour, R. P., Robin, M., de Fontbrune, F. S., Clave, E., Maki, G., Granier, C., et al. (2014) · Transplantation and Cellular Therapy · doi:10.1016/j.bbmt.2014.01.001 · Cited by 75 · see citing works
The effects of a moderate physical activity intervention on physical fitness and cognition in healthy elderly with low levels of physical activity: a randomized controlled trial
Galle, S. A., Deijen, J. B., Milders, M. V., Greef, M. H. G. D., Scherder, E., van Duijn, C. M., & Drent, M. L. (2023) · Alzheimer s Research & Therapy · doi:10.1186/s13195-022-01123-3 · Cited by 80 · see citing works
The statistical approach in trial-based economic evaluations matters: get your statistics together!
Mutubuki, E. N., Alili, M. E., Bosmans, J. E., Oosterhuis, T., Snoek, F. J., Ostelo, R., van Tulder, M. W., & van Dongen, J. M. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-06513-1 · Cited by 72 · see citing works
A multifaceted workplace intervention for low back pain in nurses' aides
Rasmussen, C. D. N., Holtermann, A., Bay, H., Søgaard, K., & Jørgensen, M. B. (2015) · Pain · doi:10.1097/j.pain.0000000000000234 · Cited by 69 · see citing works
“randomised participants without imputations because mixed models inherently account for missing values.(36)”
Background
Effectiveness of energy conservation management on fatigue and participation in multiple sclerosis: A randomized controlled trial
Blikman, L. J., van Meeteren, J., Twisk, J. W. R., de Laat, F. A., de Groot, V., Beckerman, H., Stam, H. J., Bussmann, J. B. J., & TREFAMS-ACE study group (2017) · Multiple Sclerosis Journal · doi:10.1177/1352458517702751 · Cited by 67 · see citing works
Self-citation
Health-Related Quality of Life Trajectories of Extremely Low Birth Weight Survivors into Adulthood
Saigal, S., Ferro, M. A., Lieshout, R. J. V., Schmidt, L. A., Morrison, K. M., & Boyle, M. H. (2016) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2016.08.018 · Cited by 66 · see citing works
Effect of Prehabilitation Before Total Knee Replacement for Knee Osteoarthritis on Functional Outcomes
Nguyen, C., Boutron, I., Rören, A., Anract, P., Beaudreuil, J., Biau, D. J., Boisgard, S., Daste, C., Durand‐Zaleski, I., Eschalier, B., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.1462 · Cited by 70 · see citing works
An Athletic Coach–Delivered Middle School Gender Violence Prevention Program
Miller, E. P., Jones, K. A., Ripper, L. M., Paglisotti, T., Mulbah, P., & Abebe, K. Z. (2020) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2019.5217 · Cited by 65 · see citing works
Treatment effects on compulsive exercise and physical activity in eating disorders
Mathisen, T. F., Bratland‐Sanda, S., Rosenvinge, J. H., Friborg, O., Pettersen, G., Vrabel, K. A., & Sundgot-Borgen, J. K. (2018) · Journal of Eating Disorders · doi:10.1186/s40337-018-0215-1 · Cited by 66 · see citing works
Imagery Rescripting as a stand-alone treatment for posttraumatic stress disorder related to childhood abuse: A randomized controlled trial
Raabe, S., Ehring, T., Marquenie, L., Arntz, A., & Kindt, M. (2022) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2022.101769 · Cited by 64 · see citing works
Clinical course and prognosis of musculoskeletal pain in patients referred for physiotherapy: does pain site matter?
Andersen, N.-B. D. V., Kent, P. M., Hjort, J., & Christiansen, D. H. (2017) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-017-1487-3 · Cited by 62 · see citing works
“Although we conducted analyses using all the available data by use of longitudinal regression modelling, which is robust to missing data [51, 52], this cannot preclude a selection bias.”
Method
Disruption and recovery of testicular function during and after androgen abuse: the HAARLEM study
Smit, D. L., Buijs, M. M., de Hon, O. M., den Heijer, M., & de Ronde, W. (2020) · Human Reproduction · doi:10.1093/humrep/deaa366 · Cited by 60 · see citing works
Developmental trajectories of behaviour problems and prosocial behaviours of children with intellectual disabilities in a population‐based cohort
Bailey, T., Totsika, V., Hastings, R. P., Hatton, C., & Emerson, E. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13080 · Cited by 59 · see citing works
Applied Mixed Model Analysis
Twisk, J. W. R. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108635660 · Cited by 59 · see citing works
Self-citation
Multiple imputation of missing covariate values in multilevel models with random slopes: a cautionary note
Grund, S., Lüdtke, O., & Robitzsch, A. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0590-3 · Cited by 59 · see citing works
Does aerobic exercise improve or impair cardiorespiratory fitness and health among cleaners? A cluster randomized controlled trial
Korshøj, M., Lidegaard, M., Skotte, J. H., Krustrup, P., Krause, N., Søgaard, K., & Holtermann, A. (2014) · Scandinavian Journal of Work Environment & Health · doi:10.5271/sjweh.3475 · Cited by 58 · see citing works
“Due to some differences in missing data between groups, at baseline and follow-up, the mixed-model analysis was applied, since it allows an intention-to-treat analysis with missing observations (62).”
Method
Impact of Hypertonic Saline Solutions on Sputum Expectoration and Their Safety Profile in Patients with Bronchiectasis: A Randomized Crossover Trial
Herrero‐Cortina, B., Alcaraz-Serrano, V., Vilaró, J., Torres, A., & Polverino, E. (2018) · Journal of Aerosol Medicine and Pulmonary Drug Delivery · doi:10.1089/jamp.2017.1443 · Cited by 57 · see citing works
Educating Dutch General Practitioners in Dementia Advance Care Planning: A Cluster Randomized Controlled Trial
Tilburgs, B., Koopmans, R. T. C. M., Vernooij-Dassen, M. J. F. J., Adang, E., Schers, H., Teerenstra, S., van de Pol, M. H. J., Smits, C. H. M., Engels, Y., & Perry, M. (2019) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2019.09.010 · Cited by 57 · see citing works
Intention-to-treat analysis when only a baseline value is available
Twisk, J. W. R., Rijnhart, J. J. M., Hoekstra, T., Schuster, N. A., ter Wee, M. M., & Heymans, M. W. (2020) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2020.100684 · Cited by 54 · see citing works
Self-citation
Effect of the dr. Bart application on healthcare use and clinical outcomes in people with osteoarthritis of the knee and/or hip in the Netherlands; a randomized controlled trial
Pelle, T., Bevers, K., van der Palen, J. A. M., van den Hoogen, F. H. J., & van den Ende, C. H. M. (2020) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2020.02.831 · Cited by 54 · see citing works
Is physical exercise and dietary therapy a feasible alternative to cognitive behavior therapy in treatment of eating disorders? A randomized controlled trial of two group therapies
Mathisen, T. F., Rosenvinge, J. H., Friborg, O., Vrabel, K. R., Bratland‐Sanda, S., Pettersen, G., & Sundgot‐Borgen, J. (2020) · International Journal of Eating Disorders · doi:10.1002/eat.23228 · Cited by 52 · see citing works
Does digital infrastructure improve public Health? A quasi-natural experiment based on China's Broadband policy
Liu, Y., Liu, K., Zhang, X., & Guo, Q. (2024) · Social Science & Medicine · doi:10.1016/j.socscimed.2024.116624 · Cited by 52 · see citing works
High-intensity versus low-intensity resistance training in patients with knee osteoarthritis: A randomized controlled trial
de Zwart, A. H., Dekker, J., Roorda, L. D., van der Esch, M., Lips, P. T. A. M., van Schoor, N. M., Heijboer, A. C., Turkstra, F., Gerritsen, M., HÄKKINEN, A. H., et al. (2022) · Clinical Rehabilitation · doi:10.1177/02692155211073039 · Cited by 51 · see citing works
“Mixed model analyses are a well-tested and adequate method to analyse datasets that include missing data.(30) Another limitation is that we had difficulty recruiting sufficient patients for our study with knee osteoarthritis and vitamin D deficiency.”
Method
Body image self-conscious emotions get worse throughout adolescence and relate to physical activity behavior in girls and boys.
Sabiston, C., Doré, I., Lucibello, K., Pila, E., Brunet, J., Thibault, V., & Bélanger, M. (2022) · Social Science & Medicine (1967) · doi:10.1016/j.socscimed.2022.115543 · Cited by 46 · see citing works
A multi-faceted workplace intervention targeting low back pain was effective for physical work demands and maladaptive pain behaviours, but not for work ability and sickness absence: Stepped wedge cluster randomised trial
Rasmussen, C. D. N., Holtermann, A., Jørgensen, M. B., Ørberg, A., Mortensen, O. S., & Søgaard, K. (2016) · Scandinavian Journal of Public Health · doi:10.1177/1403494816653668 · Cited by 51 · see citing works
“[22] Twisk J, de Boer m, de Vente W, et al.”
BackgroundMethod
Association of plasma YKL-40 with brain amyloid-β levels, memory performance, and sex in subjective memory complainers
Vergallo, A., Lista, S., Lemercier, P., Chiesa, P. A., Zetterberg, H., Blennow, K., Potier, M., Habert, M., Baldacci, F., Cavedo, E., et al. (2020) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2020.07.009 · Cited by 48 · see citing works
Procalcitonin biomarker kinetics fails to predict treatment response in perioperative abdominal infection with septic shock
Jung, B., Molinari, N., Nasri, M., Hajjej, Z., Chanques, G., Jean‐Pierre, H., Panaro, F., & Jaber, S. (2013) · Critical Care · doi:10.1186/cc13082 · Cited by 45 · see citing works
“Missing biomarker data were neither deleted nor replaced or imputed as missing data are handled by the mixed logistic regression [28].”
A systematic review finds variable use of the intention-to-treat principle in musculoskeletal randomized controlled trials with missing data
Joseph, R., Sim, J., Ogollah, R., & Lewis, M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.09.002 · Cited by 50 · see citing works
“Also, the use of MI before performing the MMRM is unnecessary, as there is no obvious gain from doing so [52].”
Method
Conducting Trial-Based Economic Evaluations Using R: A Tutorial
Ben, Â. J., van Dongen, J. M., Alili, M. E., Eßer, J., Broulíková, H. M., & Bosmans, J. E. (2023) · PharmacoEconomics · doi:10.1007/s40273-023-01301-7 · Cited by 48 · see citing works
“Moreover, despite MI being shown to be unnecessary when using likelihood-based models for analysing effects [31], for analysing costs MI has been shown to produce more accurate estimates [32].”
Method
Bone Mineral Density in Transgender Adolescents Treated With Puberty Suppression and Subsequent Gender-Affirming Hormones
van der Loos, M. A. T. C., Vlot, M. C., Klink, D. T., Hannema, S. E., den Heijer, M., & Wiepjes, C. M. (2023) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.4588 · Cited by 48 · see citing works
Enhanced Fuzzy Clustering for Incomplete Instance with Evidence Combination
Liu, Z., & Letchmunan, S. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3638061 · Cited by 47 · see citing works
“Interestingly, the literature [42] argued that multiple imputations are not always superior to single estimation in any case by analyzing a large number of data with mixed-model analyses.”
BackgroundMethod
Improving the Multidisciplinary Treatment of Chronic Pain by Stimulating Body Awareness
van der Maas, L. C. C., Köke, A. J. A., Pont, M., Bosscher, R. J., Twisk, J. W. R., Janssen, T. W. J., & Peters, M. L. (2014) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000000138 · Cited by 47 · see citing works
“It has been proven that this method is even better than multiple imputation techniques (65).”
MethodSelf-citation
Blended psychosocial support for partners of patients with ALS and PMA: results of a randomized controlled trial.
De Wit, J., Beelen, A., Drossaert, C. H. C., Kolijn, R., Van Den Berg, L. H., SchrÖder, C. D., & Visser-Meily, J. M. A. (2020) · Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration · doi:10.1080/21678421.2020.1757114 · Cited by 46 · see citing works
Association of Education and Intracranial Volume With Cognitive Trajectories and Mortality Rates Across the Alzheimer Disease Continuum
van Loenhoud, A. C., Groot, C., Bocancea, D. I., Barkhof, F., Teunissen, C. E., Scheltens, P., van der Flier, W. M., & Ossenkoppele, R. (2022) · Neurology · doi:10.1212/wnl.0000000000200116 · Cited by 46 · see citing works
“We did not conduct multiple imputation, as the linear mixed models described in our Statistical Analysis are capable of handling these types of variability.(26,27)”
Method
A systematic survey of the methods literature on the reporting quality and optimal methods of handling participants with missing outcome data for continuous outcomes in randomized controlled trials
Zhang, Y., Alyass, A., Vanniyasingam, T., Sadeghirad, B., Flórez, I. D., Pichika, S. C., Kennedy, S. A., Abdulkarimova, U., Zhang, Y., Iljon, T., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.016 · Cited by 46 · see citing works
Surgery for Degenerative Cervical Myelopathy: A Nationwide Registry-Based Observational Study With Patient-Reported Outcomes
Gulati, S., Vangen-Lønne, V., Nygaard, O. P., Gulati, A. M., Hammer, T. A., Johansen, T. O., Peul, W. C., Salvesen, Ø. O., & Solberg, T. K. (2021) · Neurosurgery · doi:10.1093/neuros/nyab259 · Cited by 48 · see citing works
Is sleep disturbance in patients with chronic pain affected by physical exercise or ACT-based stress management? - A randomized controlled study
Wiklund, T., Linton, S. J., Alföldi, P., & Gerdle, B. (2018) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-018-2020-z · Cited by 45 · see citing works
“Mixed model analysis handles missing data; no imputation method was applied since there are reports that imputations before a mixed model analysis is not necessary and simulations indicated that imputations before mixed models could result in unstable results [30].”
Method
Telehealth coaching to improve self-management for secondary prevention after stroke: A randomized controlled trial of Stroke Coach
Sakakibara, B. M., Lear, S. A., Barr, S. I., Goldsmith, C. H., Schneeberg, A., Silverberg, N. D., Yao, J. K., & Eng, J. J. (2021) · International Journal of Stroke · doi:10.1177/17474930211017699 · Cited by 48 · see citing works
Effect of tailored antibiotic stewardship programmes on the appropriateness of antibiotic prescribing in nursing homes
van Buul, L. W., van der Steen, J. T., Achterberg, W. P., Schellevis, F., Essink, R. T. G. M., de Greeff, S. C., Natsch, S., Sloane, P. D., Zimmerman, S. I., Twisk, J. W. R., et al. (2015) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkv051 · Cited by 45 · see citing works
Self-citation
Are People With Whiplash-Associated Neck Pain Different From People With Nonspecific Neck Pain?
Anstey, R., Kongsted, A., Kamper, S. J., & Hancock, M. J. (2016) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2016.6588 · Cited by 45 · see citing works
Explorative Prospective Evaluation of Short-Term Subjective Effects of Hormonal Treatment in Trans People—Results from the European Network for the Investigation of Gender Incongruence
van Dijk, D., Dekker, M. J. H. J., Conemans, E. B., Wiepjes, C. M., de Goeij, E. G. M., Overbeek, K. A., Fisher, A. D., den Heijer, M., & T'Sjoen, G. G. (2019) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2019.05.009 · Cited by 46 · see citing works
Long-term Immune Response to Hepatitis B Virus Vaccination Regimens in Adults With Human Immunodeficiency Virus 1
Launay, O., Rosenberg, A. R., Rey, D., Pouget, N., Michel, M., Reynes, J., Neau, D., Raffi, F., Piroth, L., Carrat, F., et al. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0741 · Cited by 44 · see citing works
“Nadir CD4 count/μL, median (range) 197 (0-800) 272 (3-908) 206 (3-754) 311 (39-779) 180 (0-601) 218 (13-534)”
Development of Hip Bone Geometry During Gender-Affirming Hormone Therapy in Transgender Adolescents Resembles That of the Experienced Gender When Pubertal Suspension Is Started in Early Puberty
van der Loos, M. A. T. C., Hellinga, I., Vlot, M. C., Klink, D. T., den Heijer, M., & Wiepjes, C. M. (2020) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.4262 · Cited by 44 · see citing works
“However, by analyzing the data with a mixed model, the missing data were randomly imputed.((19)) Therefore, the effect of missing data was minimalized while maintaining the largest study sample.”
Method
Transform-Us! cluster RCT: 18-month and 30-month effects on children’s physical activity, sedentary time and cardiometabolic risk markers
Salmon, J., Arundell, L., Cerin, E., Ridgers, N. D., Hesketh, K. D., Daly, R. M., Dunstan, D. W., Brown, H. L., Gatta, J. D., Gatta, P. A. D., et al. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2022-105825 · Cited by 43 · see citing works
The association between persistent cognitive difficulties and depression and functional outcomes in people with major depressive disorder
Matcham, F., Simblett, S. K., Leightley, D., Dalby, M., Siddi, S., Haro, J. M., Lamers, F., Penninx, B. W. J. H., Bruce, S., Nica, R. I., et al. (2022) · Psychological Medicine · doi:10.1017/s0033291722003671 · Cited by 43 · see citing works
Some data quality issues at ClinicalTrials.gov
Chaturvedi, N., Mehrotra, B., Kumari, S., Gupta, S., Subramanya, H. S., & Saberwal, G. (2019) · Trials · doi:10.1186/s13063-019-3408-2 · Cited by 43 · see citing works
Daily Exercises and Education for Preventing Low Back Pain in Children: Cluster Randomized Controlled Trial
Hill, J. J., & Keating, J. L. (2014) · Physical Therapy · doi:10.2522/ptj.20140273 · Cited by 43 · see citing works
Longitudinal Changes in Muscle Mass, Muscle Strength, and Physical Performance in Acutely Hospitalized Older Adults
Aarden, J. J., Reijnierse, E. M., van der Schaaf, M., van der Esch, M., Reichardt, L. A., van Seben, R., Bosch, J. A., Twisk, J. W. R., Maier, A. B., Engelbert, R. H. H., et al. (2021) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2020.12.006 · Cited by 43 · see citing works
Self-citation
Improvements in Blood Pressure Among Undiagnosed Hypertensive Participants in a Community-Based Lifestyle Intervention, Mississippi, 2010
Zoellner, J., Thomson, J. L., Landry, A. S., Anderson‐Lewis, C., Connell, C. L., Molaison, E. F., & Yadrick, K. (2014) · Preventing Chronic Disease · doi:10.5888/pcd11.130269 · Cited by 42 · see citing works
“Maximum likelihood estimation was used to handle missing data in the repeated measures (20).”
Web-Based Aftercare for Women With Bulimia Nervosa Following Inpatient Treatment: Randomized Controlled Efficacy Trial
Jacobi, C., Beintner, I., Fittig, E., Trockel, M., Braks, K., Schade‐Brittinger, C., & Dempfle, A. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7668 · Cited by 42 · see citing works
“Multiple imputation before conducting longitudinal mixed effects modeling does not add value and is not necessary [46,47].”
Effectiveness and cost-effectiveness of proactive and multidisciplinary integrated care for older people with complex problems in general practice: an individual participant data meta-analysis
Blom, J. W., van den Hout, W. B., den Elzen, W. P. J., Drewes, Y. M., Bleijenberg, N., Fabbricotti, I. N., Jansen, A. P. D., Kempen, G. I. J. M., Koopmans, R. T. C. M., Looman, W., et al. (2018) · Age and Ageing · doi:10.1093/ageing/afy091 · Cited by 42 · see citing works
Adherence to a Plant-Based Diet and Consumption of Specific Plant Foods—Associations with 3-Year Weight-Loss Maintenance and Cardiometabolic Risk Factors: A Secondary Analysis of the PREVIEW Intervention Study
Zhu, R., Fogelholm, M., Poppitt, S. D., Silvestre, M. P., Møller, G., Huttunen‐Lenz, M., Stratton, G., Sundvall, J., Råman, L., Jalo, E., et al. (2021) · Nutrients · doi:10.3390/nu13113916 · Cited by 42 · see citing works
“Missing data were not handled with multiple imputation because this method did not increase precision [28,29].”
Method
A scoping review of statistical methods for trial‐based economic evaluations: The current state of play
Alili, M. E., van Dongen, J. M., Eßer, J., Heymans, M. W., van Tulder, M. W., & Bosmans, J. E. (2022) · Health Economics · doi:10.1002/hec.4603 · Cited by 41 · see citing works
“First, studies have shown that multiple imputation is not required when analyzing partly missing, clinical effect data using a longitudinal mixed model analysis (Peters et al., 2012; Twisk et al., 2013).”
BackgroundSelf-citation
Effects of a school-based physical activity intervention on academic performance in 14-year old adolescents: a cluster randomized controlled trial – the School in Motion study
Solberg, R. B., Steene-Johannessen, J., Anderssen, S. A., Ekelund, U., Säfvenbom, R., Haugen, T., Berntsen, S., Åvitsland, A., Lerum, Ø., Resaland, G. K., et al. (2021) · BMC Public Health · doi:10.1186/s12889-021-10901-x · Cited by 37 · see citing works
“Multiple imputation when performing mixed models analyses can lead to unstable results [26], and when the analysis was rerun on the imputed data the estimates were attenuated and some no longer significant.”
Method
Efficacy of Videoconference Group Acceptance and Commitment Therapy (ACT) and Behavioral Activation Therapy for Depression (BATD) for Chronic Low Back Pain (CLBP) Plus Comorbid Depressive Symptoms: A Randomized Controlled Trial (IMPACT Study)
Sanabria-Mazo, J. P., Colomer-Carbonell, A., Borràs, X., Castaño-Asins, J. R., McCracken, L. M., Montero-Marin, J., Pérez-Aranda, A., Edo, S., Sanz, A., Feliu-Soler, A., et al. (2023) · The Journal of Pain · doi:10.1016/j.jpain.2023.04.008 · Cited by 40 · see citing works
An Internet-supported Physical Activity Intervention Delivered in Secondary Schools Located in Low Socio-economic Status Communities: Study Protocol for the Activity and Motivation in Physical Education (AMPED) Cluster Randomized Controlled Trial
Lonsdale, C., Lester, A., Owen, K., White, R. L., Moyes, I., Peralta, L. R., Kirwan, M., Maeder, A., Bennie, A., MacMillan, F., et al. (2015) · BMC Public Health · doi:10.1186/s12889-015-2583-7 · Cited by 43 · see citing works
Tic-Related Versus Tic-Free Obsessive-Compulsive Disorder
de Vries, F. E., Cath, D. C., Hoogendoorn, A. W., van Oppen, P., Glas, G. G., Veltman, D. J., van den Heuvel, O. A., & van Balkom, A. J. L. M. (2016) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.14m09736 · Cited by 41 · see citing works
Blood pressure and age-related GFR decline in the general population
Eriksen, B. O., Stefansson, V. T. N., Jenssen, T. G., Mathisen, U. D., Schei, J., Solbu, M. D., Wilsgaard, T., & Melsom, T. (2017) · BMC Nephrology · doi:10.1186/s12882-017-0496-7 · Cited by 41 · see citing works
“All of the subjects included in the baseline RENIS cohort were included in the linear mixed regression analyses regardless of whether they were examined at follow-up because linear mixed models allow for missing observations at one or more points in time [41, 42].”
Method
Effects of telephone-based health coaching on patient-reported outcomes and health behavior change: A randomized controlled trial
Dwinger, S., Rezvani, F., Kriston, L., Herbarth, L., Härter, M. C., & Dirmaier, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0236861 · Cited by 41 · see citing works
“If there were data missing due to lost to follow-up, there was no imputation done since mixed model analyses provide unbiased estimates under the assumption that data are missing at random conditional on the variables in the model [66].”
Method
Childhood trauma and attachment style predict the four-year course of obsessive compulsive disorder: Findings from the Netherlands obsessive compulsive disorder study.
Lee, T.-B., van Oppen, P., van Balkom, A. V., Eikelenboom, M., Hendriks, G., & Anholt, G. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.12.028 · Cited by 33 · see citing works
Just as Tall on Testosterone; a Neutral to Positive Effect on Adult Height of GnRHa and Testosterone in Trans Boys
Willemsen, L. A., Boogers, L. S., Wiepjes, C. M., Klink, D. T., van Trotsenburg, A. S. P., den Heijer, M., & Hannema, S. E. (2022) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgac571 · Cited by 39 · see citing works
Negative and positive affect regulation in a transdiagnostic internet-based protocol for emotional disorders: Randomized controlled trial
Díaz-García, A., González-Robles, A., García-Palacios, A., Fernández-Álvarez, J., Castilla, D., Bretón, J. M., Baños, R. M., Quero, S., & Botella, C. (2021) · Journal of Medical Internet Research · doi:10.2196/21335 · Cited by 36 · see citing works
“Therefore, patterns of missingness were not found to be MNAR and the decision to continue the analysis with the available data was made [84,85].”
Background
A compassion-based program to reduce psychological distress in medical students: A pilot randomized clinical trial
Rojas, B., Catalan, E., Diez, G., & Roca, P. (2023) · PLOS ONE · doi:10.1371/journal.pone.0287388 · Cited by 38 · see citing works
“Given that imputation of missing values is not necessary before performing longitudinal mixed-model analysis [48], we did not impute the missing data.”
Method
Effectiveness of Resource Groups for Improving Empowerment, Quality of Life, and Functioning of People With Severe Mental Illness
Tjaden, C. D., Mulder, C. L., den Hollander, W., Castelein, S., Delespaul, P., Keet, R., van Weeghel, J., & Kroon, H. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.2880 · Cited by 36 · see citing works
Protocol for a cluster randomised controlled trial of an intervention to improve the mental health support and training available to secondary school teachers – the WISE (Wellbeing in Secondary Education) study
Kidger, J., Evans, R., Tilling, K. M., Hollingworth, W., Campbell, R. M., Ford, T. J., Murphy, S., Araya, R., Morris, R. W., Kadir, B., et al. (2016) · BMC Public Health · doi:10.1186/s12889-016-3756-8 · Cited by 39 · see citing works
Efficacy of a mindful-eating programme to reduce emotional eating in patients suffering from overweight or obesity in primary care settings: a cluster-randomised trial protocol
Morillo‐Sarto, H., Barceló‐Soler, A., Herrera‐Mercadal, P., Pantilie, B., Navarro‐Gil, M., García‐Campayo, J., & Montero‐Marín, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-031327 · Cited by 38 · see citing works
Efficacy of an internet-based exposure treatment for flying phobia (NO-FEAR Airlines) with and without therapist guidance: a randomized controlled trial
Campos, D., Bretón-López, J., Botella, C., Mira, A., Castilla, D., Mor, S., Baños, R., & Quero, S. (2019) · BMC Psychiatry · doi:10.1186/s12888-019-2060-4 · Cited by 32 · see citing works
Resistance band training or general exercise in multidisciplinary rehabilitation of low back pain? A randomized trial
Iversen, V. M., Vasseljen, O., Mork, P. J., Gismervik, S., Bertheussen, G. F., Salvesen, Ø. O., & Fimland, M. S. (2018) · Scandinavian Journal of Medicine and Science in Sports · doi:10.1111/sms.13091 · Cited by 37 · see citing works
Effectiveness of Later-Stage Exercise Programs vs Usual Medical Care on Physical Function and Activity After Total Knee Replacement
Piva, S. R., Schneider, M., Moore, C. G., Catelani, M. B., Gil, A. B., Klatt, B. A., DiGioia, A. M., Almeida, G. J., Khoja, S. S., Sowa, G., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.0018 · Cited by 35 · see citing works
Impact of MyDiabetesPlan, a Web-Based Patient Decision Aid on Decisional Conflict, Diabetes Distress, Quality of Life, and Chronic Illness Care in Patients With Diabetes: Cluster Randomized Controlled Trial
Yu, C. H. Y., Choi, D., Bruno, B. A., Thorpe, K. E., Straus, S. E., Cantarutti, P., Chu, K., Frydrych, P., Hoang‐Kim, A., Ivers, N. M., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/16984 · Cited by 36 · see citing works
“To account for missing data, we conducted a fully adjusted mixed effect model using repeated measurements [40-42].”
Method
Long-Term Effects on Loneliness of a Computer-Tailored Intervention for Older Adults With Chronic Diseases: A Randomized Controlled Trial
Boekhout, J. M., Volders, E., Bolman, C. A. W., de Groot, R. H. M., & Lechner, L. (2021) · Journal of Aging and Health · doi:10.1177/08982643211015027 · Cited by 34 · see citing works
“Analyses were conducted on an intention-to-treat basis without any ad hoc imputation (Twisk et al., 2013).”
Method
Patient-reported Quality of Life in Patients with Primary Metastatic Prostate Cancer Treated with Androgen Deprivation Therapy with and Without Concurrent Radiation Therapy to the Prostate in a Prospective Randomised Clinical Trial; Data from the HORRAD Trial
Boevé, L. M. S., Hulshof, M. C. C. M., Verhagen, P. C. M. S., Twisk, J. W. R., Witjes, W. P. J., de Vries, P. W., van Moorselaar, R. J. A., van Andel, G., & Vis, A. N. (2020) · European Urology · doi:10.1016/j.eururo.2020.08.023 · Cited by 34 · see citing works
Self-citation

Authors: Twisk, J., de Boer, M., de Vente, W., & Heymans, M.
Affiliations: Vrije Universiteit Amsterdam (NL)
Corresponding author: Twisk, Jos
ORCID: Twisk, Jos
Heymans, Martijn
Venue: Journal of Clinical Epidemiology
Publisher: Elsevier BV
Published: 2013-09
Volume: 66
Issue: 9
Pages: 1022-1028
ISSN: 0895-4356 (ISSN-L)
DOI: 10.1016/j.jclinepi.2013.03.017
OpenAlex: W1974992958
Cited by: 568
References: 43
Citations by year: 2026: 58 · 2025: 68 · 2024: 74 · 2023: 57 · 2022: 61 · 2021: 52 · 2020: 46 · 2019: 45 · 2018: 27 · 2017: 26 · 2016: 21 · 2015: 19 · 2014: 10 · 2013: 4
FWCI: 15.69
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Journal of Clinical Epidemiology (publishedVersion)
PubMed (publishedVersion)
UvA-DARE (University of Amsterdam) (publishedVersion)
Pure Amsterdam UMC (publishedVersion)
Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
VU Research Portal (publishedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: multiple imputation, missing data, longitudinal analysis, mixed models, missing completely at random, missing at random, missing not at random, time-varying covariates
MeSH terms: Data Interpretation, Statistical, Humans, Longitudinal Studies, Patient Dropouts, Research Design, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multiple imputation of missing values was not necessary before performing a longitudinal mixed-model analysis

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks