Cited by 152Statistical Methods and Bayesian InferenceStatistical Methods and ApplicationsMeta-analysis and systematic reviews

The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.

Analysis of Variance of Multiply Imputed Data

Open access

van Ginkel, Joost R., Kroonenberg, Pieter M.·2014·Multivariate Behavioral Research

DOI 10.1080/00273171.2013.855890PMC PMC4029775

Open this paper in CoChat — free account

Abstract

As a procedure for handling missing data, Multiple imputation consists of estimating the missing data multiple times to create several complete versions of an incomplete data set. All these data sets are analyzed by the same statistical procedure, and the results are pooled for interpretation. So far, no explicit rules for pooling F tests of (repeated-measures) analysis of variance have been defined. In this article we outline the appropriate procedure for the results of analysis of variance (ANOVA) for multiply imputed data sets. It involves both reformulation of the ANOVA model as a regression model using effect coding of the predictors and applying already existing combination rules for regression models. The proposed procedure is illustrated using 3 example data sets. The pooled results of these 3 examples provide plausible F and p values.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1080/00273171.2013.855890

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
mice: Multivariate Imputation by Chained Equations in R
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870
Inference and missing data
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,006
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627
Journal of Statistical Software
de Leeuw, J. (2009) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.10 · Cited by 3,651
Multilevel Analysis
Hox, J., Moerbeek, M., & van de Schoot, R. (2002) · doi:10.4324/9781410604118 · Cited by 2,419

Show all 33 references

Research works printed but not linked: 4

Dunn , L. M. and Dunn , L. M. 1981 . Peabody Picture Test – Revised manual for Forms L and M , Circle Pines, MN : American Guidance Service .
Snijders , T. and Bosker , R. 1999 . Multilevel analysis , London , UK : Sage .
Winer , B. J. 1971 . Statistical principles in experimental designs, 2nd ed. , New York, NY : McGraw-Hill .
Van Ginkel , J. R. and Kroonenberg , P. M. 2011, July . Multiple imputation and (repeated-measures) analysis of variance , Hong Kong : Presentation given at the 17th international meeting of the Psychometric Society .

Sorted by citations

Rebutting Existing Misconceptions About Multiple Imputation as a Method for Handling Missing Data
van Ginkel, J. R., Linting, M., Rippe, R. C. A., & van der Voort, A. (2019) · Journal of Personality Assessment · doi:10.1080/00223891.2018.1530680 · Cited by 553
“…are less user-friendly than the automatic pooling by SPSS. Additionally, the paper promoting the SPSS macro for pooling the results of ANOVA (Van Ginkel & Kroonenberg, 2014a) has been misquoted by many authors, suggesting that combination rules for ANOVA are either not available or too…”
MethodInfluential
It's time: A meta-analysis on the self-control-deviance link
Vazsonyi, A. T., Mikuška, J., & Kelley, E. L. (2017) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2016.10.001 · Cited by 504
PLAY Project Home Consultation Intervention Program for Young Children With Autism Spectrum Disorders
Solomon, R., Egeren, L. A. V., Mahoney, G. J., Huber, M. S. Q., & Zimmerman, P. (2014) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0000000000000096 · Cited by 208
Electrical fingerprint of the amygdala guides neurofeedback training for stress resilience
Keynan, J. N., Cohen, A., Jackont, G., Green, N., Goldway, N., Davidov, A., Meir-Hasson, Y., Raz, G., Intrator, N., Fruchter, E., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0484-3 · Cited by 158
Characterizing gender differences in nonsuicidal self-injury: Evidence from a large clinical sample of adolescents and adults
Victor, S. E., Muehlenkamp, J. J., Hayes, N. A., Lengel, G. J., Styer, D. M., & Washburn, J. J. (2018) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2018.01.009 · Cited by 146
“48. van Ginkel JR, Kroonenberg PM.”
MethodInfluential
The contemporary face of gender differences and similarities in depression throughout adolescence: Development and chronicity
Salk, R. H., Petersen, J. L., Abramson, L. Y., & Hyde, J. S. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.03.071 · Cited by 134
“…we should first test the between-subjects effect of gender in a repeated measures MANOVA; however, formulas for pooling F-tests of analysis of variance for multiply imputed data sets (which would be necessary with our longitudinal data) are still in development (van Ginkel & Kroonenberg, 2014).”
Method
Rethinking interhemispheric imbalance as a target for stroke neurorehabilitation
Xu, J., Branscheidt, M., Schambra, H. M., Steiner, L., Widmer, M., Diedrichsen, J., Goldsmith, J., Lindquist, M. A., Kitago, T., Luft, A. R., et al. (2019) · Annals of Neurology · doi:10.1002/ana.25452 · Cited by 131
“22)  CS Stimulus Intensity (% MSO) Patient 57 (17) 57 (15) 53 (13) 55 (15) 55 (16) 0.”
Music Therapy Enhances Executive Functions and Prefrontal Structural Neuroplasticity after Traumatic Brain Injury: Evidence from a Randomized Controlled Trial
Siponkoski, S., Martínez‐Molina, N., Kuusela, L., Laitinen, S., Holma, M., Ahlfors, M., Jordan-Kilkki, P., Ala-Kauhaluoma, K., Melkas, S., Pekkola, J., et al. (2019) · Journal of Neurotrauma · doi:10.1089/neu.2019.6413 · Cited by 137
“91 This procedure is equivalent to a repeated measures (mixed-model) ANOVA.”
A randomized controlled trial comparing two cognitive-behavioral programs for adolescent girls with subclinical depression: A school-based program (Op Volle Kracht) and a computerized program (SPARX)
Poppelaars, M., Tak, Y. R., Lichtwarck‐Aschoff, A., Engels, R. C. M. E., Lobel, A., Merry, S., Lucassen, M. F. G., & Granic, I. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.03.005 · Cited by 114
“W M., Van Vlokhoven, B. S., Rensink, H. F. W., & Engels, R. C. M. E. (2012).”
Influential
An Internet-Based Intervention for Chronic Pain
Lin, J., Paganini, S., Sander, L., Lüking, M., Ebert, D. D., Buhrman, M., Andersson, G., & Baumeister, H. (2017) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2017.0681 · Cited by 110
Effects of Meditation versus Music Listening on Perceived Stress, Mood, Sleep, and Quality of Life in Adults with Early Memory Loss: A Pilot Randomized Controlled Trial
Innes, K. E., Selfe, T. K., Khalsa, D. S., & Kandati, S. (2016) · Journal of Alzheimer s Disease · doi:10.3233/jad-151106 · Cited by 114
“We used multiple imputation to replace any missing data in our intention-to-treat (ITT) analyses [106, 107].”
Analysing data from the psycholinguistic visual-world paradigm: Comparison of different analysis methods
Ito, A., & Knoeferle, P. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01969-3 · Cited by 119
“Note, however, that there are ways of analysing dependent variables with missing values using an ANOVA (e.g., van Ginkel & Kroonenberg, 2014).”
Method
Methods for significance testing of categorical covariates in logistic regression models after multiple imputation: power and applicability analysis
Eekhout, I. E., van de Wiel, M. A., & Heymans, M. W. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0404-7 · Cited by 106
“Van Ginkel and Kroonenberg developed a method to pool the F-tests of an ANOVA, but this procedure is rather complicated and not available in all software packages [21].”
Method
Overcoming Disembodiment: The Effect of Movement Therapy on Negative Symptoms in Schizophrenia—A Multicenter Randomized Controlled Trial
Martin, L. A. L., Koch, S. C., Hirjak, D., & Fuchs, T. A. E. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00483 · Cited by 104
“Keywords: schizophrenia, negative symptoms, embodiment/disembodiment, Dance Movement Therapy (DMT), Body Psychotherapy (BPT), randomized controlled trial, multiple imputation”
Background
A Pilot RCT of Psychodynamic Group Art Therapy for Patients in Acute Psychotic Episodes: Feasibility, Impact on Symptoms and Mentalising Capacity
Montag, C., Haase, L., Seidel, D., Bayerl, M., Gallinat, J., Herrmann, U., & Dannecker, K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112348 · Cited by 97
Major depressive disorder with melancholia displays robust alterations in resting state heart rate and its variability: implications for future morbidity and mortality
Kemp, A. H., Quintana, D. S., Quinn, C. R., Hopkinson, P., & Harris, A. W. F. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01387 · Cited by 95
“All statistical analyses were performed using IBM SPSS Statistics, Version 21 with SPSS R Essentials plug-in and R statistics version 2.14.2.”
BackgroundMethodInfluential
Meditation and Music Improve Memory and Cognitive Function in Adults with Subjective Cognitive Decline: A Pilot Randomized Controlled Trial
Innes, K. E., Selfe, T. K., Khalsa, D. S., & Kandati, S. (2017) · Journal of Alzheimer s Disease · doi:10.3233/jad-160867 · Cited by 99
“In our intention-to-treat (ITT) analyses, we used the SPSS multiple imputation function (10 iterations) to replace missing data [60].”
Aerobic, resistance, or combined exercise training and cardiovascular risk profile in overweight or obese adults: the CardioRACE trial
Lee, D.-C., Brellenthin, A. G., Lanningham-Foster, L. M., Kohut, M. L., & Li, Y. (2024) · European Heart Journal · doi:10.1093/eurheartj/ehad827 · Cited by 108
“26 We used linear mixed effects models with repeated measures including both baseline and follow-up data.”
Method
An Internet-Based Cognitive Behavioral Therapy Program Adapted to Patients With Cardiovascular Disease and Depression: Randomized Controlled Trial
Johansson, P., Westas, M., Andersson, G., Alehagen, U., Broström, A., Jaarsma, T., Mourad, G., & Lundgren, J. (2019) · JMIR Mental Health · doi:10.2196/14648 · Cited by 77
Significance Tests and Estimates for R2 for Multiple Regression in Multiply Imputed Datasets: A Cautionary Note on Earlier Findings, and Alternative Solutions
van Ginkel, J. R. (2019) · Multivariate Behavioral Research · doi:10.1080/00273171.2018.1540967 · Cited by 73
“Some work (Van Ginkel & Kroonenberg, 2014) suggests that for large percentages of missingness and small M, Rt2 may become too low for what is considered a reasonable estimate of the error degrees of freedom.”
BackgroundMethodResultInfluential
A Randomized-Controlled Trial of the Triple P-Positive Parenting Program Seminar Series with Indonesian Parents
Sumargi, A. M., Sofronoff, K., & Morawska, A. (2014) · Child Psychiatry & Human Development · doi:10.1007/s10578-014-0517-8 · Cited by 69
“It should be noted that the results of MANOVAs were reported in the ranges of F tests because the procedure to pool the results have not yet developed (JR van Ginkel, personal communication, September 29, 2014).”
Method
Comorbidity of Arithmetic and Reading Disorder
Raddatz, J., Kuhn, J., Holling, H., Moll, K., & Dobel, C. (2016) · Journal of Learning Disabilities · doi:10.1177/0022219415620899 · Cited by 70
“We therefore conducted multiple imputation ANOVAs (cf., van Ginkel & Kroonenberg, 2014) pooling analysis results over 10 imputed data sets, using all basic numerical tasks as well as demographic variables (age, gender, IQ) as covariates.”
Method
Cognitive bias modification and CBT as early interventions for adolescent social and test anxiety: two-year follow-up of a randomized controlled trial
de Hullu, E., Sportel, B. E., Nauta, M. H., & de Jong, P. J. (2017) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2016.11.011 · Cited by 63
“Since SPSS does not provide the F-statistics on the pooled data, we used the combination rules as described by van Ginkel and Kroonenberg (2014), using a SPSS macro (Van Ginkel, 2014).”
Method
Teaching College in the Time of COVID-19: Gender and Race Differences in Faculty Emotional Labor
Berheide, C. W., Carpenter, M. A., & Cotter, D. A. (2022) · Sex Roles · doi:10.1007/s11199-021-01271-0 · Cited by 65
“Separate univariate ANOVAs revealed a significant effect of intersectional identity on self-directed emotional labor (F(3, 165) = 3.77, p = .01; partial η2 = .06).”
MethodInfluential
Exposure to polychlorinated biphenyls and organochlorine pesticides and risk of dementia, Alzheimer’s disease and cognitive decline in an older population: a prospective analysis from the Canadian Study of Health and Aging
Medehouenou, T. C. M., Ayotte, P., Carmichael, P., Kröger, E., Verreault, R., Lindsay, J., Dewailly, É., Tyas, S. L., Bureau, A., & Laurin, D. (2019) · Environmental Health · doi:10.1186/s12940-019-0494-2 · Cited by 63
Effects of a Brief Mindfulness Curriculum on Self-reported Executive Functioning and Emotion Regulation in Hong Kong Adolescents
Lam, K. N., & Seiden, D. Y. (2019) · Mindfulness · doi:10.1007/s12671-019-01257-w · Cited by 69
A Randomized Controlled Trial in Routine Clinical Practice Comparing Acceptance and Commitment Therapy with Cognitive Behavioral Therapy for the Treatment of Major Depressive Disorder
A-Tjak, J. G. L., Morina, N., Topper, M., & Emmelkamp, P. M. G. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000486807 · Cited by 61
Children with Sexual Behavior Problems: Clinical Characteristics and Relationship to Child Maltreatment
Allen, B. (2016) · Child Psychiatry & Human Development · doi:10.1007/s10578-016-0633-8 · Cited by 60
“This method is equivalent to computing a series of analyses of covariance (ANCOVAs); however, using ANCOVA with multiple imputation is difficult as rules for pooling ANOVA-based results across datasets are not available [42].”
Method
Cognitive effects of music and dance training in children.
LaPlume, A. A., & Wiseheart, M. (2018) · Archives of Scientific Psychology · doi:10.1037/arc0000048 · Cited by 68
Dialectical behavior therapy self‐help for binge‐eating disorder: A randomized controlled study
Carter, J. C., Kenny, T. E., Singleton, C., Wijk, M. V., & Heath, O. J. (2019) · International Journal of Eating Disorders · doi:10.1002/eat.23208 · Cited by 55
Discrimination Matters: Relations of Perceived Discrimination to Student Mental Health
Weeks, M. R., & Sullivan, A. L. (2019) · School Mental Health · doi:10.1007/s12310-019-09309-1 · Cited by 53
Effects of Dance Movement Therapy on Adult Patients with Autism Spectrum Disorder: A Randomized Controlled Trial
Mastrominico, A., Fuchs, T. A. E., Manders, E. J., Steffinger, L., Hirjak, D., Sieber, M., Thomas, E., Holzinger, A., Konrad, A., Bopp, N., et al. (2018) · Behavioral Sciences · doi:10.3390/bs8070061 · Cited by 60
Differences in university students’ motivation between a required and an elective physical activity education policy
Kim, M. S., & Cardinal, B. J. (2018) · Journal of American College Health · doi:10.1080/07448481.2018.1469501 · Cited by 55
Are risk factors for preterm and early-term live singleton birth the same? A population-based study in France
Delnord, M., Blondel, B., Prunet, C., & Zeitlin, J. A. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-018745 · Cited by 44
Key Aspects of Prognostic Model Development and Interpretation From a Clinical Perspective
Hoesseini, A., van Leeuwen, N., Sewnaik, A., Steyerberg, E. W., de Jong, R. J. B., Lingsma, H. F., & Offerman, M. P. J. (2021) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2021.3505 · Cited by 42
Adaptive meta-learning extreme learning machine with golden eagle optimization and logistic map for forecasting the incomplete data of solar irradiance
Boriratrit, S., Fuangfoo, P., Srithapon, C., & Chatthaworn, R. (2023) · Energy and AI · doi:10.1016/j.egyai.2023.100243 · Cited by 44
Effects of Mantra Meditation versus Music Listening on Knee Pain, Function, and Related Outcomes in Older Adults with Knee Osteoarthritis: An Exploratory Randomized Clinical Trial (RCT)
Innes, K. E., Selfe, T. K., Kandati, S., Wen, S., & Huysmans, Z. (2018) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2018/7683897 · Cited by 46
A Randomized Controlled Trial of Group Triple P With Chinese Parents in Mainland China
Guo, M., Morawska, A., & Sanders, M. R. (2016) · Behavior Modification · doi:10.1177/0145445516644221 · Cited by 38
“The approach developed by van Ginkel et al. (van Ginkel, 2014; van Ginkel & Kroonenberg, 2014) was used to pool the results of two-way repeated measures ANOVAs and one-way repeated measures ANOVAs.”
Method
A randomized clinical trial of motivational enhancement therapy for alcohol problems in partner violent men
Murphy, C. M., Ting, L. A., Jordan, L. C., Musser, P. H., Winters, J. J., Poole, G. M., & Pitts, S. C. (2018) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2018.03.004 · Cited by 37
Lay-worker Delivered Home Visiting Promotes Early Childhood Development and Reduces Violence in Rwanda: A Randomized Pilot
Barnhart, D. A., Farrar, J., Murray, S. M., Brennan, R. T., Antonaccio, C. M., Sezibera, V., Ingabire, C., Godfroid, K., Bazubagira, S. M., Uwimana, O., et al. (2020) · Journal of Child and Family Studies · doi:10.1007/s10826-020-01709-1 · Cited by 34
Effects of a 12-week yoga versus a 12-week educational film intervention on symptoms of restless legs syndrome and related outcomes: an exploratory randomized controlled trial
Innes, K. E., Selfe, T. K., Montgomery, C., Hollingshead, N. A., Huysmans, Z., Srinivasan, R., Wen, S., Hausmann, M. J., Sherman, K. J., & Klatt, M. D. (2020) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.8134 · Cited by 32
“In our intention-to-treat analyses, we used the SPSS multiple imputation function (10 iterations) to handle missing data.(74) As this was a pilot feasibility study, alphawas set at 0.”
Dentoskeletal comparison of miniscrew-anchored maxillary protraction with hybrid and conventional hyrax expanders: A randomized clinical trial
Miranda, F., Bastos, J. C. D. C., dos Santos, A. M., Janson, G., Lauris, J. R. P., & Garib, D. G. (2021) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2021.02.017 · Cited by 32
A clinical effectiveness trial comparing ACT and CBT for inpatients with depressive and mixed mental disorders
Samaan, M., Diefenbacher, A., Schade, C., Dambacher, C., Pontow, I.-M., Pakenham, K., & Fydrich, T. (2021) · Psychotherapy Research · doi:10.1080/10503307.2020.1802080 · Cited by 31
“This enables use of the multiple imputation technique for missing data estimation with the ITT sample (van Ginkel & Kroonenberg, 2014).”
Method
Revisiting the immigrant paradox: Suicidal ideations and suicide attempts among immigrant and non‐immigrant adolescents
Vazsonyi, A. T., Mikuška, J., & Gaššová, Z. (2017) · Journal of Adolescence · doi:10.1016/j.adolescence.2017.05.008 · Cited by 29
Impairments in cognitive empathy and alexithymia occur independently of executive functioning in college students with autism
Ziermans, T., de Bruijn, Y., Dijkhuis, R., Staal, W., & Swaab, H. (2019) · Autism · doi:10.1177/1362361318817716 · Cited by 28
“…(MICE; Azur et al., 2011) was applied for multivariate missing data with the predictive mean matching approach for nonnormally distributed data (Van Ginkel and Kroonenberg, 2014) Group characteristics were tested for differences with chisquare tests (sex) and independent samples t tests (age, IQ).”
MethodInfluential
Dissecting the Genetic Architecture of Shoot Growth in Carrot ( Daucus carota L.) Using a Diallel Mating Design
Turner, S., Maurizio, P. L., Valdar, W. S. J., Yandell, B. S., & Simon, P. W. (2017) · G3 Genes Genomes Genetics · doi:10.1534/g3.117.300235 · Cited by 25
The psychological effects of physical exercise: A controlled study of the placebo effect
Arbinaga, F., Ozcorta, E. J. F., Buñuel, P. S.-L., & Carmona, J. (2018) · Scandinavian Journal of Psychology · doi:10.1111/sjop.12482 · Cited by 27
Thought-shape fusion in young healthy females appears after vivid imagination of thin ideals.
Wyssen, A., Coelho, J. S., Wilhelm, P., Zimmermann, G., & Munsch, S. (2016) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2016.03.010 · Cited by 25
What If I Appear Boring, Anxious, or Unattractive? Validation and Treatment Sensitivity of the Negative Self Portrayal Scale in Clinical Samples
Moscovitch, D. A., Rowa, K., Paulitzki, J. R., Antony, M. M., & McCabe, R. E. (2015) · Cognitive Therapy and Research · doi:10.1007/s10608-014-9645-5 · Cited by 22
“A total of 20 datasets were generated during the imputation process and analysis results were combined, where possible, in the manner described by Rubin (1987) using SPSS. Analysis of variance (ANOVA) results were pooled using the procedure described by Van Ginkel and Kroonenberg (2014).”
Method
A psycho‐oncological online intervention supporting partners of patients with cancer (PartnerCARE): Results from a randomized controlled feasibility trial
Bodschwinna, D., Lorenz, I., Bauereiß, N., Gündel, H., Baumeister, H., & Hönig, K. (2022) · Psycho-Oncology · doi:10.1002/pon.5917 · Cited by 20
“Since SPSS 26 does not run this analysis on Multivariate imputation data, a SPSS macro was used.(42,43) Significant results were reported using Cohen's d along with the corresponding 95% CI.”
Method
Implications of a “Null” Randomized Controlled Trial of Mindfulness and Compassion Interventions in Healthy Adults
Kaplan, D. M., Mehl, M. R., Pace, T. W. W., Negi, L. T., Silva, B. O.-D., Dodson‐Lavelle, B., Sivilli, T., Williams, A., Comstock, T., Price, B. E., et al. (2022) · Mindfulness · doi:10.1007/s12671-022-01861-3 · Cited by 18
“Missing data were handled through an intent-totreat approach using Multiple Imputation (van Ginkel & Kroonenberg, 2014).”
Method
Segregation paradox? School racial/ethnic and socioeconomic composition and racial/ethnic differences in engagement
Ackert, E. (2017) · Social Science Research · doi:10.1016/j.ssresearch.2017.10.010 · Cited by 19
The professional student program for educational resilience: Enhancing momentary engagement in classwork
Torsney, B. M., & Symonds, J. E. (2019) · The Journal of Educational Research · doi:10.1080/00220671.2019.1687414 · Cited by 23
“The five datasets imputation option is the default option on SPSS, and according to van Ginkel and Kroonenberg (2014) is a sound method for the purpose of this analysis.”
Method
Selective vs Routine Cholangiography Across a Health Care Enterprise
Gross, A. R., Said, S. A., Wehrle, C. J., Hong, H., Quick, J. D., Larson, S. L., Hossain, M. S., Naffouje, S. A., Walsh, R. M., & Augustin, T. (2024) · JAMA Surgery · doi:10.1001/jamasurg.2024.5216 · Cited by 18
Retracted
Priorities for Mediterranean marine turtle conservation and management in the face of climate change
Mazaris, A. D., Dimitriadis, C., Papazekou, M., Schofield, G., Doxa, A., Chatzimentor, A., Türkozan, O., Katsanevakis, S., Lioliou, A., Abalo‐Morla, S., et al. (2023) · Journal of Environmental Management · doi:10.1016/j.jenvman.2023.117805 · Cited by 20
A cohort study comparing the effects of medical cannabis for anxiety patients with and without comorbid sleep disturbance
Murphy, M., Erridge, S., Holvey, C., Coomber, R., Rucker, J., & Sodergren, M. H. (2023) · Neuropsychopharmacology Reports · doi:10.1002/npr2.12407 · Cited by 17
“73,74 A repeated measures one-way analysis of variance (ANOVA) was used to compare the mean difference between patients for each PROM at each time point.”
Method
Delivering Mindfulness-Based Interventions for Insomnia, Pain, and Dysfunctional Eating Through a Text Messaging App: Three Randomized Controlled Trials Investigating the Effectiveness and Mediating Mechanisms.
Li, A. C., Wong, K. K., Chio, F. H., Mak, W. W., & Poon, L. W. (2022) · Journal of Medical Internet Research · doi:10.2196/30073 · Cited by 17
“Missing data were treated using multiple imputations, and 100 imputed data sets were generated. van Ginkel and Kroonenberg’s [76] method of repeated measures analysis of variance (ANOVA) using imputed data was used in this study.”
Incorporating a Usual Care Comparator into a Study of Meditation and Music Listening for Older Adults with Subjective Cognitive Decline: A Randomized Feasibility Trial
Innes, K. E., Montgomery, C., Selfe, T. K., Wen, S., Khalsa, D. S., & Flick, M. (2021) · Journal of Alzheimer s Disease Reports · doi:10.3233/adr-200249 · Cited by 23
A Longitudinal Investigation of the Ethnic and National Identities of Children With Migration Background in Germany
Froehlich, L., Martiny, S. E., & Deaux, K. (2019) · Social Psychology · doi:10.1027/1864-9335/a000403 · Cited by 19
Psoriasis Characteristics for the Early Detection of Psoriatic Arthritis
Belman, S., Walsh, J. A., Carroll, C., Milliken, M., Haaland, B. A., Duffin, K. C., Krueger, G. G., & Feng, B. (2021) · The Journal of Rheumatology · doi:10.3899/jrheum.201123 · Cited by 17
Missing Data and Multiple Imputation in the Context of Multivariate Analysis of Variance
Finch, W. H. (2015) · The Journal of Experimental Education · doi:10.1080/00220973.2015.1011594 · Cited by 17
“As previously noted, traditionally MI has been largely limited to contexts in which the statistic of interest is a parameter to which the combining rules described earlier are easily applied (van Ginkel & Kroonenberg, 2014).”
Method
Next generation crosscutting themes: Factors that contribute to students' understandings of size and scale
Chesnutt, K. M., Jones, M. G., Hite, R. L., Cayton, E. M., Ennes, M. E., Corin, E. N., & Childers, G. M. (2018) · Journal of Research in Science Teaching · doi:10.1002/tea.21443 · Cited by 18
Recall of Affective Responses to Exercise: Examining the Influence of Intensity and Time
Sławińska, M., & Davis, P. A. (2020) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2020.573525 · Cited by 18
A Latent Class Analysis of Gambling Activity Patterns in a Canadian University Sample of Emerging Adults: Socio-demographic, Motivational, and Mental Health Correlates
Sanscartier, M. D., Edgerton, J. D., & Roberts, L. W. (2017) · Journal of Gambling Studies · doi:10.1007/s10899-017-9734-y · Cited by 18
Cost‐effectiveness of three internet‐based interventions for eating disorders: A randomized controlled trial
Rohrbach, P. J., Dingemans, A. E., van Furth, E. F., Spinhoven, P., van Ginkel, J. R., Bauer, S., & Marle, M. E. V. D. A. (2022) · International Journal of Eating Disorders · doi:10.1002/eat.23763 · Cited by 14
“Costs, both health care and societal, and effects in terms of QALYs (EQ-5D-5L) and capabilities (ICECAP-A) over the 14 month period were compared between the four conditions using analyses of variance (ANOVA) pooled across imputations (Rubin, 1987; Van Ginkel & Kroonenberg, 2014).”
Method
Convergence of Parent Report and Child Behavior using the Dyadic Parent-Child Interaction Coding System (DPICS)
Cotter, A., & Brestan‐Knight, E. (2020) · Journal of Child and Family Studies · doi:10.1007/s10826-020-01813-2 · Cited by 14
A longitudinal examination of the relations between motivation, math achievement, and STEM career aspirations among Black students
Adler, R. M., Rittle‐Johnson, B., Hickendorff, M., & Durkin, K. L. (2023) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102240 · Cited by 15
Boys have not caught up, family influences still continue: Influences on executive functioning and behavioral self‐regulation in elementary students in Germany
Gunzenhauser, C., Saalbach, H., & von Suchodoletz, A. (2017) · PsyCh Journal · doi:10.1002/pchj.180 · Cited by 13
“As pooling findings from formal repeated measurement models has proven difficult (van Ginkel & Kroonenberg, 2014) and consequently has not been implemented in SPSS 22.0, we used an analysis-of-change approach similar to repeatedmeasurement analysis of variance to answer the research question (Van…”
MethodInfluential
Effect of mindfulness-based intervention on endurance performance under pressure and performance-relevant mental attributes, an interdisciplinary perspective: Protocol for a Mindfulness-Based Peak Performance (MBPP) trial
Chang, Y., Gill, D. L., Creswell, J. D., Chen, D.-T., Lin, C., Chu, C., & Nien, J. (2023) · Contemporary Clinical Trials · doi:10.1016/j.cct.2023.107175 · Cited by 16
Web-Based Intervention Effects on Mild Cognitive Impairment Based on Apolipoprotein E Genotype: Quasi-Experimental Study
Tsolaki, A., Tsolaki, M., Pandria, N., Lazarou, E., Gkatzima, O., Zilidou, V. I., Karagianni, M. A., Iakovidou‐Kritsi, Z., Kimiskidis, V. Κ., & Bamidis, P. D. (2019) · Journal of Medical Internet Research · doi:10.2196/14617 · Cited by 12
“Moreover, we calculated the averaged median across the 20 complete datasets, as suggested by van Ginkel and Kroonenberg [65].”
Method
The relationship between social norms and sexual risk-reduction intentions and behavior among men who have sex with men: Findings from an eHealth intervention.
Basaran, A. M. B., Christensen, J. L., Miller, L. C., Appleby, P. R., & Read, S. J. (2019) · Psychology of Addictive Behaviors · doi:10.1037/adb0000467 · Cited by 12
Determinants of Non-paid Task Division in Gay-, Lesbian-, and Heterosexual-Parent Families With Infants Conceived Using Artificial Reproductive Techniques
Gelderen, L. V. R., Ellis‐Davies, K., Huijzer-Engbrenghof, M., Jorgensen, T. D., Gross, M., Winstanley, A., Rubio, B., Vecho, O., Lamb, M. E., & Bos, H. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00914 · Cited by 12
A Nonparametric, Multiple Imputation-Based Method for the Retrospective Integration of Data Sets
Carrig, M. M., Manrique‐Vallier, D., Ranby, K. W., Reiter, J. P., & Hoyle, R. H. (2015) · Multivariate Behavioral Research · doi:10.1080/00273171.2015.1022641 · Cited by 12
“(As noted by van Ginkel and Kroonenberg, 2014, and Bodner, 2008, among others, it is beneﬁcial to use large m when fractions of missing information are large; doing so stabilizes estimates of standard errors and degrees of freedom.”
Background
The christian religious education teachers’ attitudes toward the five-stage lesson plan framework in Kenya: A gender-based analysis
Saoke, V. O., Musafiri, C. M., Ndwiga, Z. N., & Githaiga, P. W. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19104 · Cited by 10
Maternal exposure to legacy PFAS compounds PFOA and PFOS is associated with disrupted cytokine homeostasis in neonates: The Upstate KIDS study (2008–2010)
Jones, L., Ghassabian, A., Yeung, E. H., Mendola, P., Kannan, K., & Bell, E. M. (2025) · Environment International · doi:10.1016/j.envint.2025.109288 · Cited by 10
Mental Health, Behavior Change Skills, and Eating Behaviors in Postpartum Women
Yu, Y. H., Ma, Q., Fernandez, I. D., & Groth, S. W. (2021) · Western Journal of Nursing Research · doi:10.1177/01939459211021625 · Cited by 12
“Post-imputation analyses over these multiple imputed datasets were then combined using the Rubin’s rule (van Ginkel & Kroonenberg, 2014).”
Method
Language acquisition of early sequentially bilingual children is moderated by short‐term memory for order in developmental language disorder: Findings from the HelSLI study
Lahti‐Nuuttila, P., Laasonen, M. R., Smolander, S., Kunnari, S., Arkkila, E. P., & Service, E. (2021) · International Journal of Language & Communication Disorders · doi:10.1111/1460-6984.12635 · Cited by 10
“Again, the results from all 20 samples were pooled using small-sample degrees of freedom (Reiter 2007, van Ginkel and Kroonenberg 2014).”
Background
Accuracy of Metabolic Cost Predictive Equations During Military Load Carriage
Vine, C. A. J., Coakley, S. L., Blacker, S. D., Doherty, J., Hale, B. J., Walker, E. F., Rue, C. A., Lee, B. J., Flood, T. R., Knapik, J. J., et al. (2020) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000003644 · Cited by 10
“To manage missing data of included subjects (8%), multiple imputation procedures were conducted using a modified version of the procedures described by van Ginkel and Kroonenberg (32).”
Method
Does curve pattern impact on the effects of physiotherapeutic scoliosis specific exercises on Cobb angles of participants with adolescent idiopathic scoliosis: A prospective clinical trial with two years follow-up
Fan, Y., To, M. K.-T., Yeung, E., Wu, J., He, R. X., Xu, Z., Zhang, R., Li, G., Cheung, K., & Cheung, J. P. Y. (2021) · PLoS ONE · doi:10.1371/journal.pone.0245829 · Cited by 10
“001); hence, the degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity [40].”
Method
Exposure to perfluoroalkyl substances and neonatal immunoglobulin profiles in the upstate KIDS study (2008–2010)
Jones, L., Ghassabian, A., Lawrence, D. A., Sundaram, R., Yeung, E. H., Kannan, K., & Bell, E. M. (2022) · Environmental Pollution · doi:10.1016/j.envpol.2022.119656 · Cited by 10
Using an ideal observer analysis to investigate the visual perceptual efficiency of individuals with a history of non-suicidal self-injury when identifying emotional expressions
Ziebell, L., Collin, C. A., Rainville, S. J. M., Mazalu, M., & Weippert, M. V. (2020) · PLoS ONE · doi:10.1371/journal.pone.0227019 · Cited by 10
“However, because no explicit rules have been defined for pooling F-tests of (repeated-measures) analysis of variance [47] and this function is not available for an ANOVA calculation in IBM SPSS Statistics 24, the data for the median error term of the 9 imputed datasets was used to obtain the mean squared value for the…”
Method
Can 3 mg·kg−1 of Caffeine Be Used as An Effective Nutritional Supplement to Enhance the Effects of Resistance Training in Rugby Union Players?
Tamilio, R. A., Clarke, N. D., Duncan, M. J., Morris, R. O., Grgić, J., & Tallis, J. A. (2021) · Nutrients · doi:10.3390/nu13103367 · Cited by 13
“All intervention data (RTF and RPE) were analysed using an imputation method [48,49] followed by a 3-factor mixed model ANOVA with a between factor of Group (caffeine or placebo) and within factors of Session (1–14) and Set (Set1 and Set2).”
Method
Extrapulmonary symptoms of patients with asthma treated in specialist pulmonary care
Salsman, M. L., Nordberg, H. O., Wïttchen, H., Klotsche, J., Mühlig, S., Riedel, O., & Ritz, T. (2021) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2021.110538 · Cited by 9
Handling missing data in longitudinal clinical trials: three examples from the pediatric psychology literature
Peugh, J., & Mara, C. A. (2024) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsae070 · Cited by 8
Psychological distress and symptom-related burnout in asthma during the COVID-19 pandemic
Salsman, M. L., Nordberg, H. O., Howell, J., Berthet-Miron, M. M., Rosenfield, D., & Ritz, T. (2023) · Journal of Behavioral Medicine · doi:10.1007/s10865-023-00412-y · Cited by 10
“The FDR critical value was calculated for the fifteen MR analyses investigating our primary aims (Benjamini & Hochberg, 1995; van Ginkel & Kroonenberg, 2014).”
How Do You Like Your Forgiveness? Communication Style Preferences and Effects
Merolla, A. J., Zhang, S., McCullough, J. L., & Sun, S. (2017) · Communication Studies · doi:10.1080/10510974.2017.1377743 · Cited by 8
“Data were analyzed with the repeated measures ANOVA procedures outlined by Van Ginkel and Kroonenberg (2014), which enable pooled estimates of F tests.”
Method
Short-Term Memory for Serial Order Moderates Aspects of Language Acquisition in Children With Developmental Language Disorder: Findings From the HelSLI Study
Lahti‐Nuuttila, P., Service, E., Smolander, S., Kunnari, S., Arkkila, E. P., & Laasonen, M. R. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.608069 · Cited by 8
“Again, the results from all 20 samples were pooled using small-sample degrees of freedom (Reiter, 2007; Van Ginkel and Kroonenberg, 2014).”
BackgroundMethodInfluential
Corresponding Changes in Sleep and Discrimination: A Three-year Longitudinal Study Among Ethnically/Racially Diverse Adolescents
Lorenzo, K., Xie, M., Cham, H., El‐Sheikh, M., & Yip, T. (2024) · Journal of Youth and Adolescence · doi:10.1007/s10964-024-02086-4 · Cited by 7
Boys have caught up, family influences still continue: Influences on executive functioning and behavioral self‐regulation in elementary students in Germany
Gunzenhauser, C., Saalbach, H., & von Suchodoletz, A. (2017) · PsyCh Journal · doi:10.1002/pchj.157 · Cited by 7
“As pooling findings from formal repeated measurement models has proven difficult (van Ginkel & Kroonenberg, 2014) and consequently has not been implemented in SPSS 22.0, we used an analysis of change approach similar to repeatedmeasurement analysis of variance to answer the research question (Van…”
Influential
Retracted
Understanding the perceived business obstacles and determinants of credit adoption by textile firms: evidences from World Bank’s enterprises survey
Singh, S., Amini, M., Jamshed, M., Sharma, H. P., & Khan, W. (2023) · Research Journal of Textile and Apparel · doi:10.1108/rjta-12-2022-0155 · Cited by 7
From Serodiscordant to Magnetic: The Feasibility and Acceptability of a Pilot Psychoeducational Group Intervention Designed to Improve Relationship Quality
Eaton, A. D., Cattaneo, J., Watchorn, J. M., Bilbao-Joseph, C., Bowler, S., Hazelton, M., Myslik, J., Ross, A., & Chambers, L. A. (2017) · Social Work With Groups · doi:10.1080/01609513.2017.1369922 · Cited by 9
“Although it is possible to manually pool the results of a repeated measures ANOVA with multiple imputed data, this process is complicated and was beyond the scope of this study (van Ginkel & Kroonenberg, 2014).”
Method
Effectiveness of a numeracy intelligent tutoring system in kindergarten: A conceptual replication
Grimes, K. R., Park, S., McClelland, A. M., Park, J., Lee, Y. R., Nozari, M., Umer, Z., Zaparolli, B., & Bryant, D. P. (2021) · Journal of Numerical Cognition · doi:10.5964/jnc.6931 · Cited by 10
The Efficacy of a Group Program Integrating Compassionate Mind Training with Cognitive Behavioral Therapy for College Students With Test Anxiety in Mainland China
Guo, M., Liu, A., Wang, X., & Liu, Y. (2024) · Mindfulness · doi:10.1007/s12671-024-02342-5 · Cited by 7
“Furthermore, the approach developed by van Ginkel was used to pool the results of two-way repeated measures ANOVAs (van Ginkel & Kroonenberg, 2014).”
Influential
Evaluation of multiple-imputation procedures for three-mode component models
van Ginkel, J. R., & Kroonenberg, P. M. (2017) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2017.1355368 · Cited by 6
“an overall F-test) have also been available for a long time [8], but were only recently explicitly worked out for analysis of variance [28].”
BackgroundSelf-citation
Outcomes of In Situ Training for Disclosure as a Standalone and a Booster to a Child Protective Behaviors Education Program
White, C., Shanley, D. C., Zimmer‐Gembeck, M. J., Walsh, K., Hawkins, R. M. F., & Lines, K. (2018) · Child Maltreatment · doi:10.1177/1077559518816877 · Cited by 6
COVID-stressed schools struggled to teach mathematics
Lange‐Küttner, C. (2024) · Acta Psychologica · doi:10.1016/j.actpsy.2024.104174 · Cited by 6
Imputation and Missing Data
Momeni, A., Pincus, M. R., & Libien, J. M. (2017) · Springer eBooks · doi:10.1007/978-3-319-60543-2_8 · Cited by 6
Healthy Start Home Visiting Program—School-Based Model: Cluster Randomized Controlled Trial
Leung, C., Tsang, S., Heung, K., & Tam, M. (2018) · Research on Social Work Practice · doi:10.1177/1049731518786578 · Cited by 6
The perceived adultism scale: Initial validation of an instrument measuring youth experiences of age-based oppression
Hall, S. F., & 4theRecord Research Team (2025) · Applied Developmental Science · doi:10.1080/10888691.2024.2435378 · Cited by 5
“Significance testing is based on Rubin’s (1987) degrees of freedom adjustment, which assumes infinite degrees of freedom for the complete dataset case (van Ginkel & Kroonenberg, 2014).”
Imputação de dados na análise de variância em experimentos no Delineamento Inteiramente Casualizado
Stochero, E. L. M., Jacobi, L. F., & Lúcio, A. D. (2020) · Ciência e Natura · doi:10.5902/2179460x40447 · Cited by 5

Authors: van Ginkel, J. R., & Kroonenberg, P. M.
Affiliations: Leiden University
Leiden University (NL)
ORCID: Kroonenberg, Pieter M.
Venue: Multivariate Behavioral Research
Publisher: Informa UK Limited
Published: 2014-01
Volume: 49
Issue: 1
Pages: 78-91
ISSN: 0027-3171 (ISSN-L)
1532-7906
DOI: 10.1080/00273171.2013.855890
PMC: PMC4029775
OpenAlex: W2155931297
Cited by: 152
References: 33
Citations by year: 2026: 6 · 2025: 9 · 2024: 11 · 2023: 12 · 2022: 12 · 2021: 13 · 2020: 13 · 2019: 17 · 2018: 16 · 2017: 20 · 2016: 12 · 2015: 5 · 2014: 5
FWCI: 14.80
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Locations: Multivariate Behavioral Research (publishedVersion)
PubMed (publishedVersion)
PubMed Central (acceptedVersion)
Funders: National Institute on Aging (R01AG008235)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords: multiple imputation, analysis of variance, missing data, repeated measures ANOVA

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Analysis of Variance of Multiply Imputed Data

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks