The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Analysis of Variance of Multiply Imputed Data
Open accessAbstract
As a procedure for handling missing data, Multiple imputation consists of estimating the missing data multiple times to create several complete versions of an incomplete data set. All these data sets are analyzed by the same statistical procedure, and the results are pooled for interpretation. So far, no explicit rules for pooling F tests of (repeated-measures) analysis of variance have been defined. In this article we outline the appropriate procedure for the results of analysis of variance (ANOVA) for multiply imputed data sets. It involves both reformulation of the ANOVA model as a regression model using effect coding of the predictors and applying already existing combination rules for regression models. The proposed procedure is illustrated using 3 example data sets. The pooled results of these 3 examples provide plausible F and p values.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Missing values were handled by multivariate multiple imputation using Rubin’s method.26”
From Aerobic, resistance, or combined exercise training and cardiovascular risk profile in overweight or obese adults: the CardioRACE trial (2024) · cites this paper as [26]
In the citing paper
“as no consensus on how to best pool F statistics was available [34].”
From An Internet-Based Cognitive Behavioral Therapy Program Adapted to Patients With Cardiovascular Disease and Depression: Randomized Controlled Trial (2019) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,385
mice: Multivariate Imputation by Chained Equations in R
Buuren, S. V., & Groothuis-Oudshoorn, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v045.i03 · Cited by 13,870
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,006
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627
Journal of Statistical Software
de Leeuw, J. (2009) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.10 · Cited by 3,651
Hox, J., Moerbeek, M., & van de Schoot, R. (2002) · doi:10.4324/9781410604118 · Cited by 2,419
Research works printed but not linked: 4
- Dunn , L. M. and Dunn , L. M. 1981 . Peabody Picture Test – Revised manual for Forms L and M , Circle Pines, MN : American Guidance Service .
- Snijders , T. and Bosker , R. 1999 . Multilevel analysis , London , UK : Sage .
- Winer , B. J. 1971 . Statistical principles in experimental designs, 2nd ed. , New York, NY : McGraw-Hill .
- Van Ginkel , J. R. and Kroonenberg , P. M. 2011, July . Multiple imputation and (repeated-measures) analysis of variance , Hong Kong : Presentation given at the 17th international meeting of the Psychometric Society .
Cited by
Sorted by citations
Items 101–152 of 152
Biegała, M., & Hydzik, A. (2016) · Journal of Medical Physics · doi:10.4103/0971-6203.189490 · Cited by 5
Stahn, V., & Ditton, H. (2017) · Zeitschrift für Erziehungswissenschaft · doi:10.1007/s11618-017-0757-z · Cited by 5
Does Setting Goals Enhance Parenting Intervention Effects? A Field Experiment
van Aar, J., Leijten, P. H. O., Overbeek, G. J., Thomaes, S. C. E., & Rothman, A. J. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.05.004 · Cited by 5
Adeyeye, T. E., Jones, L., Yeung, E. H., Mendola, P., Lawrence, D. A., Lin, S., & Bell, E. M. (2023) · American Journal of Reproductive Immunology · doi:10.1111/aji.13688 · Cited by 5
Tang, X., Kikas, E., Pakarinen, E., Laursen, B. P., & Lerkkanen, M. (2022) · Journal of Research in Reading · doi:10.1111/1467-9817.12385 · Cited by 6
Are Consumers Ready for Flying Taxis? A Choice-based Conjoint Analysis of eVTOLs in Germany
Zaps, R., & Chankov, S. M. (2022) · doi:10.1109/ieem55944.2022.9989956 · Cited by 4
“For example, [20] test several long-distance alternatives, including car, bus, airplane, and train, whereas [21] provide a simpler choice between car and train, and [22] give an example of route and mode selection in urban travel.”
BackgroundEffects of a Soft Baby Carrier on Fathers’ Behavior and Hormones: A Randomized Controlled Trial
Verhees, M. W. F. T., Lotz, A. M., de Moor, M. H. M., van IJzendoorn, M. H., Fidder, A. A. E. J., Buisman, R. S. M., & Bakermans‐Kranenburg, M. J. (2023) · Journal of Child and Family Studies · doi:10.1007/s10826-023-02678-x · Cited by 4
“To allow pooling of F -tests of the multiply imputed datasets, ANOVAs were carried out using the SPSS Mixed Models procedure using effect coding, including a random intercept of person to account for the repeated measures within persons (see Van Ginkel & Kroonenberg, 2014).”
Multiple imputation to balance unbalanced designs for two-way analysis of variance
van Ginkel, J. R., & Kroonenberg, P. M. (2021) · Methodology · doi:10.5964/meth.6085 · Cited by 4
“However, Van Ginkel and Kroonenberg (2014) did not carry out any simulations to back up this claim.”
BackgroundMethodInfluentialSelf-citationZhao, L., Dehé, N., & Murphy, V. A. (2017) · Journal of Pragmatics · doi:10.1016/j.pragma.2017.10.015 · Cited by 4
Bradshaw, E. L., Sahdra, B. K., Ciarrochi, J., Parker, P. D., Martos, T., & Ryan, R. M. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/srb75 · Cited by 5
Durbeej, N., Salari, R., Sarkadi, A., Kankaanpää, R., Derluyn, I., Verelst, A., & Osman, F. A. (2024) · BMC Public Health · doi:10.1186/s12889-024-19412-x · Cited by 3
Nardy, J. R., Duden, A. S., Martensen, A. C., Henkens, K., Verweij, P. A., & Verburg, R. W. (2025) · Land Use Policy · doi:10.1016/j.landusepol.2025.107559 · Cited by 3
Vallejo, G., Fernández, P., Livacic-Rojas, P. E., & Herrero, E. T. (2018) · Psicothema · doi:10.7334/psicothema2018.192 · Cited by 3
“These rules have been developed, for example, for pooling the point estimates of the parameters of interest, such as a single regression coeffi cient or the difference between two sample means (Rubin & Schenker, 1986), multivariate tests of hypotheses (Li, Raghunathan, & Rubin, 1991), p-values (Li, Meng, Raghunathan,…”
InfluentialHoesseini, A., Dronkers, E. A. C., Dieleman, E., Herdt, M. J. D., Wieringa, M. H., van Velthuysen, M. F., Offerman, M. P. J., Sewnaik, A., Monserez, D., Keereweer, S., et al. (2025) · BMC Cancer · doi:10.1186/s12885-025-14100-4 · Cited by 3
“However, when the missing (completely) at random assumption is plausible, multiple imputation can be applied to handle the missing data [42, 43].”
Two-Stage Limited-Information Estimation for Structural Equation Models of Round-Robin Variables
Jorgensen, T. D., Bhangale, A. M., & Rosseel, Y. (2024) · Stats · doi:10.3390/stats7010015 · Cited by 3
“The pooled point and SE estimates can be used to calculate pooled Wald-type statistics [49,50]; various methods have also been proposed for pooled score [51] and likelihood-ratio test (LRT) [52,53] statistics, which are asymptotically equivalent [54].”
MethodSusetyo, B., & Fitrianto, A. (2024) · CAUCHY Jurnal Matematika Murni dan Aplikasi · doi:10.18860/ca.v9i1.24824 · Cited by 3
“According to [31], a MAPE value of less than 10% indicates that the model is very good; between 10-20%, the model is good, and 20-50% indicates the model is still feasible to use to predict data.”
BackgroundMethodSchubert, J. C., & Wrenger, K. (2016) · DergiPark (Istanbul University) · Cited by 3
Kalon, L., Freund, H., Rinn, A., Watkins, P. C., Zarski, A., & Lehr, D. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.119664 · Cited by 2
Leal, T. Y., Garib, D. G., Eto, H. C., Rosa, R., Miranda, F., Soares, A., Fukushiro, A. P., & Berretin‐Felix, G. (2025) · Clinical Oral Investigations · doi:10.1007/s00784-025-06483-2 · Cited by 2
“To address this, imputation techniques were applied to handle missing data resulting from non-adherence [ 26].”
Combination rules for homoscedastic and heteroscedastic MANOVA models from multiply imputed datasets
Vallejo, G., Fernández, P., & Rosário, P. J. S. L. F. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01429-w · Cited by 2
The Development of a Community Counseling Training Clinic for Latino Immigrants
Gutierez, D., Revens, K. E., & DeHaven, M. J. (2019) · Collaborations A Journal of Community-Based Research and Practice · doi:10.33596/coll.26 · Cited by 2
Mechanisms of change in posttraumatic headache–related disability: A mediation model
Nabity, P. S., Reed, D. E., McGeary, C. A., Houle, T. T., Jaramillo, C. A., Resick, P. A., Eapen, B. C., Litz, B. T., Mintz, J., Penzien, D. B., et al. (2023) · Headache The Journal of Head and Face Pain · doi:10.1111/head.14480 · Cited by 2
Epe‐Jungeblodt, F., Pauli, P., Schwab, F. J., & Andreatta, M. (2024) · Journal of Clinical Psychology · doi:10.1002/jclp.23702 · Cited by 2
van Ginkel, J. R., & Kroonenberg, P. M. (2015) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0000000000000137 · Cited by 2
Self-citationFoelber, K. J. (2017) · JMU Scholoraly Commons (James Madison University) · Cited by 2
Endrighi, R., Zhao, Y., Hughes, R. B., Kumar, D., & Borrelli, B. (2021) · Annals of Behavioral Medicine · doi:10.1093/abm/kaab077 · Cited by 1
Ceccacci, A., Markoulakis, R., & LEVITT, A. J. (2022) · Health & Social Care in the Community · doi:10.1111/hsc.13927 · Cited by 1
Evoke - Developing Skills in Youth to Solve the World’s Most Complex Problems
Freeman, B., & Hawkins, R. (2017) · World Bank, Washington, DC eBooks · doi:10.1596/29167 · Cited by 1
“First, pairwise and listwise deletion are acceptable techniques for making valid inferences on incomplete data when the values are MCAR (Little & Rubin, 2002; van Ginkel & Kroonenberg, 2014).”
MethodMeneghini, V., Barbosa, A. R., Lourenço, C. L. M., & Borgatto, A. F. (2022) · Ageing International · doi:10.1007/s12126-022-09491-9 · Cited by 1
“The pooled analyses were performed through the combination of rules described by van Ginkel and Kroonenberg (2014), using an SPSS macro (van Ginkel, 2014).”
MethodJones, L., & Bell, E. M. (2024) · Research Square · doi:10.21203/rs.3.rs-4345399/v1 · Cited by 1
Giesbrecht, E. (2016) · cIRcle (University of British Columbia) · doi:10.14288/1.0302682
Predictors of Positive Adjustment in a Sample of Children Impacted by Hurricane Katrina
Piscitello, J. (2017) · doi:10.31390/gradschool_theses.4337
“49 Van Ginkel, J. R., & Kroonenberg, P. M. (2014).”
MethodHealthcare payer type and HIV health:a retrospective analysis
Kay, E. S. (2018)
Assessing the Effectiveness of Goal Setting in Group Treatment in a Correctional Institution
Kostiuk, M. A. (2019) · Digital Commons - DU (University of Denver)
Kaplan, D. M. (2020) · UA Campus Repository (The University of Arizona)
van Ginkel, J. R., & Kroonenberg, P. M. (2020) · Methodology · doi:10.5964/meth.4327
“However, Van Ginkel and Kroonenberg (2014) did not carry out any simulations to back up this claim.”
InfluentialSelf-citationRetractedSolomon, R., Egeren, L. A. V., & Huber, M. Q. (2015) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0000000000000154
Westas, M. (2022) · Linköping University Medical Dissertations · doi:10.3384/9789179293796
Alade, A. G. C., Okolo, A., Akinrefon, S., & Dike, J. (2024) · International Journal of Development Mathematics (IJDM) · doi:10.62054/ijdm/0104.9
Davies, S. J., Doig, E., Gullo, M. J., & Gullo, H. (2025) · Canadian Journal of Occupational Therapy · doi:10.1177/00084174251327307
“A modi fi ed approach involving a series of multiple regression analyses controlling for baseline performance at timepoint one to test whether the CO-OP treatment group predicted differential change on key measures was employed (van Ginkel & Kroonenberg, 2014).”
Alade, S. P., Okolo, A., Akinrefon, A. A., & John, D. I. (1970) · FUDMA Journal of Sciences · doi:10.33003/fjs-2024-0804-2669
Does the Inclusion of Science Content and Skills in Preschool Curricula Matter?
Saçkes, M., & Trundle, K. C. (2025) · Early Childhood Education Journal · doi:10.1007/s10643-025-01955-2
Fictorie, V., Schuengel, C., de Moor, M. H. M., Xerxa, Y., Tierolf, B., Jonkman, C., Visser, M. M., & Steketee, M. (2025) · Development and Psychopathology · doi:10.1017/s0954579425100722
Hornfeck, F., Bovenschen, I., & Kappler, S. (2026) · Journal of Child and Family Studies · doi:10.1007/s10826-025-03216-7
“Due to the restrictions of SPSS (van Ginkel & Kroonenberg, 2014), repeated measures ANOVAs were not conducted with imputed data sets.”
InfluentialFigueroa, J., Ha, T., Anderson, S. F., Maras, O., & Quiroz, S. I. (2026) · Journal of Research on Adolescence · doi:10.1111/jora.70167
Designing Novel Visual Physiologic Monitor Displays for Patient Care
Cummins, M. L., Sicard, C., Yang, S., Schlesinger, J. J., & Shive, J. (2026) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208261445900
“F -tests and post-hoc comparisons were pooled across im-putations using Rubin ’ s rules using an algorithm written in SPSS (van Ginkel, 2010; van Ginkel & Kroonenberg, 2014).”
Tracking Divergence of Objective and Subjective Health Across Age and Historical Time
Park, R., Gerstorf, D., Drewelies, J., Eibich, P., Smith, J., Lindenberger, U., Hoppmann, C. A., Steinhagen‐Thiessen, E., Demuth, I., & Ram, N. (2026) · International Journal of Behavioral Development · doi:10.1177/01650254261446800
Synthetic Longitudinal Tabular Data Generation via CopulaPreprint
Cai, H., Yu, W., Lu, R., Chattopadhyay, I., Zhang, X., & Liu, J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.03.742474
“The ensemble average is This nested design parallels multiple-imputation variance decomposition (Rubin, 1987; van Ginkel and Kroonenberg, 2014), but is used here as a descriptive diagnostic rather than for Rubin-style inference.”
Kinkopf, K. M., Trombley, T. M., Seckinger, J., Agarwal, S. C., Giuffra, V., Minozzi, S., Campana, S., & Riccomi, G. (2026) · American Journal of Biological Anthropology · doi:10.1002/ajpa.70341
“Numerous advanced methods have also been addressed in the literature which deal with missing data specifically in reference to PCA (Takane and Oshima-Takane 2003; van Ginkel 2023; van Ginkel and Kroonenberg 2014).”
Metadata
- Authors
- van Ginkel, J. R., & Kroonenberg, P. M.
- Affiliations
- Leiden University
- Leiden University (NL)
- Venue
- Multivariate Behavioral Research
- Publisher
- Informa UK Limited
- Published
- 2014-01
- Volume
- 49
- Issue
- 1
- Pages
- 78-91
- ISSN
- 0027-3171 (ISSN-L)
- 1532-7906
- PMC
- PMC4029775
- OpenAlex
- W2155931297
- Cited by
- 152
- References
- 33
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 11 · 2023: 12 · 2022: 12 · 2021: 13 · 2020: 13 · 2019: 17 · 2018: 16 · 2017: 20 · 2016: 12 · 2015: 5 · 2014: 5
- FWCI
- 14.80
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Multivariate Behavioral Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute on Aging (R01AG008235)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- multiple imputation, analysis of variance, missing data, repeated measures ANOVA
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
