The DOI matches its Crossref record; 20 of the 21 references it cites were found in a registry or an index.
Sufficient Sample Sizes for Multilevel Modeling
Abstract
Abstract. An important problem in multilevel modeling is what constitutes a sufficient sample size for accurate estimation. In multilevel analysis, the major restriction is often the higher-level sample size. In this paper, a simulation study is used to determine the influence of different sample sizes at the group level on the accuracy of the estimates (regression coefficients and variances) and their standard errors. In addition, the influence of other factors, such as the lowest-level sample size and different variance distributions between the levels (different intraclass correlations), is examined. The results show that only a small sample size at level two (meaning a sample of 50 or less) leads to biased estimates of the second-level standard errors. In all of the other simulated conditions the estimates of the regression coefficients, the variance components, and the standard errors are unbiased and accurate.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Based on a simulation study on sufficient sample sizes for multilevel modeling by Maas and Hox (2005), the target sample size was set at 100 participants from both countries.”
From Visualizing Uncertainty to Promote Clinicians’ Understanding of Measurement Error (2023) · cites this paper as Maas and Hox (2005)
In the citing paper
“Here we hope to address pseudoreplication in single-cell analyses by demonstrating what a large and long-standing body of statistical literature already confirms: applying statistical inference to replicates that are not statistically independent without properly accounting for their correlation structure will inflate type 1 error rates and lead to spurious results4–6,9,25–28.”
From A practical solution to pseudoreplication bias in single-cell studies (2021) · cites this paper as [27]
In the citing paper
“As the denominator of most statistical tests (e.g., Wald test) is a function of the variance, not accounting for the positive correlation among sampling units underestimates the true standard error and leads to false positives27,28.”
From A practical solution to pseudoreplication bias in single-cell studies (2021) · cites this paper as [27]
In the citing paper
“Moreover, Maas and Hox (2005) found that, for sample sizes greater than 50, the estimates of the regression coefficients, the variance components, and the standard errors are unbiased and accurate.”
From Moving the Needle: Evidence of an Effective Study Strategy Intervention in a Community College Biology Course (2022) · cites this paper as Maas and Hox (2005)
In the citing paper
“Thus, while our course clusters are only moderate in size and our overall sample size is above 50, both are large enough to carry out the linear mixed models that will yield unbiased estimates of regression coefficients and standard errors.”
From Moving the Needle: Evidence of an Effective Study Strategy Intervention in a Community College Biology Course (2022) · cites this paper as Maas and Hox (2005)
In the citing paper
“However, the negligible school effects meant that two-level modeling (class, student) was sufficient, with the sample size also more than adequate for this approach (Field, 2013;Maas & Hox, 2005).”
From Teacher–Student Dialogue During Classroom Teaching: Does It Really Impact on Student Outcomes? (2019) · cites this paper as Maas & Hox, 2005
In the citing paper
“The design effect for the two outcomes of interest was smaller than 2 (science identity 1.77; research self-efficacy 1.57). Thus, multilevel modeling (individuals nested within cohorts) is not necessary (Satorra and Muthen, 1995; Maas and Hox, 2005).”
From Outcomes and Processes in the Meyerhoff Scholars Program: STEM PhD Completion, Sense of Community, Perceived Program Benefit, Science Identity, and Research Self-Efficacy (2016) · cites this paper as Maas and Hox, 2005
In the citing paper
“Although students were nested within the 16 cohorts, multilevel modeling was not used in this study, because 50 or fewer clusters lead to biased parameter estimates in multilevel analysis (Maas and Hox, 2005).”
From Outcomes and Processes in the Meyerhoff Scholars Program: STEM PhD Completion, Sense of Community, Perceived Program Benefit, Science Identity, and Research Self-Efficacy (2016) · cites this paper as Maas and Hox, 2005
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Multilevel analysis : an introduction to basic and advanced multilevel modeling
Snijders, T. A. B., & Bosker, R. (1999) · Cited by 6,830
Multilevel analysis techniques and applications
Hox, J. J., Moerbeek, M., & van de Schoot, R. (2002) · Cited by 6,006
Self-citationGoldstein, H. (2006) · Technometrics · doi:10.1198/tech.2006.s417 · Cited by 4,960
Klein, K. J., & Kozlowski, S. W. J. (2000) · Cited by 4,914
Gill, J. (2005) · Encyclopedia of Social Measurement · doi:10.1016/b0-12-369398-5/00166-3 · Cited by 3,126
Introducing Multilevel Modeling
Kreft, I., & de Leeuw, J. (1998) · doi:10.4135/9781849209366 · Cited by 2,997
Statistical Strategies for Small Sample Research
Hoyle, R. H. (1999) · Cited by 1,896
Complex Sample Data in Structural Equation Modeling
Muthen, B. O., & Satorra, A. (1995) · Sociological Methodology · doi:10.2307/271070 · Cited by 1,586
Gulliford, M. C., Ukoumunne, O. C., & Chinn, S. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009904 · Cited by 853
An Introduction to Multilevel Modeling Techniques
Heck, R. H., & Thomas, S. L. (2020) · Routledge eBooks · doi:10.4324/9780429060274 · Cited by 712
Multilevel Statistical Models (3rd ed.)
Lethonen, R. (2005) · Quarterly Publications of the American Statistical Association · Cited by 544
A User's Guide to MLwiN version 2.0
Rasbash, J., Steele, F., Browne, W. J., & Goldstein, H. (2004) · Bristol Research (University of Bristol) · Cited by 457
Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models
Browne, W. J., & Draper, D. (2000) · Computational Statistics · doi:10.1007/s001800000041 · Cited by 210
Likelihood Ratio Tests for Fixed Model Terms using Residual Maximum Likelihood
Welham, S. J., & Thompson, R. (1997) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00092 · Cited by 147
Analysis of longitudinal data using the hierarchical linear model
Snijders, T. A. B. (1996) · Quality & Quantity · doi:10.1007/bf00170145 · Cited by 100
The analysis of longitudinal, multilevel data
Raudenbush, S. W. (1989) · International Journal of Educational Research · doi:10.1016/0883-0355(89)90024-4 · Cited by 94
Variance Component Testing in Multilevel Models
Berkhof, J. H., & Snijders, T. A. B. (2001) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986026002133 · Cited by 81
Small sample inference for the fixed effects in the mixed linear model
Manor, O., & Zucker, D. M. (2003) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2003.10.005 · Cited by 45
Elston, D. A. (1998) · Biometrics · doi:10.2307/2533859 · Cited by 44
Cited by
Sorted by citations
Three-level meta-analysis of dependent effect sizes
Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0261-6 · Cited by 1,130
Convergence of Structural Equation Modeling and Multilevel Modeling
Kline, R. B. (2014) · SAGE Publications Ltd eBooks · doi:10.4135/9781446268261.n31 · Cited by 1,080
Kuo, Y.-C., Walker, A. E., Schroder, K. E. E., & Belland, B. R. (2014) · The Internet and Higher Education · doi:10.1016/j.iheduc.2013.10.001 · Cited by 1,082
Alternative methods for assessing mediation in multilevel data: The advantages of multilevel SEM
Preacher, K. J., Zhang, Z., & Zyphur, M. J. (2011) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2011.557329 · Cited by 889
Multilevel Modelling of Country Effects: A Cautionary Tale
Bryan, M. L., & Jenkins, S. P. (2015) · European Sociological Review · doi:10.1093/esr/jcv059 · Cited by 874
Metadata
- Authors
- Maas, C. J. M., & Hox, J. J.
- Affiliations
- Utrecht University (NL)
- Venue
- Methodology
- Publisher
- Hogrefe Publishing Group
- Published
- 2005-01
- Volume
- 1
- Issue
- 3
- Pages
- 86-92
- ISSN
- 1614-1881 (ISSN-L)
- 1614-2241
- OpenAlex
- W2112146495
- Cited by
- 3,582
- References
- 21
- Citations by year
- 2026: 292 · 2025: 282 · 2024: 190 · 2023: 220 · 2022: 241 · 2021: 289 · 2020: 293 · 2019: 258 · 2018: 268 · 2017: 248 · 2016: 201 · 2015: 196 · 2014: 157 · 2013: 130 · 2012: 105
- FWCI
- 15.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Methodology (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Keywords
- sample size, multilevel modeling, variance components, intraclass correlation coefficient, estimation bias, regression coefficients, simulation study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
