The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction
Abstract
Studies on small sample properties of multilevel models have become increasingly prominent in the methodological literature in response to the frequency with which small sample data appear in empirical studies. Simulation results generally recommend that empirical researchers employ restricted maximum likelihood estimation (REML) with a Kenward-Roger correction with small samples in frequentist contexts to minimize small sample bias in estimation and to prevent inflation of Type-I error rates. However, simulation studies focus on recommendations for best practice, and there is little to no explanation of why traditional maximum likelihood (ML) breaks down with smaller samples, what differentiates REML from ML, or how the Kenward-Roger correction remedies lingering small sample issues. Due to the complexity of these methods, most extant descriptions are highly mathematical and are intended to prove that the methods improve small sample performance as intended. Thus, empirical researchers have documentation that these methods are advantageous but still lack resources to help understand what the methods actually do and why they are needed. This tutorial explains why ML falters with small samples, how REML circumvents some issues, and how Kenward-Roger works. We do so without equations or derivations to support more widespread understanding and use of these valuable methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,723
(2010) · Cited by 7,034
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552 · cited 6×
“REML does not completely solve issues related to inflated Type-I error rates for fixed effects, so the Kenward-Roger correction (Kenward & Roger, 1997) has been shown to maintain nominal Type-I error rates and therefore its use has been recommended as best practice (e.g., McNeish & Stapleton,…”
BackgroundMethodInfluentialAn Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Sufficient Sample Sizes for Multilevel Modeling
Maas, C. J. M., & Hox, J. J. (2005) · Methodology · doi:10.1027/1614-2241.1.3.86 · Cited by 3,582
Bayesian structural equation modeling: A more flexible representation of substantive theory
Muthén, B., & Asparouhov, T. (2012) · Psychological Methods · doi:10.1037/a0026802 · Cited by 1,562
Explaining Fixed Effects: Random Effects Modeling of Time-Series Cross-Sectional and Panel Data
Bell, A., & Jones, K. (2015) · Political Science Research and Methods · doi:10.1017/psrm.2014.7 · Cited by 1,515
EFRON, B., & HINKLEY, D. V. (1978) · Biometrika · doi:10.1093/biomet/65.3.457 · Cited by 952
Multilevel mixed linear model analysis using iterative generalized least squares
GOLDSTEIN, H. (1986) · Biometrika · doi:10.1093/biomet/73.1.43 · Cited by 802
Cited by
Fixed and random effects models: making an informed choice
Bell, A., Fairbrother, M., & Jones, K. (2019) · Quality & Quantity · doi:10.1007/s11135-018-0802-x · Cited by 1,085
Kuhlman, S. T. W., Smith, P. N., Marie, L., Fadoir, N. A., & Hudson, K. (2021) · Archives of Suicide Research · doi:10.1080/13811118.2020.1767246 · Cited by 25
Stiegemeier, D., Bringeland, S., Kraus, J., & Baumann, M. (2022) · Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/3543174.3546847 · Cited by 14
Baweja, R., Waschbusch, D. A., Pelham, W. E., Pelham, W. E., & Waxmonsky, J. G. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.699687 · Cited by 11
Thompson, E. M., Albertella, L., Viskovich, S., Pakenham, K. I., & Fontenelle, L. F. (2024) · Behaviour Research and Therapy · doi:10.1016/j.brat.2024.104595 · Cited by 8
Metadata
- Authors
- McNeish, D.
- Venue
- Multivariate Behavioral Research
- Publisher
- Informa UK Limited
- Published
- 2017-07-17
- Volume
- 52
- Issue
- 5
- Pages
- 661-670
- ISSN
- 0027-3171 (ISSN-L)
- 1532-7906
- OpenAlex
- W2734608070
- Cited by
- 278
- References
- 31
- FWCI
- 9.80
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction. https://app.cochat.ai/papers/10.1080/00273171.2017.1344538
- The paper
- McNeish, D. (2017). Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2017.1344538
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.