Selecting the Best Linear Mixed Model Under REML
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Restricted maximum likelihood (REML) estimation of the parameters of the mixed model has become commonplace, even becoming the default option in many statistical software packages. However, a review of the literature indicates a need to update and clarify model selection techniques under REML, as ambiguities exist on the appropriateness of existing information criteria in this setting. A simulation study as well as an application assisted in gaining an understanding of the performance of information criteria in selecting the best model when using REML estimation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Various covariance models were compared using Akaike's Information Criterion to determine the best fit to the data (Gurka 2006), taking into account correlations between observations over time. The best-fitting structure included unstructured covariance matrix.”
From Longitudinal association between social media use and psychological distress among adolescents (2020) · cites this paper as Gurka 2006
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879 · cited 2×
“Model selection tools such as Akaike’s information criterion, or AIC (Akaike 1974), the corrected AIC, or AICC (Hurvich and Tsai 1989), the consistent AIC, or CAIC (Bozdogan 1987), and Schwarz’s Bayesian information criterion, or BIC (Schwarz 1978), are also computed automatically”
MethodEstimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033 · cited 2×
“Model selection tools such as Akaike’s information criterion, or AIC (Akaike 1974), the corrected AIC, or AICC (Hurvich and Tsai 1989), the consistent AIC, or CAIC (Bozdogan 1987), and Schwarz’s Bayesian information criterion, or BIC (Schwarz 1978), are also computed automatically”
MethodModel selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495 · cited 2×
“Additionally, the area of model selection has received increased attention in recent years as datasets and the models that analyze them have become more and more complex (Burnham and Anderson 2002).”
BackgroundKass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Regression and time series model selection in small samples
HURVICH, C. M., & TSAI, C.-L. (1989) · Biometrika · doi:10.1093/biomet/76.2.297 · Cited by 6,586 · cited 14×
“Model selection tools such as Akaike’s information criterion, or AIC (Akaike 1974), the corrected AIC, or AICC (Hurvich and Tsai 1989), the consistent AIC, or CAIC (Bozdogan 1987), and Schwarz’s Bayesian information criterion, or BIC (Schwarz 1978), are also computed automatically Matthew J. Gurka is Assistant Profess…”
BackgroundMethodInfluentialBozdogan, H. (1987) · Psychometrika · doi:10.1007/bf02294361 · Cited by 4,468 · cited 2×
“…tools such as Akaike’s information criterion, or AIC (Akaike 1974), the corrected AIC, or AICC (Hurvich and Tsai 1989), the consistent AIC, or CAIC (Bozdogan 1987), and Schwarz’s Bayesian information criterion, or BIC (Schwarz 1978), are also computed automatically Matthew J. Gurka is Assistant…”
MethodKass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,312
Linear and Nonlinear Models for the Analysis of Repeated Measurements
Vonesh, E., & Chinchilli, V. M. (1996) · CRC Press eBooks · doi:10.1201/9781482293272 · Cited by 989 · cited 2×
“Table 1 displays the formulas for the AIC, AICC, CAIC, and BIC from both angles, based on formulas familiar to readers of Vonesh and Chinchilli (1997).”
BackgroundMethodCovariance structure selection in general mixed models
Wolfinger, R. (1993) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919308813143 · Cited by 748 · cited 2×
“For such a comparison, the employment of information criteria calculated from maximum likelihood (ML) parameter estimates is recommended, even after first fitting the model using REML estimation (Wolfinger 1993).”
MethodBayesian inference for variance components using only error contrasts
HARVILLE, D. A. (1974) · Biometrika · doi:10.1093/biomet/61.2.383 · Cited by 547 · cited 3×
“When viewed in the formulation of Harville (1974), the restricted likelihood (3) depends on the terms from the originally noted model (1), rather than a model of a transformation of the original observations.”
BackgroundMethodRegression and Time Series Model Selection
McQuarrie, A. D. R., & Tsai, C.-L. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/3573 · Cited by 513 · cited 2×
“Model selection tools such as Akaike’s information criterion, or AIC (Akaike 1974), the corrected AIC, or AICC (Hurvich and Tsai 1989), the consistent AIC, or CAIC (Bozdogan 1987), and Schwarz’s Bayesian information criterion, or BIC (Schwarz 1978), are also computed automatically”
BackgroundMethodLikelihood Ratio Tests for Fixed Model Terms using Residual Maximum Likelihood
Welham, S. J., & Thompson, R. (1997) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00092 · Cited by 146 · cited 1×
“Welham and Thompson (1997) introduced an adjusted likelihood-ratio test for the fixed effects under REML.”
MethodRegression Model Selection—A Residual Likelihood Approach
Shi, P., & Tsai, C.-L. (2002) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00335 · Cited by 123 · cited 2×
“Additionally, Shi and Tsai (2002) raised the point that information criteria under ML are calculated from biased estimators and hence may not be suitable.”
BackgroundRegression and time series model selection using variants of the schwarz information criterion
Neath, A. A., & Cavanaugh, J. E. (1997) · Communication in Statistics- Theory and Methods · doi:10.1080/03610929708831934 · Cited by 98 · cited 1×
“This constant term has appeared in a previous comparison of different versions of the BIC (Neath and Cavanaugh 1997), and its inclusion will be assessed here as well.”
BackgroundTesting transformations for the linear mixed model
Gurka, M. J., Edwards, L. J., & Nylander‐French, L. A. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.05.018 · Cited by 10
Self-citation
Cited by
On the unnecessary ubiquity of hierarchical linear modeling.
McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017) · Psychological Methods · doi:10.1037/met0000078 · Cited by 742
“This topic has been recently researched (e.g., Gurka, 2006; Lukočienė, Varriale, & Vermunt, 2010; Whittaker & Furlow, 2009) although no consensus has been reached.”
BackgroundJanssen, I. (2008) · Archives of Internal Medicine · doi:10.1001/archinte.168.14.1568 · Cited by 514
“Goodness of fit was assessed with the Bayes information criterion.(35) An autoregressive covariance structure was used in all models to account for the dependence of measures over time.”
MethodNew Important Developments in Small Area Estimation
Pfeffermann, D. (2013) · Statistical Science · doi:10.1214/12-sts395 · Cited by 407
“Gurka (2006) validates by simulations that one can use also in this case the mAIC with ψ̂REML, despite the use of different fixed effects design matrices under different models.”
MethodVAN STRIEN, M. J., KELLER, D., & HOLDEREGGER, R. (2012) · Molecular Ecology · doi:10.1111/j.1365-294x.2012.05687.x · Cited by 214
“The inclusion of (nonsignificant) explanatory variables can result in a decrease in the total variance estimate by REML, which can increase the R2b estimate.”
MethodInfluentialFixed and Random Effects Selection in Mixed Effects Models
Ibrahim, J. G., Zhu, H., García, R. I. L., & Guo, R. (2010) · Biometrics · doi:10.1111/j.1541-0420.2010.01463.x · Cited by 206
“…appropriate penalty parameters to produce estimates having proper asymptotic properties (Fan and Li, 2001), whereas existing selection criteria (Kowalchuck et al., 2004; Gurka, 2006; Claeskens and Con-sentino, 2008; Liang et al., 2008) are computationally difficult for general mixed effects models.”
BackgroundMethod
Metadata
- Authors
- Gurka, M. J.
- Affiliations
- University of Virginia (US)
- Corresponding author
- Gurka, Matthew J
- ORCID
- Gurka, Matthew J
- Venue
- The American Statistician
- Publisher
- Informa UK Limited
- Published
- 2006-02
- Volume
- 60
- Issue
- 1
- Pages
- 19-26
- ISSN
- 0003-1305 (ISSN-L)
- 1537-2731
- OpenAlex
- W2124269493
- Cited by
- 200
- References
- 14
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 8 · 2023: 11 · 2022: 13 · 2021: 6 · 2020: 14 · 2019: 14 · 2018: 12 · 2017: 7 · 2016: 5 · 2015: 9 · 2014: 10 · 2013: 9 · 2012: 27
- FWCI
- 6.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The American Statistician (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Keywords
- linear mixed models, restricted maximum likelihood, model selection, information criteria
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.