Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Restricted maximum likelihood (REML) is now well established as a method for estimating the parameters of the general Gaussian linear model with a structured covariance matrix, in particular for mixed linear models. Conventionally, estimates of precision and inference for fixed effects are based on their asymptotic distribution, which is known to be inadequate for some small-sample problems. In this paper, we present a scaled Wald statistic, together with an F approximation to its sampling distribution, that is shown to perform well in a range of small sample settings. The statistic uses an adjusted estimator of the covariance matrix that has reduced small sample bias. This approach has the advantage that it reproduces both the statistics and F distributions in those settings where the latter is exact, namely for Hotelling T2 type statistics and for analysis of variance F-ratios. The performance of the modified statistics is assessed through simulation studies of four different REML analyses and the methods are illustrated using three examples.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Accurate model parameter estimations were ensured by use of the residual maximum likelihood approach with the Kenward and Roger, 1997 approximation of degrees of freedom.”
From Longitudinal association between social media use and psychological distress among adolescents (2020) · cites this paper as Kenward and Roger, 1997
In the citing paper
“Not only does the data in the motivating example have different flowering days for each genotype (see for example Fig. 2), but the proposed methodology also accounts for the different strata in the Wald test by approximating the denominator degrees of freedom (Kenward and Roger, 1997).”
From Incorporating environmental covariates to explore genotype × environment × management (G × E × M) interactions: A one-stage predictive model (2023) · cites this paper as Kenward and Roger, 1997
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Random-Effects Models for Longitudinal Data
Laird, N. M., & Ware, J. H. (1982) · Biometrics · doi:10.2307/2529876 · Cited by 8,923
Recovery of inter-block information when block sizes are unequal
PATTERSON, H. D., & THOMPSON, R. (1971) · Biometrika · doi:10.1093/biomet/58.3.545 · Cited by 4,058
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667 · cited 1×
“Restricted (or residual) maximum likelihood (REML) has become well established for the estimation of the parameters of Z (Patterson and Thompson, 1971; Harville, 1977; Thompson, 1980; Robinson, 1991) and the corresponding estimator of 3, 3 say, has a generalized least squares form that uses the REML estimate of Z.”
BackgroundThat BLUP is a Good Thing: The Estimation of Random Effects
Robinson, G. K. (1991) · Statistical Science · doi:10.1214/ss/1177011926 · Cited by 1,686
Gilmour, A. R., Thompson, R., & Cullis, B. R. (1995) · Biometrics · doi:10.2307/2533274 · Cited by 915
15 Design and Analysis of Cross-Over Trials
Kenward, M. G., & Jones, B. (2007) · Handbook of Statistics · doi:10.1016/s0169-7161(07)27015-4 · Cited by 871
Self-citationPrinciples of Multivariate Analysis
Ling, R. F. (1990) · Technometrics · doi:10.1080/00401706.1990.10484645 · Cited by 812
Satterthwaite, F. E. (1941) · Psychometrika · doi:10.1007/bf02288586 · Cited by 809
Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477102 · Cited by 445
Cited by
Items 1,001–1,051 of 1,051
Ito, M., Horikoshi, M., Kato, N., Oe, Y., Fujisato, H., Nakajima, S., Kanie, A., Miyamae, M., Takebayashi, Y., Horita, R., et al. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.02.005 · Cited by 39
Do-Not-Resuscitate Orders in the Last Days of Life
Bailey, F. A., Allen, R. S., Williams, B. R., Goode, P. S., Granstaff, S., Redden, D. T., & Burgio, K. L. (2012) · Journal of Palliative Medicine · doi:10.1089/jpm.2011.0321 · Cited by 37
“26 Separate bivariate models were constructed to examine the relationships between each of the outcomes and each potential predictor.”
Tan, K. S., Libon, D. J., Rascovsky, K., Grossman, M., & Xie, S. X. (2013) · Alzheimer Disease & Associated Disorders · doi:10.1097/wad.0b013e31827bdc6f · Cited by 36
Marcato, F., van den Brand, H., Kemp, B., Engel, B., Schnabel, S. K., Hoorweg, F. A., Wolthuis-Fillerup, M., & van Reenen, K. (2022) · Journal of Dairy Science · doi:10.3168/jds.2021-20637 · Cited by 35
Population regulation of African buffalo in the Mara–Serengeti ecosystem
Dublin, H. T., & Ogutu, J. O. (2015) · Wildlife Research · doi:10.1071/wr14205 · Cited by 34
“The denominator degrees of freedom for testing for the significance of the transition in mean population size in 1994 was synthesised using the method of Kenward and Roger (1997).”
MethodAnalysis of Data from Non-Orthogonal Multistratum Designs in Industrial Experiments
Gilmour, S. G., & Goos, P. (2009) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2009.00662.x · Cited by 31
“…differences, R with the default options, GenStat using the average information optimization criterion option, SAS with the Kenward–Roger method (Kenward and Roger, 1997) for obtaining the degrees of freedom and JMP7 all gave the same estimates and standard errors, with σ̂2B = 0, σ̂2 = 6:495…”
MethodBeymer, P. N., Rosenberg, J. M., & Schmidt, J. A. (2020) · Learning and Individual Differences · doi:10.1016/j.lindif.2019.101812 · Cited by 31
Response of Fusarium thapsinum to Sorghum brown midrib Lines and to Phenolic Metabolites
Funnell-Harris, D. L., Sattler, S. E., & Pedersen, J. F. (2014) · Plant Disease · doi:10.1094/pdis-09-13-0980-re · Cited by 29
“The KENWARD-ROGER option was specified to calculate denominator degrees of freedom because of heterogeneous variances and to reduce standard error and F-statistic bias (25,30).”
Marcato, F., van den Brand, H., Kemp, B., Engel, B., Wolthuis-Fillerup, M., & van Reenen, K. (2020) · Frontiers in Veterinary Science · doi:10.3389/fvets.2020.576469 · Cited by 30
“Here and in subsequent analyses, for all fixed effects, approximate F-tests were used (20).”
MethodL. Varga, N., Gaugler, T., & Talarico, J. (2019) · Memory & Cognition · doi:10.3758/s13421-018-0887-4 · Cited by 28
Storage of Miscanthus -derived carbon in rhizomes, roots, and soil
Christensen, B. T., Lærke, P. E., Jørgensen, U., Kandel, T. P., & Thomsen, I. K. (2016) · Canadian Journal of Soil Science · doi:10.1139/cjss-2015-0135 · Cited by 29
Moritz, S., Wess, N., Treszl, A., & Jelinek, L. (2011) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-010-9169-6 · Cited by 29
(2014) · General Linear Model Journal · Cited by 28
Jackson, M. L., Larry Williams, W., Hayes, S. C., Humphreys, T., Gauthier, B., & Westwood, R. (2016) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2015.11.002 · Cited by 25
Supporting Metacognition during Exploratory Search with the OrgBox
Crescenzi, A., Ward, A. R., Li, Y., & Capra, R. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462955 · Cited by 25
“Restricted maximum likelihood was used with Kenward-Rogers degrees-of-freedom correction [24] for small sample inference.”
MethodMarcato, F., van den Brand, H., Kemp, B., Engel, B., Schnabel, S. K., Jansen, C. A., Rutten, V. P. M. G., Koets, A. P., Hoorweg, F. A., de Vries-Reilingh, G., et al. (2022) · Journal of Dairy Science · doi:10.3168/jds.2021-20636 · Cited by 24
Michiels, B., Tanious, R., De, T. K., & Onghena, P. (2020) · Behavior Research Methods · doi:10.3758/s13428-019-01266-6 · Cited by 23
Mumford, M. H., Forknall, C. R., Rodriguez, D., Eyre, J. X., & Kelly, A. M. (2023) · Field Crops Research · doi:10.1016/j.fcr.2023.109133 · Cited by 22
Funnell-Harris, D. L., O’Neill, P. M., Sattler, S. E., Gries, T., Berhow, M. A., & Pedersen, J. F. (2017) · European Journal of Plant Pathology · doi:10.1007/s10658-017-1148-2 · Cited by 20
“The KENWARD-ROGER option was specified to calculate denominator degrees of freedom to reduce standard error and F-statistic bias (Kenward and Roger 1997; Littell et al. 2006).”
MethodMarcato, F., van den Brand, H., Jansen, C. A., Rutten, V. P. M. G., Kemp, B., Engel, B., Wolthuis-Fillerup, M., & van Reenen, K. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246959 · Cited by 21
Majidi, S., Reid, M. W., Fogel, J., Anderson, B., Klingensmith, G. J., Cain, C., Berget, C., & Raymond, J. K. (2021) · Pediatric Diabetes · doi:10.1111/pedi.13212 · Cited by 21
“Changes in mean scores on clinical and psychosocial outcomes over the study were compared between groups using linear mixed-effects models adjusted for age and sex and accounting for baseline values, fit via restricted maximum likelihood, and modified for small sample inference using the degrees of freedom calculation…”
MethodMallapur, A., Miller, C., Christman, M. C., & Estevez, I. (2009) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2008.11.005 · Cited by 21
“We used the Kenward–Roger method to adjust the degrees of freedom due to the random effect in the model (Kenward and Roger, 1997).”
MethodAtypical Emotional Electrodermal Activity in Toddlers with Autism Spectrum Disorder
Vernetti, A., Shic, F., Boccanfuso, L., Macari, S., Kane‐Grade, F., Milgramm, A., Hilton, E., Heymann, P., Goodwin, M. S., & Chawarska, K. (2020) · Autism Research · doi:10.1002/aur.2374 · Cited by 20
Argaw, T., Fenta, B. A., Zegeye, H., Azmach, G., & Funga, A. (2025) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2025.1472282 · Cited by 19
Kim, J. Y., Lester, C., & Yang, X. J. (2025) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251326795 · Cited by 21
“The Kenward-Roger method of estimating degrees of freedom was used, which adjusts the variance-covariance matrix of the fi xed effects using a Taylor series expansion, then calculates degrees of freedom (Kenward & Roger, 1997).”
Chaput, J.-P., Weippert, M., LeBlanc, A. G., Hjorth, M. F., Michaelsen, K. F., Katzmarzyk, P. T., Tremblay, M. S., Barreira, T. V., Broyles, S. T., Fogelholm, M., et al. (2016) · Frontiers in Pediatrics · doi:10.3389/fped.2016.00024 · Cited by 19
“Denominator degrees of freedom for statistical tests pertaining to fixed effects were calculated using the Kenward and Roger approximation method (14).”
MethodMartins, J., Rosário, P., Cunha, J., Carlos Núñez, J., Vallejo, G., & Moreira, T. (2024) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102230 · Cited by 24
Altered markers of stress in depressed adolescents after acute social media use
Shafi, R. M. A., Nakonezny, P. A., Miller, K. A., Desai, J., Almorsy, A. G., Ligezka, A. N., Morath, B. A., Romanowicz, M., & Croarkin, P. E. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.01.055 · Cited by 19
“Restricted maximum likelihood estimation and type 3 tests of fixed effects were used, with the Kenward-Roger correction applied to the compound symmetry covariance structure (Kenward and Roger, 1997).”
MethodMoritz, S., Hauschildt, M., Murray, S. C., Pedersen, A., Krausz, M., & Jelinek, L. (2018) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2017.12.009 · Cited by 18
De Paepe, A. L., Crombez, G., & Legrain, V. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.05.015 · Cited by 18
“Kenward-Roger approximations to the degrees of freedom were used to adjust for small sample sizes (Kenward & Roger, 1997).”
Pappa, E., Baah, F., Lynch, J., Shiel, L., Blackman, G., Raihani, N., & Bell, V. (2025) · Schizophrenia Bulletin · doi:10.1093/schbul/sbae225 · Cited by 19
“24 A Freeman–Tukey double arcsine transformed proportion was employed to stabilize the variance due to the potential for low estimates of prevalence for some themes.”
Zvomuya, F., Larney, F. J., DeMaere, P. R., & Olson, A. F. (2007) · Soil Science Society of America Journal · doi:10.2136/sssaj2006.0370 · Cited by 16
Rosales, C. B., Denman, C. A., Bell, M. L., Cornejo, E., Ingram, M., del Carmen Castro Vásquez, M., Gonzalez-Fagoaga, J. E., Aceves, B., Nuño, T., Anderson, E. J., et al. (2021) · International Journal of Epidemiology · doi:10.1093/ije/dyab072 · Cited by 15
Long-term historical and projected herbivore population dynamics in Ngorongoro crater, Tanzania
Moehlman, P. D., Ogutu, J. O., Piepho, H.-P., Runyoro, V. A., Coughenour, M. B., & Boone, R. B. (2020) · PLOS ONE · doi:10.1371/journal.pone.0212530 · Cited by 14
Stiegemeier, D., Bringeland, S., Kraus, J., & Baumann, M. (2022) · Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/3543174.3546847 · Cited by 14
“Accounting for the relatively small sample size, multi-level models were estimated with REML and Kenward-Roger correction [29, 39].”
MethodRandom regression models for multi-environment, multi-time data from crop breeding selection trials
De Faveri, J., Verbyla, A. P., & Rebetzke, G. (2023) · Crop & Pasture Science · doi:10.1071/cp21732 · Cited by 12
Argaw, T., Fenta, B. A., & Assefa, E. (2024) · PLOS ONE · doi:10.1371/journal.pone.0301534 · Cited by 11
Sano, H., Hara, A., Asayama, K., Miyazaki, S., Kikuya, M., Imai, Y., & Ohkubo, T. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040524 · Cited by 11
Zvomuya, F., Larney, F. J., DeMaere, P. R., & Olson, A. F. (2008) · Nutrient Cycling in Agroecosystems · doi:10.1007/s10705-007-9137-z · Cited by 10
“Mixed-model F-tests based on the Kenward-Roger (Kenward and Roger 1997) adjusted denominator degrees of freedom approximation were used to assess significance of fixed effects and interactions in the model.”
MethodParent Perspectives after the PRISM-P Randomized Trial: A Mixed-Methods Analysis
Rosenberg, A. R., Zhou, C., Bradford, M. C., Barton, K., Junkins, C. C., Taylor, M., Kross, E. K., Curtis, J. R., Dionne-Odom, J. N., & Yi-Frazier, J. P. (2021) · Journal of Palliative Medicine · doi:10.1089/jpm.2020.0720 · Cited by 9
“20 Missing data were assumed missing at random (MAR) and handled with multiple imputation.”
A power approximation for the Kenward and Roger Wald test in the linear mixed model
Kreidler, S. M., Ringham, B. M., Muller, K. E., & Glueck, D. H. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254811 · Cited by 9
Testing for Lack of Fit in Blocked, Split-Plot, and Other Multi-Stratum Designs
Goos, P., & Gilmour, S. G. (2017) · Journal of Quality Technology · doi:10.1080/00224065.2017.11918000 · Cited by 9
Cowan, J. S., Saxton, A. M., Liu, H., Leonas, K. K., Inglis, D., & Miles, C. A. (2016) · HortScience · doi:10.21273/hortsci.51.3.245 · Cited by 8
Landburg, G., Amatamsir, C., Muys, B., & Hermy, M. (2021) · Journal of Forest Research · doi:10.1080/13416979.2021.1913305 · Cited by 8
Moritz, S., Bernardini, J., & Lion, D. (2020) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-019-01080-3 · Cited by 7
Linear and Non-Linear Regression Methods in Epidemiology and Biostatistics
Vittinghoff, E., McCulloch, C. E., Glidden, D. V., & Shiboski, S. C. (2011) · Essential Statistical Methods for Medical Statistics · doi:10.1016/b978-0-444-53737-9.50006-2 · Cited by 6
Siegmann, S., van Dijk, L. L., Field, N. L., Sayers, G., Sugrue, K., Gort, G., van Reenen, C. G., Bokkers, E. A. M., & Conneely, M. (2025) · Journal of Dairy Science · doi:10.3168/jds.2024-26219 · Cited by 6
Supplementation with selenium, iron, and vitamin E in calves under immunological challenge
Salles, M. S. V., Figueiroa, F. J. F., Bittar, C. M. M., Gomes, V., Marques, R. S., da Silveira, J. A. G., Facury Filho, E. J., de Freitas, J. E., & Saran Netto, A. (2025) · Frontiers in Animal Science · doi:10.3389/fanim.2025.1540495 · Cited by 5
Sauer, K. S., Bailer, J., Jungmann, S. M., & Witthöft, M. (2023) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2022.08.001 · Cited by 3
DECoVaC: Design of Experiments with Controlled Variability Components
Boquet, T., Delisle, L., Kochetkov, D., Schucher, N., Atighehchian, P., Oreshkin, B., & Cornebise, J. (2019) · arXiv · doi:10.48550/arxiv.1909.09859 · Cited by 1
“We can use an ANOVA with a correction for the degrees of freedom for the number of comparisons performed (Kenward & Roger, 1997).”
MethodExplaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Metadata
- Authors
- Kenward, M. G., & Roger, J. H.
- Affiliations
- University of Kent (GB)
- Venue
- Biometrics
- Publisher
- JSTOR
- Published
- 1997-09
- Volume
- 53
- Issue
- 3
- Pages
- 983
- ISSN
- 0006-341X (ISSN-L)
- OpenAlex
- W1981747359
- Cited by
- 4,552
- References
- 23
- Citations by year
- 2026: 217 · 2025: 196 · 2024: 216 · 2023: 213 · 2022: 261 · 2021: 299 · 2020: 287 · 2019: 270 · 2018: 247 · 2017: 240 · 2016: 286 · 2015: 256 · 2014: 217 · 2013: 189 · 2012: 218
- FWCI
- 7.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrics (publishedVersion)
- PubMed (publishedVersion)
- Rothamsted Repository (Rothamsted Repository) (submittedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Applied Mathematics › Statistical and numerical algorithms
- Keywords
- restricted maximum likelihood, small sample inference, fixed effects, Wald statistic, covariance matrix estimation, small sample bias, Hotelling's T2 statistic, linear mixed models, linear Gaussian models
- MeSH terms
- Analysis of Variance, Biometry, Clinical Trials as Topic, Humans, Intermittent Claudication, Nitrogen Fixation, Plants, Probability, Regression Analysis, Research Design, Rhizobium, Soil Microbiology, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Cross-Over Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.