Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Restricted maximum likelihood (REML) is now well established as a method for estimating the parameters of the general Gaussian linear model with a structured covariance matrix, in particular for mixed linear models. Conventionally, estimates of precision and inference for fixed effects are based on their asymptotic distribution, which is known to be inadequate for some small-sample problems. In this paper, we present a scaled Wald statistic, together with an F approximation to its sampling distribution, that is shown to perform well in a range of small sample settings. The statistic uses an adjusted estimator of the covariance matrix that has reduced small sample bias. This approach has the advantage that it reproduces both the statistics and F distributions in those settings where the latter is exact, namely for Hotelling T2 type statistics and for analysis of variance F-ratios. The performance of the modified statistics is assessed through simulation studies of four different REML analyses and the methods are illustrated using three examples.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Accurate model parameter estimations were ensured by use of the residual maximum likelihood approach with the Kenward and Roger, 1997 approximation of degrees of freedom.”
From Longitudinal association between social media use and psychological distress among adolescents (2020) · cites this paper as Kenward and Roger, 1997
In the citing paper
“Not only does the data in the motivating example have different flowering days for each genotype (see for example Fig. 2), but the proposed methodology also accounts for the different strata in the Wald test by approximating the denominator degrees of freedom (Kenward and Roger, 1997).”
From Incorporating environmental covariates to explore genotype × environment × management (G × E × M) interactions: A one-stage predictive model (2023) · cites this paper as Kenward and Roger, 1997
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Random-Effects Models for Longitudinal Data
Laird, N. M., & Ware, J. H. (1982) · Biometrics · doi:10.2307/2529876 · Cited by 8,923
Recovery of inter-block information when block sizes are unequal
PATTERSON, H. D., & THOMPSON, R. (1971) · Biometrika · doi:10.1093/biomet/58.3.545 · Cited by 4,058
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667 · cited 1×
“Restricted (or residual) maximum likelihood (REML) has become well established for the estimation of the parameters of Z (Patterson and Thompson, 1971; Harville, 1977; Thompson, 1980; Robinson, 1991) and the corresponding estimator of 3, 3 say, has a generalized least squares form that uses the REML estimate of Z.”
BackgroundThat BLUP is a Good Thing: The Estimation of Random Effects
Robinson, G. K. (1991) · Statistical Science · doi:10.1214/ss/1177011926 · Cited by 1,686
Gilmour, A. R., Thompson, R., & Cullis, B. R. (1995) · Biometrics · doi:10.2307/2533274 · Cited by 915
15 Design and Analysis of Cross-Over Trials
Kenward, M. G., & Jones, B. (2007) · Handbook of Statistics · doi:10.1016/s0169-7161(07)27015-4 · Cited by 871
Self-citationPrinciples of Multivariate Analysis
Ling, R. F. (1990) · Technometrics · doi:10.1080/00401706.1990.10484645 · Cited by 812
Satterthwaite, F. E. (1941) · Psychometrika · doi:10.1007/bf02288586 · Cited by 809
Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477102 · Cited by 445
Cited by
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
“Thus approximate methods for computing the approximate degrees of freedom for t distributions, or the denominator degrees of freedom for F statistics (Satterthwaite 1946; Kenward and Roger 1997), are at best ad hoc solutions.”
MethodElexacaftor–Tezacaftor–Ivacaftor for Cystic Fibrosis with a Single Phe508del Allele
Middleton, P. G., Mall, M. A., Dřevínek, P., Lands, L. C., McKone, E. F., Polineni, D., Ramsey, B. W., Taylor-Cousar, J. L., Tullis, E., Vermeulen, F., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1908639 · Cited by 2,198
Khuri, A. I., & Mukhopadhyay, S. (2010) · WIREs Computational Statistics · doi:10.1002/wics.73 · Cited by 2,113
Evaluating significance in linear mixed-effects models in R
Luke, S. G. (2017) · Behavior Research Methods · doi:10.3758/s13428-016-0809-y · Cited by 1,704
“3 Both the Kenward-Roger (Kenward & Roger, 1997) and Satterthwaite (1941) approaches are used to estimate denominator degrees of freedom for F statistics or degrees of freedom for t statistics.”
MethodFatigue and cognitive impairment in Post-COVID-19 Syndrome: a systematic review and meta-analysis
Ceban, F., Ling, S., Lui, L. M. W., Lee, Y., Gill, H., Teopiz, K. M., Rodrigues, N. B., Subramaniapillai, M., Di Vincenzo, J. D., Cao, B., et al. (2022) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2021.12.020 · Cited by 1,554
“In anticipation of marked heterogeneity, the meta::metaprop function (Balduzzi et al., 2019) was used to pool proportions, indicated as the number of cases exhibiting fatigue or cognitive impairment (events) divided by the size of the sample (observations), via the random-effects restricted maximum-likelihood model (R…”
Robustness of linear mixed-effects models to violations of distributional assumptions
Schielzeth, H., Dingemanse, N. J., Nakagawa, S., Westneat, D. F., Allegue, H., Teplitsky, C., Réale, D., Dochtermann, N. A., Garamszegi, L. Z., & Araya‐Ajoy, Y. G. (2020) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13434 · Cited by 1,393
“There are approaches for obtaining robust estimators for the fixed effects for cases of small sample size and for models with violated distributional assumptions (e.g. Kenward & Roger, 1997; Kenward & Roger, 2009; Royall, 1986; Royall & Tsou, 2003).”
BackgroundUsing simulation studies to evaluate statistical methods
Morris, T. P., White, I. R., & Crowther, M. J. (2019) · Statistics in Medicine · doi:10.1002/sim.8086 · Cited by 1,305
“The sample estimate V̂ar(̂) should be a consistent estimate of the sampling variance of ̂ (see for example (18)).”
BackgroundJudd, C. M., Westfall, J., & Kenny, D. A. (2012) · Journal of Personality and Social Psychology · doi:10.1037/a0028347 · Cited by 1,236
Halekoh, U., & Højsgaard, S. (2014) · Journal of Statistical Software · doi:10.18637/jss.v059.i09 · Cited by 1,324
“Throughout this paper we shall refer to Kenward and Roger (1997) as KR. (b) The second contribution of this paper is to determine either the full null distribution or moments of the null distribution of the LR test statistic (3) by a parametric bootstrap approach (Davison and Hinkley 1997, Chapter…”
BackgroundMethodInfluentialUsing simulation studies to evaluate statistical methods
Morris, T., White, I. R., & Crowther, M. (2019) · UCL Discovery (University College London) · Cited by 1,011
Multilevel Modelling of Country Effects: A Cautionary Tale
Bryan, M. L., & Jenkins, S. P. (2016) · European Sociological Review · doi:10.1093/esr/jcv059 · Cited by 866
“More well known are the small-sample corrections to SEs with associated sample size adjustments that have been developed for the REML estimator (Kenward and Roger, 1997, 2009), and which are available in SAS’s PROC MIXED and R’s ASREML package.”
Method15 Design and Analysis of Cross-Over Trials
Kenward, M. G., & Jones, B. (2007) · Handbook of Statistics · doi:10.1016/s0169-7161(07)27015-4 · Cited by 871
Self-citationLongitudinal and Panel Data: Analysis and Applications in the Social Sciences
Frees, E. W. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511790928 · Cited by 839
Accounting for natural and extraneous variation in the analysis of field experiments
Gilmour, A. R., Cullis, B. R., Verbyla, A. P., & Verbyla, A. P. (1997) · Journal of Agricultural, Biological, and Environmental Statistics · doi:10.2307/1400446 · Cited by 752
“Kenward and Roger (1997) have examinedthis issue for the testing of xed e ects in REML.4 ExamplesIn this section we present the analysis of two examples.”
BackgroundMethodMultivariate meta-analysis for non-linear and other multi-parameter associations
Gasparrini, A., Armstrong, B., & Kenward, M. G. (2012) · Statistics in Medicine · doi:10.1002/sim.5471 · Cited by 772
“Suitable adjustments for the underestimation of (co)variance error matrices due to the uncertainty in the estimation of ‰ are available [35, 36] although not yet been implemented in mvmeta.”
The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration
McNeish, D. M., & Stapleton, L. M. (2016) · Educational Psychology Review · doi:10.1007/s10648-014-9287-x · Cited by 715
“Kenward and Roger (1997, 2009) showed by proof, simulation, and real-data example that their denominator degrees of freedom and covariance matrix adjustment can greatly reduce the underestimation of fixed-effect standard errors when the number of clusters is small in continuous outcome models with…”
BackgroundMethodInfluentialAddolorato, G., Leggio, L., Ferrulli, A., Cardone, S., Vonghia, L., Mirijello, A., Abenavoli, L., D'Angelo, C., Caputo, F., Zambon, A., et al. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61814-5 · Cited by 685
Long-term population declines in Afro-Palearctic migrant birds
Sanderson, F. J., Donald, P. F., Pain, D. J., Burfield, I. J., & van Bommel, F. P. J. (2006) · Biological Conservation · doi:10.1016/j.biocon.2006.02.008 · Cited by 669
Gunnar, M. R., Wewerka, S., Frenn, K., Long, J. D., & Griggs, C. (2009) · Development and Psychopathology · doi:10.1017/s0954579409000054 · Cited by 668
“In addition, the Kenward and Roger (1997) method of estimated degrees of freedom for small sample size were used in all hypothesis tests.”
MethodAn R2 statistic for fixed effects in the linear mixed model
Edwards, L. J., Muller, K. E., Wolfinger, R. D., Qaqish, B. F., & Schabenberger, O. (2008) · Statistics in Medicine · doi:10.1002/sim.3429 · Cited by 627
“Under REML estimation, the Kenward-Roger F [19] apparently provides the most accurate inference in small samples, while the Satterthwaite method also does well [20].”
MethodExtending DerSimonian and Laird's methodology to perform multivariate random effects meta‐analyses
Jackson, D., White, I. R., & Thompson, S. G. (2010) · Statistics in Medicine · doi:10.1002/sim.3602 · Cited by 594
“The standard use of the estimated between-study variance matrix as the true matrix when pooling the studies’ results will be adopted, although Kenward and Roger [14] give small sample procedures where REML is used.”
MethodRubin, M. R., Bilezikian, J. P., McMahon, D. J., Jacobs, T., Shane, E., Siris, E., Udesky, J., & Silverberg, S. J. (2008) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2007-1215 · Cited by 593
“Estimated within-subject change in outcome between specific time points was taken from the differences between model estimated means for each time, after adjustment of the degrees of freedom, to test the statistical significance of the comparison (12).”
MethodOn Using Bayesian Methods to Address Small Sample Problems
McNeish, D. (2016) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2016.1186549 · Cited by 570
“…methods like restricted maximum likelihood (REML) estimation (Harville, 1977), the Kenward–Roger standard error and degree of freedom correction (Kenward & Roger, 1997, 2009), or Skene–Kenward corrections (Skene & Kenward, 2010a, 2010b) that have been found to perform well in the case of small…”
MethodThe analysis of designed experiments and longitudinal data by using smoothing splines
Verbyla, A. P., Cullis, B. R., Kenward, M. G., & Welham, S. J. (1999) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00154 · Cited by 570
“Kenward and Roger (1997) have provided methods for adjusting the standard errors of ®xed e ects in the general linear model, allowing for the estimation of variance components; these methods are computationally intensive, however.”
MethodSelf-citationViola, A. U., James, L. M., Schlangen, L. J., & Dijk, D.-J. (2008) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.1268 · Cited by 565
“All of the P-values were based on Kenward-Roger’s corrected degrees of freedom (30).”
MethodGilmour, A. R., Gogel, B. J., Cullis, B. R., Welham, S. J., & Thompson, R. (2002) · Rothamsted Repository (Rothamsted Repository) · Cited by 700
A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models
Chung, Y., Rabe-Hesketh, S., Dorie, V., Gelman, A., & Liu, J. (2013) · Psychometrika · doi:10.1007/s11336-013-9328-2 · Cited by 554
Brauer, M., & Curtin, J. J. (2018) · Psychological Methods · doi:10.1037/met0000159 · Cited by 555
Burke, D. L., Ensor, J., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7141 · Cited by 538
“However, there are also various methods for small-sample inference based on the t-distribution, also known as denominator-degrees-of-freedom adjustments, including Satterthwaite and Kenward-Roger [82].”
MethodAhles, T. A., Saykin, A. J., McDonald, B. C., Li, Y., Furstenberg, C. T., Hanscom, B. S., Mulrooney, T. J., Schwartz, G. N., & Kaufman, P. A. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2009.27.0827 · Cited by 535
“Hypothesis tests for the fixed effects were carried out by using the Wald F statistics and the Kenward-Roger adjustment as a precaution for inflated error variance in slightly unbalanced group sizes.(33,34) Linear combinations of fixed effects were evaluated by t tests.”
MethodMcCauley, E., Berk, M. S., Asarnow, J. R., Adrian, M., Cohen, J., Korslund, K., Avina, C., Hughes, J., Harned, M., Gallop, R., et al. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.1109 · Cited by 531
Multivariate meta‐analysis: Potential and promise
Jackson, D., Riley, R., & White, I. R. (2011) · Statistics in Medicine · doi:10.1002/sim.4172 · Cited by 530
“For example, Kenward and Roger [55] give small sample approximations for REML whose use in the context of meta-analysis awaits investigation.”
BackgroundPER3 polymorphism predicts sleep structure and waking performance
Viola, A. U., Archer, S. N., James, L. M., Groeger, J. A., Lo, J. C. Y., Skene, D. J., von Schantz, M., & Dijk, D.-J. (2007) · Current Biology · doi:10.1016/j.cub.2007.01.073 · Cited by 520
Cohen, L., Warneke, C., Fouladi, R. T., Rodriguez, M. A., & Chaoul‐Reich, A. (2004) · Cancer · doi:10.1002/cncr.20236 · Cited by 502
“02 of restricted maximum-likelihood estimation with profile residual variances was used; tests of model fixed-effects parameters were conducted using the Prasad-Rao-Jeske-Kackar-Harville method for obtaining fixed-effects standard errors, and the Kenward–Roger approach was used to calculate degrees of freedom; this ap…”
MethodAn Algorithm for a Letter-Based Representation of All-Pairwise Comparisons
Piepho, H.-P. (2004) · Journal of Computational and Graphical Statistics · doi:10.1198/1061860043515 · Cited by 484
“1 Heteroscedastic model analyzed by method of Kenward and Roger (1997) 2 GLM tted to proportions by quasi-likelihood with logit-link and variance function var( · ) = ¿ · (1 ¡ · ) (McCullagh and Nelder 1989, sec.”
MethodInfluentialStatistical analysis of fNIRS data: A comprehensive review
Tak, S., & Ye, J. C. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.06.016 · Cited by 465
“This idea is the so-called restricted maximum likelihood (ReML) method (Friston et al., 2011; Graser et al., 1987; Harville, 1977; Kenward and Roger, 1997; Searle, 1979).”
Design and Analysis of Quality of Life Studies in Clinical Trials
Fairclough, D. L. (2002) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420035438 · Cited by 461
An improved approximation to the precision of fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (2009) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2008.12.013 · Cited by 457
Self-citationLinear mixed-effects modeling approach to FMRI group analysis
Chen, G., Saad, Z. S., Britton, J. C., Pine, D. S., & Cox, R. W. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.01.047 · Cited by 447
“When economical and parsimonious assumptions such as compound symmetry or sphericity are violated, corrections such as methods proposed by Kenward and Roger (1997) inflate the estimated variances and then adjust the degrees of freedom through Satterthwaite (1946) correction.”
MethodDarwin's beautiful contrivances: evolutionary and functional evidence for floral adaptation
Harder, L. D., & Johnson, S. D. (2009) · New Phytologist · doi:10.1111/j.1469-8137.2009.02914.x · Cited by 439
Modeling Clustered Data with Very Few Clusters
McNeish, D., & Stapleton, L. M. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2016.1167008 · Cited by 430
“Although Ferron et al. (2009) generally found that a Kenward-Roger correction was able to estimate standard errors appropriately even for extremely small numbers of clusters, the data-generation model in this study was much larger, and so the slight dip in performance was anticipated (e.g., McNeish & Stapleton, 2014).”
BackgroundMethodInfluentialLongitudinal Change in CSF Biomarkers in Autosomal-Dominant Alzheimer’s Disease
Fagan, A. M., Xiong, C., Jasielec, M. S., Bateman, R. J., Goate, A. M., Benzinger, T. L. S., Ghetti, B., Martins, R. N., Masters, C. L., Mayeux, R., et al. (2014) · Science Translational Medicine · doi:10.1126/scitranslmed.3007901 · Cited by 434
Fixed effects, random effects and GEE: What are the differences?
Gardiner, J. C., Luo, Z., & Roman, L. A. (2009) · Statistics in Medicine · doi:10.1002/sim.3478 · Cited by 434
“Kenward–Roger and Satterthwaite) to account for the downward bias in standard errors due to the estimation of parameters in G and Ri and improve the accuracy of the F-test [35, 36].”
BackgroundMethodInfluentialMisra (2016) · doi:10.1177/0013916514539755 · Cited by 430
“The advantage of using this type of degree of freedom is the increased accuracy of the distribution of the test statistics because it accounts for the increased variability from the estimation of random effects, especially when the data are unbalanced (Kenward & Roger, 1997).”
BackgroundInitial phase 2 trial of a nicotinic agonist in schizophrenia
Freedman, R., Olincy, A., Buchanan, R. W., Harris, J. G., Gold, J. M., Johnson, L., Allensworth, D., Guzman-Bonilla, A., Clement, B., Ball, M. P., et al. (2008) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2008.07071135 · Cited by 426
“The Kenward and Roger method of calculating the denominator degrees of freedom was used (29).”
MethodEarly onset of spring increases the phenological mismatch between plants and pollinators
Kudo, G., & Ida, T. Y. (2013) · Ecology · doi:10.1890/12-2003.1 · Cited by 425
Attenuation of cortisol across development for victims of sexual abuse
Trickett, P. K., Noll, J. G., Susman, E. J., Shenk, C. E., & Putnam, F. W. (2010) · Development and Psychopathology · doi:10.1017/s0954579409990332 · Cited by 405
“Restricted maximum likelihood estimation with an estimated degrees of freedom procedure (Kenward & Roger, 1997) was used to arrive at valid parameter estimates under the assumption of ignorable missing data.”
MethodColor and emotion: effects of hue, saturation, and brightness
Wilms, L., & Oberfeld, D. (2018) · Psychological Research · doi:10.1007/s00426-017-0880-8 · Cited by 401
“The degrees of freedom were computed by the method of Kenward and Roger (1997), which has been demonstrated to be superior to alternative methods (Arnau, Bono, & Vallejo, 2009; Keselman et al., 2002; Kowalchuk, Keselman, Algina, & Wolfinger, 2004).”
MethodInfluentialRaunig, D. L., McShane, L. M., Pennello, G., Gatsonis, C., Carson, P. L., Voyvodic, J. T., Wahl, R. L., Kurland, B. F., Schwarz, A. J., Gönen, M., et al. (2015) · Statistical Methods in Medical Research · doi:10.1177/0962280214537344 · Cited by 394
“A more complete discussion may be found in Kenward and Roger.16 In general, random effects randomly choose from a large population of levels of a factor while fixed effects do not randomly select, though the distinction is not always clearly defined in practice.”
Regional variability of imaging biomarkers in autosomal dominant Alzheimer's disease
Benzinger, T. L. S., Blazey, T., Jack, C. R., Koeppe, R. A., Su, Y., Xiong, C., Raichle, M. E., Snyder, A. Z., Ances, B. M., Bateman, R. J., et al. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1317918110 · Cited by 391
“Kenward MG, Roger JH (1997) Small sample inference for fixed effects from restricted maximum likelihood.”
MethodKiecolt-Glaser, J. K., Bennett, J. M., Andridge, R., Peng, J., Shapiro, C. L., Malarkey, W. B., Emery, C. F., Layman, R., Mrozek, E. E., & Glaser, R. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2013.51.8860 · Cited by 389
“The Kenward-Roger adjustment to the df was used to control type I error rates.(39) Lipopolysaccharide-stimulated cytokine responses were natural log (ln) transformed to better approximate normality of residuals.”
MethodMultilevel modelling of medical data
Goldstein, H., Browne, W., & Rasbash, J. (2002) · Statistics in Medicine · doi:10.1002/sim.1264 · Cited by 388
“The log-likelihood is calculated for the ML estimates since this is preferable for purposes of model comparison [17].”
MethodStatistical analysis of longitudinal neuroimage data with Linear Mixed Effects models
Bernal-Rusiel, J. L., Greve, D. N., Reuter, M., Fischl, B., & Sabuncu, M. R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.10.065 · Cited by 360
“Details on the computation of κ , C ˆ o vKR ˆ β ( ) and the effective degrees of freedom can be found in (Kenward and Roger, 1997).”
BackgroundMethodEvenness drives consistent diversity effects in intensive grassland systems across 28 European sites
KIRWAN, L., LÜSCHER, A., SEBASTIÀ, M. T., FINN, J. A., COLLINS, R. P., PORQUEDDU, C., HELGADOTTIR, A., BAADSHAUG, O. H., BROPHY, C., CORAN, C., et al. (2007) · Journal of Ecology · doi:10.1111/j.1365-2745.2007.01225.x · Cited by 378
“*The approximate degrees of freedom (d.f.) are based on the Kenward-Roger approximation ( Kenward & Roger 1997 ) and used in approximate t-tests for coefficients.”
MethodWorking memory plasticity in old age: practice gain, transfer, and maintenance
Li, S.-C., Schmiedek, F., Huxhold, O., Röcke, C., Smith, J., & Lindenberger, U. (2008) · Psychology and Aging · doi:10.1037/a0014343 · Cited by 369
“Analyses were conducted with SAS PROC MIXED, and degrees of freedom were calculated with the Kenward-Roger method (Kenward & Roger, 1997).”
MethodA Hitchhiker's Guide to Mixed Models for Randomized Experiments
Piepho, H. P., Büchse, A., & Emrich, K. (2003) · Journal of Agronomy and Crop Science · doi:10.1046/j.1439-037x.2003.00049.x · Cited by 368
“The currently best approximation method is that proposed by Kenward and Roger (1997), which is available via the DDFM option of the MIXED procedure.”
Methodfor the ISCOLE Research Group, Roman-Viñas, B., Chaput, J.-P., Katzmarzyk, P. T., Fogelholm, M., Lambert, E. V., Maher, C., Maia, J., Olds, T., Onywera, V., et al. (2016) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-016-0449-8 · Cited by 368
“The denominator degrees of freedom for statistical tests pertaining to fixed effects were calculated using the Kenward and Roger approximation [33].”
MethodA Systems Biology View of Responses to Lignin Biosynthesis Perturbations in Arabidopsis
Vanholme, R., Storme, V., Vanholme, B., Sundin, L., Christensen, J. H., Goeminne, G., Halpin, C., Rohde, A., Morreel, K., & Boerjan, W. (2012) · The Plant Cell · doi:10.1105/tpc.112.102574 · Cited by 365
“The Kenward-Roger method (Kenward and Roger, 1997), recommended by Verbeke and Molenberghs (2009) when sample sizes are small at each level of the data set, was used to calculate denominator degrees of freedom for these approximate F tests.”
MethodLinear and generalized linear mixed models
Bolker, B. M. (2015) · Ecological Statistics · doi:10.1093/acprof:oso/9780199672547.003.0014 · Cited by 339
Westfall, J., Van Boven, L., Chambers, J. R., & Judd, C. M. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615569849 · Cited by 363
Johnson, M. T. J., Lajeunesse, M. J., & Agrawal, A. A. (2006) · Ecology Letters · doi:10.1111/j.1461-0248.2005.00833.x · Cited by 346
“Sampling date was treated as a fixed effect and the degrees of freedom were adjusted using the Kenward‐Roger method ( Kenward & Roger 1997 ).”
MethodFournier, J. C., DeRubeis, R. J., Shelton, R. C., Hollon, S. D., Amsterdam, J. D., & Gallop, R. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0015401 · Cited by 340
“For all models, full maximum likelihood estimation procedures were used, and the degrees of freedom for hypothesis tests were estimated with the Kenward-Roger approximation (Kenward & Roger, 1997).”
MethodPapapoulos, S., Chapurlat, R., Libanati, C., Brandi, M. L., Brown, J. P., Czerwiński, E., Krieg, M.-A., Man, Z., Mellström, D., Radominski, S. C., et al. (2012) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.1479 · Cited by 344
“Kenward and Roger’s approach((8)) was followed for estimating the denominator degrees of freedom for the hypothesis test.”
BackgroundMcNeish, D., & Kelley, K. (2019) · Psychological Methods · doi:10.1037/met0000182 · Cited by 336
LeBlanc, A. G., Katzmarzyk, P. T., Barreira, T. V., Broyles, S. T., Chaput, J.-P., Church, T. S., Fogelholm, M., Harrington, D. M., Hu, G., Kuriyan, R., et al. (2015) · PLOS ONE · doi:10.1371/journal.pone.0129622 · Cited by 335
“The KenwardRoger approximation (DDFM = KR) was used to calculate degrees of freedom [28].”
MethodKatcher, H. I., Legro, R. S., Kunselman, A. R., Gillies, P. J., Demers, L. M., Bagshaw, D. M., & Kris-Etherton, P. M. (2008) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/87.1.79 · Cited by 317
An extended mixed‐effects framework for meta‐analysis
Sera, F., Armstrong, B., Blangiardo, M., & Gasparrini, A. (2019) · Statistics in Medicine · doi:10.1002/sim.8362 · Cited by 323
“Solutions can be found in the LME models literature, such as the use of t or F distributions, 18 adjustments for standard errors and degrees of freedom, 64 and use of mixture distributions.”
Tracking atrophy progression in familial Alzheimer's disease: a serial MRI study
Ridha, B. H., Barnes, J., Bartlett, J. W., Godbolt, A., Pepple, T., Rossor, M. N., & Fox, N. C. (2006) · The Lancet Neurology · doi:10.1016/s1474-4422(06)70550-6 · Cited by 326
Stress, Inflammation, and Yoga Practice
Kiecolt-Glaser, J. K., Christian, L., Preston, H., Houts, C. R., Malarkey, W. B., Emery, C. F., & Glaser, R. (2010) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3181cb9377 · Cited by 327
Longitudinal ComBat: A method for harmonizing longitudinal multi-scanner imaging data
Beer, J. C., Tustison, N. J., Cook, P. A., Davatzikos, C., Sheline, Y. I., Shinohara, R. T., & Linn, K. A. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117129 · Cited by 319
“To test for additive site or scanner effects, we also fit models omitting the site or scanner fixed effects and used the package pbkrtest (Halekoh and Højsgaard, 2014) to carry out tests of their joint significance using the Kenward-Roger (KR) approach (Kenward and Roger, 1997).”
MethodScheipl, F., Greven, S., & Küchenhoff, H. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2007.10.022 · Cited by 321
“The method proposed by Kenward and Roger (1997) that additionally adjusts for small-sample bias in the estimation of the covariance matrix is implemented as well.”
MethodDream: powerful differential expression analysis for repeated measures designs
Hoffman, G. E., & Roussos, P. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btaa687 · Cited by 319
Should multiple imputation be the method of choice for handling missing data in randomized trials?
Sullivan, T. R., White, I. R., Salter, A. B., Ryan, P., & Lee, K. J. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280216683570 · Cited by 308
“Treatment effect estimates for LMMs in this article were obtained using restricted maximum likelihood estimation with degrees of freedom calculated according to the Kenward–Roger method.(47) MI overall, MI by group and the LMM produced unbiased treatment effect estimates across the 16 simulation scenarios (4 missing d…”
MethodObesity Risk for Female Victims of Childhood Sexual Abuse: A Prospective Study
Noll, J. G., Zeller, M. H., Trickett, P. K., & Putnam, F. W. (2007) · Pediatrics · doi:10.1542/peds.2006-3058 · Cited by 303
Mohile, S. G., Epstein, R. M., Hurria, A., Heckler, C. E., Canin, B., Culakova, E., Duberstein, P., Gilmore, N., Xu, H., Plumb, S., et al. (2020) · JAMA Oncology · doi:10.1001/jamaoncol.2019.4728 · Cited by 301
The Milieu Intérieur Consortium, Patin, E., Hasan, M., Bergstedt, J., Rouilly, V., Libri, V., Urrutia, A., Alanio, C., Scepanovic, P., Hammer, C., et al. (2018) · Nature Immunology · doi:10.1038/s41590-018-0049-7 · Cited by 296
Sheridan, S. M., Bovaird, J. A., Glover, T. A., Andrew Garbacz, S., Witte, A., & Kwon, K. (2012) · School Psychology Review · doi:10.1080/02796015.2012.12087374 · Cited by 296
“The Kenward– Rogers method (Kenward & Rogers, 1997) was used to determine the denominator degrees of freedom for all tests of fixed effects and all tests of fixed effects were two-tailed hypothesis tests.”
Methodvan der Lely, S., Frey, S., Garbazza, C., Wirz-Justice, A., Jenni, O. G., Steiner, R., Wolf, S., Cajochen, C., Bromundt, V., & Schmidt, C. (2015) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2014.08.002 · Cited by 290
“p values were based on corrected degrees of freedom by Kenward and Roger [31].”
MethodMultivariate Random-effects Meta-analysis
White, I. R. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900103 · Cited by 286
Unethical for the sake of the group: Risk of social exclusion and pro-group unethical behavior.
Thau, S., Derfler-Rozin, R., Pitesa, M., Mitchell, M. S., & Pillutla, M. M. (2015) · Journal of Applied Psychology · doi:10.1037/a0036708 · Cited by 289
Carpenter, J. R., Roger, J. H., & Kenward, M. G. (2013) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2013.834911 · Cited by 286
Self-citationPractical and statistical issues in missing data for longitudinal patient-reported outcomes
Bell, M. L., & Fairclough, D. L. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280213476378 · Cited by 285
“All mixed models used the Kenward-Roger degrees of freedom, which performs well for both small and large samples.(27) All generalized estimating equations (GEEs) assumed an independent working correlation matrix.”
MethodAdequacy of approximations to distributions of test statistics in complex mixed linear models
Schaalje, G. B., McBride, J. B., & Fellingham, G. W. (2002) · Journal of Agricultural, Biological, and Environmental Statistics · doi:10.1198/108571102726 · Cited by 282
“The purpose of this article is to extend the simulation results of Fai and Cornelius (1996) and Kenward and Roger (1997) , and to examine the Proc MIXED implementation of the FC and KR methods for small samples using some of the more complex covariance structures available.”
MethodInfluentialSmid, S. C., McNeish, D., Miočević, M., & van de Schoot, R. (2020) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2019.1577140 · Cited by 280
“In the REML category, REML with and without Kenward-Roger correction are included (for more information, see Kenward & Roger, 1997, 2009; McNeish, 2016a).”
McNeish, D. (2017) · Multivariate Behavioral Research · doi:10.1080/00273171.2017.1344538 · Cited by 278
“REML does not completely solve issues related to inflated Type-I error rates for fixed effects, so the Kenward-Roger correction (Kenward & Roger, 1997) has been shown to maintain nominal Type-I error rates and therefore its use has been recommended as best practice (e.g., McNeish & Stapleton,…”
BackgroundMethodInfluentialBowles, T. M., Barrios-Masias, F. H., Carlisle, E. A., Cavagnaro, T. R., & Jackson, L. E. (2016) · Science of The Total Environment · doi:10.1016/j.scitotenv.2016.05.178 · Cited by 277
Lark, R. M., Cullis, B. R., & Welham, S. J. (2006) · European Journal of Soil Science · doi:10.1111/j.1365-2389.2005.00768.x · Cited by 276
“The Wald statistics were tested against the F-distribution and the denominator degrees of freedom were adjusted for small sample size after Kenward & Roger (1997).”
MethodTalcott, K. E., Ratnam, K., Sundquist, S. M., Lucero, A. S., Lujan, B. J., Tao, W., Porco, T. C., Roorda, A., & Duncan, J. L. (2011) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.10-6479 · Cited by 276
Predicting the effects of climate change on avian life-history traits
Winkler, D. W., Dunn, P. O., & McCulloch, C. E. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.212251999 · Cited by 272
Trial of Galcanezumab in Prevention of Episodic Cluster Headache
Goadsby, P. J., Dodick, D. W., Leone, M., Bardos, J. N., Oakes, T. M., Millen, B. A., Zhou, C., Dowsett, S. A., Aurora, S. K., Ahn, A. H., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1813440 · Cited by 265
Ryan, J. L., Heckler, C. E., Roscoe, J. A., Dakhil, S. R., Kirshner, J., Flynn, P. J., Hickok, J. T., & Morrow, G. R. (2012) · Supportive Care in Cancer · doi:10.1007/s00520-011-1236-3 · Cited by 267
“Mixed model analyses and Type 3 tests of fixed effects using the Kenward-Roger degrees of freedom procedure [16] were used to examine the change from baseline NAv and NMx on Day 1.”
MethodWu, J. C., Kelsoe, J. R., Schachat, C., Bunney, B. G., DeModena, A., Golshan, S., Gillin, J. C., Potkin, S. G., & Bunney, W. E. (2009) · Biological Psychiatry · doi:10.1016/j.biopsych.2009.02.018 · Cited by 263
“This ethod has been shown to provide good performance for small amples in longitudinal analyses (13).”
MethodMeta‐analysis of continuous outcome data from individual patients
Higgins, J. P. T., Whitehead, A., Turner, R. M., Omar, R. Z., & Thompson, S. G. (2001) · Statistics in Medicine · doi:10.1002/sim.918 · Cited by 265
Hooper, R., Teerenstra, S., de Hoop, E., & Eldridge, S. (2016) · Statistics in Medicine · doi:10.1002/sim.7028 · Cited by 261
“If we could adjust the test to control this level correctly, then the power would be reduced (corrections such as that of Kenward & Roger [28] are available but are computationally intensive, prohibiting large numbers of simulations for estimating power).”
BackgroundMethodR$^{2}$s for Correlated Data: Phylogenetic Models, LMMs, and GLMMs
Ives, A. R. (2019) · Systematic Biology · doi:10.1093/sysbio/syy060 · Cited by 256
Cluster randomized trials with a small number of clusters: which analyses should be used?
Leyrat, C., Morgan, K. E., Leurent, B., & Kahan, B. C. (2018) · International Journal of Epidemiology · doi:10.1093/ije/dyx169 · Cited by 244
Boonen, S., Marin, F., Obermayer-Pietsch, B., Simões, M. E., Barker, C., Glass, E. V., Hadji, P., Lyritis, G., Oertel, H., Nickelsen, T., et al. (2008) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2007-0711 · Cited by 252
“The models were fit using the restricted maximum likelihood method, with Kenward Roger degrees of freedom (21).”
MethodClimatic niche shifts are common in introduced plants
Atwater, D. Z., Ervine, C., & Barney, J. N. (2017) · Nature Ecology & Evolution · doi:10.1038/s41559-017-0396-z · Cited by 245
Hostinar, C. E., Johnson, A. E., & Gunnar, M. R. (2015) · Developmental Science · doi:10.1111/desc.12195 · Cited by 244
“Kenward-Roger adjusted degrees of freedom were used (Kenward & Roger, 1997).”
Human motor cortical beta bursts relate to movement planning and response errors
Little, S., Bonaiuto, J., Barnes, G., & Bestmann, S. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000479 · Cited by 244
Metadata
- Authors
- Kenward, M. G., & Roger, J. H.
- Affiliations
- University of Kent (GB)
- Venue
- Biometrics
- Publisher
- JSTOR
- Published
- 1997-09
- Volume
- 53
- Issue
- 3
- Pages
- 983
- ISSN
- 0006-341X (ISSN-L)
- OpenAlex
- W1981747359
- Cited by
- 4,552
- References
- 23
- Citations by year
- 2026: 217 · 2025: 196 · 2024: 216 · 2023: 213 · 2022: 261 · 2021: 299 · 2020: 287 · 2019: 270 · 2018: 247 · 2017: 240 · 2016: 286 · 2015: 256 · 2014: 217 · 2013: 189 · 2012: 218
- FWCI
- 7.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrics (publishedVersion)
- PubMed (publishedVersion)
- Rothamsted Repository (Rothamsted Repository) (submittedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Applied Mathematics › Statistical and numerical algorithms
- Keywords
- restricted maximum likelihood, small sample inference, fixed effects, Wald statistic, covariance matrix estimation, small sample bias, Hotelling's T2 statistic, linear mixed models, linear Gaussian models
- MeSH terms
- Analysis of Variance, Biometry, Clinical Trials as Topic, Humans, Intermittent Claudication, Nitrogen Fixation, Plants, Probability, Regression Analysis, Research Design, Rhizobium, Soil Microbiology, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Cross-Over Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.