Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Restricted maximum likelihood (REML) is now well established as a method for estimating the parameters of the general Gaussian linear model with a structured covariance matrix, in particular for mixed linear models. Conventionally, estimates of precision and inference for fixed effects are based on their asymptotic distribution, which is known to be inadequate for some small-sample problems. In this paper, we present a scaled Wald statistic, together with an F approximation to its sampling distribution, that is shown to perform well in a range of small sample settings. The statistic uses an adjusted estimator of the covariance matrix that has reduced small sample bias. This approach has the advantage that it reproduces both the statistics and F distributions in those settings where the latter is exact, namely for Hotelling T2 type statistics and for analysis of variance F-ratios. The performance of the modified statistics is assessed through simulation studies of four different REML analyses and the methods are illustrated using three examples.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Accurate model parameter estimations were ensured by use of the residual maximum likelihood approach with the Kenward and Roger, 1997 approximation of degrees of freedom.”
From Longitudinal association between social media use and psychological distress among adolescents (2020) · cites this paper as Kenward and Roger, 1997
In the citing paper
“Not only does the data in the motivating example have different flowering days for each genotype (see for example Fig. 2), but the proposed methodology also accounts for the different strata in the Wald test by approximating the denominator degrees of freedom (Kenward and Roger, 1997).”
From Incorporating environmental covariates to explore genotype × environment × management (G × E × M) interactions: A one-stage predictive model (2023) · cites this paper as Kenward and Roger, 1997
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Random-Effects Models for Longitudinal Data
Laird, N. M., & Ware, J. H. (1982) · Biometrics · doi:10.2307/2529876 · Cited by 8,923
Recovery of inter-block information when block sizes are unequal
PATTERSON, H. D., & THOMPSON, R. (1971) · Biometrika · doi:10.1093/biomet/58.3.545 · Cited by 4,058
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667 · cited 1×
“Restricted (or residual) maximum likelihood (REML) has become well established for the estimation of the parameters of Z (Patterson and Thompson, 1971; Harville, 1977; Thompson, 1980; Robinson, 1991) and the corresponding estimator of 3, 3 say, has a generalized least squares form that uses the REML estimate of Z.”
BackgroundThat BLUP is a Good Thing: The Estimation of Random Effects
Robinson, G. K. (1991) · Statistical Science · doi:10.1214/ss/1177011926 · Cited by 1,686
Gilmour, A. R., Thompson, R., & Cullis, B. R. (1995) · Biometrics · doi:10.2307/2533274 · Cited by 915
15 Design and Analysis of Cross-Over Trials
Kenward, M. G., & Jones, B. (2007) · Handbook of Statistics · doi:10.1016/s0169-7161(07)27015-4 · Cited by 871
Self-citationPrinciples of Multivariate Analysis
Ling, R. F. (1990) · Technometrics · doi:10.1080/00401706.1990.10484645 · Cited by 812
Satterthwaite, F. E. (1941) · Psychometrika · doi:10.1007/bf02288586 · Cited by 809
Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477102 · Cited by 445
Cited by
Items 201–300 of 1,051
Differentiating between mixed-effects and latent-curve approaches to growth modeling
McNeish, D., & Matta, T. H. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0976-5 · Cited by 193
“…has largely been addressed via restricted maximum likelihood estimate (REML) to address the bias in the random effect variance (Harville, 1977) and Kenward-Roger corrections to address the bias in the standard error estimates and inflated operating Type-I error rates (Kenward & Roger, 1997, 2009).”
MethodTillage and rotation effect on corn–soybean energy balances in eastern Nebraska
Rathke, G.-W., Wienhold, B. J., Wilhelm, W. W., & Diepenbrock, W. (2007) · Soil and Tillage Research · doi:10.1016/j.still.2007.08.008 · Cited by 150
“The Kenward–Roger method was used to estimate degrees of freedom (Kenward and Roger, 1997).”
MethodInhaled Mannitol Improves Lung Function in Cystic Fibrosis
Jaques, A., Daviskas, E., Turton, J. A., McKay, K., Cooper, P., Stirling, R. G., ROBERTSON, C. F., Bye, P. T. P., Souëf, P. N. L., Shadbolt, B., et al. (2008) · CHEST Journal · doi:10.1378/chest.07-2294 · Cited by 149
Murayama, K., Usami, S., & Sakaki, M. (2022) · Psychological Methods · doi:10.1037/met0000330 · Cited by 208
“Note also that, in this specific example, the equivalence holds (i.e. the results stay the same) even if we used a method that adjusts degrees of freedom and/or standard errors for small sample size such as the Kenward-Roger correction (Kenward & Roger, 1997).”
MethodSatterthwaite, T. D., Ruparel, K., Loughead, J., Elliott, M. A., Gerraty, R. T., Calkins, M. E., Hakonarson, H., Gur, R. C., Gur, R. E., & Wolf, D. H. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.03.060 · Cited by 148
“The Kenward–Roger correction for degrees of freedom was applied (Kenward and Roger, 1997).”
Malik, M., van Gelderen, E. M., Lee, J. H., Kowalski, D. L., Yen, M., Goldwater, R., Mujais, S. K., Schaddelee, M. P., de Koning, P., Kaibara, A., et al. (2012) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2012.181 · Cited by 149
What makes an effective restoration planting for woodland birds?
Lindenmayer, D. B., Knight, E., Crane, M. J., Montague‐Drake, R., Michael, D., & MacGregor, C. I. (2009) · Biological Conservation · doi:10.1016/j.biocon.2009.10.010 · Cited by 159
Complementary group Metacognitive Training (MCT) reduces delusional ideation in schizophrenia.
Moritz, S., Veckenstedt, R., Bohn, F., Hottenrott, B., Scheu, F., Randjbar, S., Aghotor, J., Köther, U., Woodward, T. S., Treszl, A., et al. (2013) · Schizophrenia Research · doi:10.1016/j.schres.2013.10.007 · Cited by 146
“We applied an unstructured covariance matrix along with standard errors and degrees of freedom (Kenward and Roger, 1997; Carpenter and Kenward, 2008).”
MethodExploring real-time in vivo redox biology of developing and aging Caenorhabditis elegans
Back, P., Vos, W. H. D., Depuydt, G., Matthijssens, F., Vanfleteren, J. R., & Braeckman, B. P. (2011) · Free Radical Biology and Medicine · doi:10.1016/j.freeradbiomed.2011.11.037 · Cited by 175
“Degrees of freedom were estimated following Kenward and Roger [26].”
MethodGurka, M. J., LoCasale‐Crouch, J., & Blackman, J. A. (2010) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/archpediatrics.2010.83 · Cited by 184
“The Kenward-Roger approximation of the degrees of freedom was used to compute P values from the overall tests.(28) These repeated-measures models used all available data on participants, including those who dropped out of the study or had intermittently missing data.”
MethodO’Neill, P. M., Shanahan, J. F., Schepers, J. S., & Caldwell, B. (2004) · Agronomy Journal · doi:10.2134/agronj2004.1660 · Cited by 145
“After to reduce small sample bias (Kenward and Roger, 1997). plot biomass sampling, the center two rows of the entire length Water, N, and hybrid were treated as fixed effects and year (30.5 m) of each plot were machine-harvested.”
MethodInfluentialHigh Generic Drug Prices and Market Competition
Dave, C. V., Kesselheim, A. S., Fox, E. R., Qiu, P., & Hartzema, A. G. (2017) · Annals of Internal Medicine · doi:10.7326/m16-1432 · Cited by 149
Li, P., & Redden, D. T. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0026-x · Cited by 142
“In contrast, the Satterthwaite and Kenward-Roger methods can provide tests with very conservative type I error rates when the total cluster number is small ( 30) and the conservativeness becomes more severe as variation in cluster sizes increases.”
BackgroundMethodInfluentialLuby, J. L., Belden, A., Harms, M. P., Tillman, R., & Barch, D. M. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1601443113 · Cited by 142
Journal De La Société Française De Statistique
Caussinus, H., & de Falguerolles, A. (2005) · Wiley eBooks · doi:10.1002/0471667196.ess7017 · Cited by 141
Escitalopram in the treatment of major depressive disorder: A meta-analysis
Kennedy, S. H., Andersen, H. F., & Thase, M. E. (2009) · Current Medical Research and Opinion · doi:10.1185/03007990802622726 · Cited by 140
INTERACTIONS BETWEEN HERBIVOROUS FISHES AND LIMITING NUTRIENTS IN A TROPICAL STREAM ECOSYSTEM
Flecker, A. S., Taylor, B. W., Bernhardt, E. S., Hood, J. M., Cornwell, W. K., Cassatt, S. R., Vanni, M. J., & Altman, N. (2002) · Ecology · doi:10.1890/0012-9658(2002)083[1831:ibhfal]2.0.co;2 · Cited by 171
“For this reason, Kenward-Roger approximate F tests and estimated degrees of freedom were used for testing (Kenward and Roger 1997, SAS Institute 1999), which can result in fractional degrees of freedom in the F tests.”
MethodHuman cadavers Vs. multimedia simulation: A study of student learning in anatomy
Saltarelli, A. J., Roseth, C. J., & Saltarelli, W. A. (2014) · Anatomical Sciences Education · doi:10.1002/ase.1429 · Cited by 154
Benyamini, Y., Gozlan, M., & Kokia, E. (2009) · British Journal of Health Psychology · doi:10.1348/135910708x279288 · Cited by 139
“Denominator degrees of freedom for the tests of fixed effects were calculated based on the method detailed by Kenward and Rogers (1997).”
MethodTokida, T., Adachi, M., Cheng, W., Nakajima, Y., Fumoto, T., Matsushima, M., Nakamura, H., Okada, M., Sameshima, R., & Hasegawa, T. (2011) · Global Change Biology · doi:10.1111/j.1365-2486.2011.02475.x · Cited by 139
“The Kenward–Roger method was used to estimate the degrees of freedom (Kenward & Roger, 1997).”
MethodSegura, P. A., Takada, H., Correa, J. A., Saadi, K. E., Koike, T., Onwona‐Agyeman, S., Ofosu-Anim, J., Sabi, E. B., Wasonga, O. V., Mghalu, J. M., et al. (2015) · Environment International · doi:10.1016/j.envint.2015.04.001 · Cited by 164
“Degrees of freedomwere adjusted using the Kenward–Roger correction (Kenward and Roger, 1997) because the groupswere unbalanced.”
The use of baseline covariates in crossover studies
Kenward, M. G., & Roger, J. H. (2010) · Biostatistics · doi:10.1093/biostatistics/kxp046 · Cited by 138
Self-citationResponse of raptors to a windfarm
Garvin, J. C., Jennelle, C. S., Drake, D., & Grodsky, S. M. (2010) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2010.01912.x · Cited by 137
“Degrees of freedom were calculated using the approximation of Kenward & Roger (1997).”
MethodMeta-analysis of individual patient data versus aggregate data from longitudinal clinical trials
Jones, A. P., Riley, R. D., Williamson, P. R., & Whitehead, A. (2009) · Clinical Trials · doi:10.1177/1740774508100984 · Cited by 137
“SAS PROC MIXED allows the option to inflate the estimated variance of the mean treatment difference to allow for the estimation of the variance and covariance components and to estimate the degrees of freedom for the t-test using Satterthwaite’s procedure [12,13].”
MethodSoybean response to nitrogen application across the United States: A synthesis-analysis
Mourtzinis, S., Kaur, G., Orlowski, J. M., Shapiro, C. A., Lee, C. D., Wortmann, C. S., Holshouser, D. L., Nafziger, E. D., Kandel, H. J., Niekamp, J. W., et al. (2017) · Field Crops Research · doi:10.1016/j.fcr.2017.09.035 · Cited by 138
Psychosocial Functioning Improves Following Adolescent Bariatric Surgery
Zeller, M. H., Modi, A. C., Noll, J. G., Long, J. D., & Inge, T. H. (2009) · Obesity · doi:10.1038/oby.2008.644 · Cited by 148
Swan, C. M., & Palmer, M. A. (2006) · Oecologia · doi:10.1007/s00442-005-0297-8 · Cited by 137
“Denominator degrees of freedom for all models were adjusted to include the random block effect by selecting the Kenward–Roger method in SAS (based on calculations in Kenward and Roger 1997 ).”
MethodGałecki, A. T., & Burzykowski, T. (2012) · Springer texts in statistics · doi:10.1007/978-1-4614-3900-4_13 · Cited by 191
Thönes, S., & Oberfeld, D. (2017) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.03.007 · Cited by 154
“The degrees of freedom were calculated according to Kenward and Roger (1997).”
MethodMiles, C., Wallace, R., Wszelaki, A., Martin, J., Cowan, J., Walters, T., & Inglis, D. (2012) · HortScience · doi:10.21273/hortsci.47.9.1270 · Cited by 118
“At Knoxville and Lubbock the Kenward and Roger (1997) method was used to determine denominator df (DDFM) for F-tests in the analysis, whereas at Mount Vernon, the Satterthwaite (1946) method was used to determine DDFM. Treatment means were compared for significant differences using Fisher’s least…”
MethodPersonality predicts individual responsiveness to the risks of starvation and predation
Quinn, J. L., Cole, E. F., Bates, J., Payne, R. W., & Cresswell, W. R. (2011) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2011.2227 · Cited by 164
Psychological Impact of Genetic Testing for Hereditary Nonpolyposis Colorectal Cancer
Gritz, E. R., Peterson, S. K., Vernon, S. W., Marani, S. K., Baile, W. F., Watts, B. G., Amos, C. I., Frazier, M. L., & Lynch, P. M. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.07.102 · Cited by 134
Bernal-Rusiel, J. L., Reuter, M., Greve, D. N., Fischl, B., & Sabuncu, M. R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.049 · Cited by 125
“Finally, hypothesis testing can be conducted based on the Satterthwaite-based approximation of a scaled F-statistic (Kenward and Roger, 1997).”
BackgroundMethodInfluentialPreventing depression: a randomized trial of interpersonal psychotherapy-adolescent skills training
Young, J. F., Mufson, L., & Gallop, R. (2010) · Depression and Anxiety · doi:10.1002/da.20664 · Cited by 133
Statistical considerations for the safety evaluation of GMOs
(GMO), E. P. O. G. M. O. (2010) · EFSA Journal · doi:10.2903/j.efsa.2010.1250 · Cited by 130
The p38 Kinases MKK4 and MKK6 Suppress Metastatic Colonization in Human Ovarian Carcinoma
Hickson, J. A., Huo, D., Griend, D. J. V., Lin, A., Rinker-Schaeffer, C. W., & Yamada, S. D. (2006) · Cancer Research · doi:10.1158/0008-5472.can-05-3676 · Cited by 137
Wozniak, J. R., Fuglestad, A. J., Eckerle, J. K., Fink, B. A., Hoecker, H. L., Boys, C. J., Radke, J. P., Kroupina, M., Miller, N. C., Brearley, A. M., et al. (2015) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.114.099168 · Cited by 144
Bodner, G., Scholl, P., Loiskandl, W., & Kaul, H. (2013) · Geoderma · doi:10.1016/j.geoderma.2013.04.015 · Cited by 131
“Denominator degrees of freedom were approximated by the Kenward Roger method (Kenward and Roger, 1997).”
MethodTokida, T., Fumoto, T., Cheng, W., Matsunami, T., Adachi, M., Katayanagi, N., Matsushima, M., Okawara, Y., Nakamura, H., Okada, M., et al. (2010) · Biogeosciences · doi:10.5194/bg-7-2639-2010 · Cited by 136
“In the covariance analysis, we assumed that the slope was the same across all experimental units (12 in total), and used the Kenward-Roger method to estimate the degrees of freedom of the denominator (Kenward and Roger, 1997).”
MethodMemories with a blind mind: Remembering the past and imagining the future with aphantasia
Dawes, A. J., Keogh, R., Robuck, S., & Pearson, J. (2022) · Cognition · doi:10.1016/j.cognition.2022.105192 · Cited by 130
“…of imagery group (controls, aphantasia), time (past, future), and the group-time interaction (fixed effects), with intercepts varying by participant (random effect). p - values and confidence intervals were derived via the Kenward-Roger approximation for degrees of freedom (Kenward & Roger, 1997).”
MethodLee, M. R., Tapocik, J. D., Ghareeb, M., Schwandt, M. L., Dias, A. A., Le, A. N., Cobbina, E., Farinelli, L. A., Bouhlal, S., Farokhnia, M., et al. (2020) · Molecular Psychiatry · doi:10.1038/s41380-018-0064-y · Cited by 129
“The Kenward–Roger correction [49] was used in all models analyzed using SAS PROC MIXED, as the use of this correction is highly recommended in repeated measures models [50].”
MethodData Transformation in Statistical Analysis of Field Trials with Changing Treatment Variance
Piepho, H. (2009) · Agronomy Journal · doi:10.2134/agronj2008.0226x · Cited by 129
Tillage impacts on soil aggregation and aggregate-associated carbon and nitrogen after 49 years
Weidhuner, A., Hanauer, A., Krausz, R. F., Crittenden, S. J., Gage, K. L., & Sadeghpour, A. (2020) · Soil and Tillage Research · doi:10.1016/j.still.2020.104878 · Cited by 157
The Analysis of Repeated Measurements with Mixed-Model Adjusted F Tests
Kowalchuk, R. K., Keselman, H. J., Algina, J., & Wolfinger, R. D. (2004) · Educational and Psychological Measurement · doi:10.1177/0013164403260196 · Cited by 127
“Available in the latest release of SAS (i.e., version 8.0) is the option of Kenward-Roger’s (Kenward & Roger, 1997) df solution through PROC MIXED.”
BackgroundMethodInfluentialO’Neill, P. M., Shanahan, J. F., & Schepers, J. S. (2006) · Crop Science · doi:10.2135/cropsci2005.06-0170 · Cited by 153
“This method uses an adjusted estimator of the covariance matrix to reduce small sample bias (Kenward and Roger, 1997).”
MethodFrontal Brain Asymmetry and Willingness to Pay
Ramsøy, T. Z., Skov, M., Christensen, M. K., & Stahlhut, C. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00138 · Cited by 126
“The DFDen is calculated using the Kenward–Roger first order approximation (Kenward and Roger, 1997), and shows the denominator degrees of freedom for the effect test (the degrees of freedom for error).”
MethodMurayama, K., Sakaki, M., Yan, V. X., & Smith, G. M. (2014) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0036914 · Cited by 74
Huntingtin suppression restores cognitive function in a mouse model of Huntington’s disease
Southwell, A. L., Kordasiewicz, H. B., Langbehn, D., Skotte, N. H., Parsons, M. P., Villanueva, E. B., Caron, N. S., Østergaard, M. E., Anderson, L. M., Xie, Y., et al. (2018) · Science Translational Medicine · doi:10.1126/scitranslmed.aar3959 · Cited by 126
“Denominator degrees of freedom for all F tests were adjusted using the KenwardRogers correction (44), which often results in a noninteger degree-offreedom value.”
MethodA review on linear mixed models for longitudinal data, possibly subject to dropout
Molenberghs, G., & Verbeke, G. (2001) · Statistical Modelling · doi:10.1177/1471082x0100100402 · Cited by 126
“Kenward and Roger (1997) proposed a scaled Wald statistic, based on an adjusted covariance estimate which accounts for the extra variability introduced by estimating a, and they show that its small sample distribution can be well approximated by an F distribution with denominator degrees of freedom…”
BackgroundBolland, J. M., Bryant, C. M., Lian, B. E., McCallum, D. M., Vazsonyi, A. T., & Barth, J. M. (2007) · American Journal of Community Psychology · doi:10.1007/s10464-007-9132-1 · Cited by 141
“estimated using a procedure developed by Satterthwaite (1941) coupled with an inflation of the estimated variance‐covariance matrix of the fixed and random effects, as detailed by Kenward and Roger (1997) .”
MethodAsarnow, J. R., Berk, M. S., Bedics, J. D., Adrian, M. C., Gallop, R., Cohen, J. A., Korslund, K. E., Hughes, J. L., Avina, C., Linehan, M. M., et al. (2021) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2021.01.016 · Cited by 170
Keating, C. L., Yonker, L. M., Vermeulen, F. L., Prais, D., Linnemann, R. W., Trimble, A. T., Kotsimbos, T. C., Mermis, J. D., Braun, A. T., O’Carroll, M., et al. (2025) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(24)00411-9 · Cited by 157
“Other secondary endpoints were analysed similarly to the primary or key secondary endpoints, as applicable (appendix pp 20–23).”
Roy, A. (2006) · Biometrical Journal · doi:10.1002/bimj.200510192 · Cited by 123
“DDFM ¼ KR specifies the Kenward and Roger (1997) correction for computing the denominator degrees of freedom for the fixed effects.”
BackgroundTurner, S., Forbes, A. B., Karahalios, A., Taljaard, M., & McKenzie, J. E. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01364-0 · Cited by 160
Evaluating models for partially clustered designs.
Baldwin, S. A., Bauer, D. J., Stice, E., & Rohde, P. (2011) · Psychological Methods · doi:10.1037/a0023464 · Cited by 122
“…of freedom in multilevel models: (a) the between-within method, (b) the Satterthwaite (1946) approximation, and (c) the Kenward-Roger method (Kenward & Roger, 1997).1 The between-within method is based on a loose analogy to repeated measures ANOVA and separates the degrees of freedom into…”
Spears, M., Donnelly, I., Jolly, L., Brannigan, M., Ito, K., McSharry, C., Lafferty, J., Chaudhuri, R., Braganza, G., Adcock, I. M., et al. (2009) · European Respiratory Journal · doi:10.1183/09031936.00158208 · Cited by 122
Shen, C., Ratan, R., Cai, Y., & Leavitt, A. (2016) · Journal of Computer-Mediated Communication · doi:10.1111/jcc4.12159 · Cited by 151
“The p value of all parameter estimates was calculated using the Kenward-Roger method, which was considered conservative among alternatives (Kenward & Roger, 1997).”
BackgroundMethodInfluentialSome general points in estimating heterogeneity variance with the DerSimonian-Laird estimator
Bohning, D. (2002) · Biostatistics · doi:10.1093/biostatistics/3.4.445 · Cited by 121
“Kennward and Roger (1997) discuss the effect of replacing the true variance–covariance matrix for small-sample inference by its estimate for the fixed-effect model.”
BackgroundThe Origins of Hopelessness Among Inner-City African-American Adolescents
Bolland, J. M., Lian, B. E., & Formichella, C. M. (2005) · American Journal of Community Psychology · doi:10.1007/s10464-005-8627-x · Cited by 143
“Degrees of freedom were estimated using a procedure developed by Satterthwaite (1941) coupled with an inflation of the estimated variance‐covariance matrix of the fixed and random effects, as detailed by Kenward and Roger (1997) .”
MethodVariability in ADHD Care in Community-Based Pediatrics
Epstein, J. N., Kelleher, K. J., Baum, R., Brinkman, W. B., Peugh, J., Gardner, W., Lichtenstein, P., & Langberg, J. (2014) · Pediatrics · doi:10.1542/peds.2014-1500 · Cited by 120
Liquet, B., Cao, K. L., Hocini, H., & Thiébaut, R. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-325 · Cited by 140
“The authors recommended to use the F-test which still poses the issue of the choice of the number of degrees of freedom with a small number of samples [29,30].”
MethodClimate warming drives local extinction: Evidence from observation and experimentation
Panetta, A. M., Stanton, M. L., & Harte, J. (2018) · Science Advances · doi:10.1126/sciadv.aaq1819 · Cited by 134
Shaffer, M. L., Baud, O., Lacaze‐Masmonteil, T., Peltoniemi, O. M., Bonsante, F., & Watterberg, K. L. (2018) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2018.10.004 · Cited by 131
Microbial inoculation for improving the growth and health of micropropagated strawberry
Vestberg, M. V., Kukkonen, S. T., Saari, K., Parikka, P., Huttunen, J., Tainio, L., Devos, N., Weekers, F., Kevers, C., Thonart, P., et al. (2004) · Applied Soil Ecology · doi:10.1016/j.apsoil.2004.05.006 · Cited by 119
“The degrees of freedom were approximated through the method introduced by Kenward and Roger (1997). For comparison of fixed effects, the ESTIMATE statement of PROC MIXED was used.”
MethodDreisoerner, A., Junker, N. M., Schlotz, W., Heimrich, J., Bloemeke, S., Ditzen, B., & van Dick, R. (2021) · Comprehensive Psychoneuroendocrinology · doi:10.1016/j.cpnec.2021.100091 · Cited by 118
“sizes were not perfectly balanced (see Table 1), degrees of freedom used for hypothesis tests for the fixed effects were based on small-sample approximation using the Kenward-Roger method [47].”
MethodPrecision and accuracy in quantifying herbivory
Johnson, M. T. J., BERTRAND, J. A., & Turcotte, M. M. (2015) · Ecological Entomology · doi:10.1111/een.12280 · Cited by 118
“© 2015 The Royal Entomological Society, Ecological Entomology, 41, 112–121 We also used this package to adjust the denominator d.f. using the Kenward–Roger correction method (Kenward & Roger, 1997).”
MethodDams have varying impacts on fish communities across latitudes: a quantitative synthesis
Turgeon, K., Turpin, C., & Gregory‐Eaves, I. (2019) · Ecology Letters · doi:10.1111/ele.13283 · Cited by 141
Gill, J. L., Bishop, S. C., McCorquodale, C. C., Williams, J. L., & Wiener, P. (2009) · Genetics Selection Evolution · doi:10.1186/1297-9686-41-36 · Cited by 118
“Statistical significance of the extra variation accounted for by the presence of the haplotype groups in the model was determined using approximate F-statistics derived from Wald statistics with denominator degrees of freedom estimated in the Genstat REML procedure [25].”
MethodCompas, B. E., Forehand, R., Thigpen, J. C., Keller, G., Hardcastle, E. J., Cole, D. A., Potts, J., Watson, K. H., Rakow, A., Colletti, C., et al. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0024254 · Cited by 117
Gilbert, J. L., Guthart, M. J., Gezan, S. A., de Carvalho, M. P., Schwieterman, M. L., Colquhoun, T. A., Bartoshuk, L. M., Sims, C. A., Clark, D. G., & Olmstead, J. W. (2015) · PLoS ONE · doi:10.1371/journal.pone.0138494 · Cited by 135
Tall, S., & Meyling, N. V. (2018) · Microbial Ecology · doi:10.1007/s00248-018-1180-6 · Cited by 117
Pechey, R., Cartwright, E., Pilling, M. A., Hollands, G. J., Vasiljevic, M., Jebb, S. A., & Marteau, T. M. (2018) · Appetite · doi:10.1016/j.appet.2018.11.013 · Cited by 127
“P-values were calculated using the robust Kenward-Roger adjustment, in order to minimize the potential for bias given the small sample size, which could otherwise inflate Type-I error rates (Kenward & Roger, 1997; McNeish, 2017).”
MethodCastro, L., Strömbäck, J., Esser, F., Aelst, P. V., de Vreese, C. H., Aalberg, T., Cardenal, A. S., Corbu, N., Hopmann, D. N., Koç-Michalska, K., et al. (2021) · The International Journal of Press/Politics · doi:10.1177/19401612211012572 · Cited by 187
“…restricted maximum likelihood (REML) and an adjustment for the relatively small number of clusters (countries) using the Kenward–Roger correction (Kenward and Roger 1997; McNeish and Stapleton 2016).2 In these models, we included several variables that research has shown to have an impact on…”
MethodQuantifying direct yield benefits of soil carbon increases from cover cropping
Vendig, I., Guzman, A., Cerda, G. D. L., Esquivel, K., Mayer, A. C., Ponisio, L. C., & Bowles, T. M. (2023) · Nature Sustainability · doi:10.1038/s41893-023-01131-7 · Cited by 145
“We used the Kenward–Roger approximation for denominator degrees of freedom in all P -value calculations 65 .”
MethodMcKendrick, R. D., Ayaz, H., Olmstead, R., & Parasuraman, R. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.103 · Cited by 136
Analysis of unbalanced mixed model data: A case study comparison of ANOVA versus REML/GLS
Littell, R. C. (2002) · Journal of Agricultural Biological and Environmental Statistics · doi:10.1198/108571102816 · Cited by 116
“Prasad and Rao (1990) and Kenward and Roger (1997) further developed the expressions.”
BackgroundMethodInfluentialAnalysis of Unbalanced Data by Mixed Linear Models Using the mixed Procedure of the SAS System
Spilke, J., Piepho, H. P., & Hu, X. (2005) · Journal of Agronomy and Crop Science · doi:10.1111/j.1439-037x.2004.00120.x · Cited by 116
Seddon, J. L., Birchwood, M., Copello, A., Everard, L., Jones, P. B., Fowler, D., Amos, T., Freemantle, N., Sharma, V., Marshall, M., et al. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbv154 · Cited by 115
Oberfeld, D., & Franke, T. (2012) · Behavior Research Methods · doi:10.3758/s13428-012-0281-2 · Cited by 137
“PROC MIXED fitting an unstructured covariance matrix and using the Kenward and Roger (1997) adjustment controlled the Type I error rate.”
InfluentialFerguson, J. C., Chechetto, R. G., Hewitt, A. J., Chauhan, B. S., Adkins, S. W., Kruger, G. R., & O’Donnell, C. C. (2015) · Crop Protection · doi:10.1016/j.cropro.2015.11.013 · Cited by 115
Fleischhacker, W., Galderisi, S., Laszlovszky, I., Szatmári, B., Barabássy, Á., Acsai, K., Szalai, E., Harsányi, J., Earley, W. R., Patel, M., et al. (2019) · European Psychiatry · doi:10.1016/j.eurpsy.2019.01.015 · Cited by 150
Carmen, J. C., Runyan, C. M., Robison, R. A., Nelson, J. L., Beckstead, B. L., Pitt, W. G., & Schaalje, G. B. (2004) · Journal of Infection and Chemotherapy · doi:10.1007/s10156-004-0319-1 · Cited by 119
Papa, A., Sewell, M. T., Garrison‐Diehn, C., & Rummel, C. (2013) · Behavior Therapy · doi:10.1016/j.beth.2013.04.009 · Cited by 133
Hodgins, K. A., & Rieseberg, L. H. (2011) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2011.02404.x · Cited by 123
Associations between sleep patterns and lifestyle behaviors in children: an international comparison
for the ISCOLE Research Group, Chaput, J.-P., Katzmarzyk, P. T., LeBlanc, A. G., Tremblay, M. S., Barreira, T. V., Broyles, S. T., Fogelholm, M., Hu, G., Kuriyan, R., et al. (2015) · International Journal of Obesity Supplements · doi:10.1038/ijosup.2015.21 · Cited by 114
Pauli, D., Andrade-Sánchez, P., Carmo‐Silva, E., Gazave, É., French, A. N., Heun, J. T., Hunsaker, D. J., Lipka, A. E., Setter, T. L., Strand, R. J., et al. (2016) · G3 Genes Genomes Genetics · doi:10.1534/g3.115.023515 · Cited by 138
“Sequential tests of fixed effects were conducted, with degrees of freedom being calculated with the Kenward and Rogers approximation (Kenward and Roger 1997) in ASReml-R version 3.0 (Gilmour et al. 2009).”
MethodInfluentialGenetic association of stomatal traits and yield in wheat grown in low rainfall environments
Shahinnia, F., Roy, J. L., Laborde, B., Sznajder, B., Kalambettu, P., Mahjourimajd, S., Tilbrook, J., & Fleury, D. (2016) · BMC Plant Biology · doi:10.1186/s12870-016-0838-9 · Cited by 112
“To detect significant allelic effect for single markers at the chromosomal region of interest, Wald statistics were applied [48].”
MethodMultilevel and Single-Level Models for Measured and Latent Variables When Data Are Clustered
Stapleton, L. M., McNeish, D., & Yang, J. S. (2016) · Educational Psychologist · doi:10.1080/00461520.2016.1207178 · Cited by 141
“Because of the bias in estimates that can be found with a small number of clusters (12 in this example), a KenwardRoger correction (Kenward & Roger, 1997) was used with the multilevel model and a Kauermann–Carroll correction (Kauermann & Carroll, 2001) was applied to the clusterrobust estimator for…”
MethodBaranek, G. T., Roberts, J. E., David, F. J., Sideris, J., Mirrett, P. L., Hatton, D. D., & Bailey, D. B. (2008) · Physical & Occupational Therapy In Pediatrics · doi:10.1300/j006v28n01_06 · Cited by 113
“Additional models were run predicting the dependent variables from measures of heart rate activity (IBI and vagal tone) and FMRP. Significance tests were conducted using the Kenward and Roger (1997) method for calculating degrees of freedom.”
MethodOffenbacher, S., Barros, S., Mendoza, L., Mauriello, S., Preisser, J., Moss, K., De Jager, M., & Aspiras, M. (2010) · Journal of Clinical Periodontology · doi:10.1111/j.1600-051x.2010.01543.x · Cited by 113
“All tests used the KenwardRoger degrees of freedom small-sample correction (Kenward & Roger 1997).”
MethodDodd, G. D., Dodd, N. A., Lanctot, A. C., & Glueck, D. A. (2013) · Radiology · doi:10.1148/radiol.12120486 · Cited by 111
Quantifying degeneration of white matter in normal aging using fractal dimension
Zhang, L., Dean, D., Liu, J. Z., Sahgal, V., Wang, X., & Yue, G. H. (2006) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2006.06.020 · Cited by 113
ESTIMATING POPULATION TREND AND PROCESS VARIATION FOR PVA IN THE PRESENCE OF SAMPLING ERROR
Staples, D. F., Taper, M. L., & Dennis, B. (2004) · Ecology · doi:10.1890/03-3101 · Cited by 111
Breastfeeding and childhood obesity: A 12‐country study
Ma, J., Qiao, Y., Zhao, P. Z., Li, W., Katzmarzyk, P. T., Chaput, J., Fogelholm, M., Kuriyan, R., Lambert, E. V., Maher, C. A., et al. (2020) · Maternal and Child Nutrition · doi:10.1111/mcn.12984 · Cited by 155
“The denominator degrees of freedom for statistical tests pertaining to fixed effects were calculated using the Kenward and Roger (1997) approximation.”
MethodStarkey, C. P., Geesink, G. H., Collins, D., Oddy, V. H., & Hopkins, D. L. (2015) · Meat Science · doi:10.1016/j.meatsci.2015.11.013 · Cited by 131
Butler, D., Cullis, B. R., Gilmour, A. R., & Gogel, B. J. (2009) · Cited by 215
Measuring Fine-Scale White-Tailed Deer Movements and Environmental Influences Using GPS Collars
Webb, S. L., Gee, K. L., Strickland, B. K., Demarais, S., & DeYoung, R. W. (2010) · International Journal of Ecology · doi:10.1155/2010/459610 · Cited by 109
“For all repeated measures designs and mixed models, we used a degrees of freedom adjustment developed by Kenward and Roger [35].”
MethodExtending the Box–Cox Transformation to the Linear Mixed Model
Gurka, M. J., Edwards, L. J., Muller, K. E., & Kupper, L. L. (2006) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2005.00391.x · Cited by 108
Shah, V. N., Harrall, K. K., Shah, C. S., Gallo, T., Joshee, P., Snell‐Bergeon, J. K., & Kohrt, W. M. (2017) · Osteoporosis International · doi:10.1007/s00198-017-4097-x · Cited by 114
“Between-study variation was modeled as a random effect, and degrees of freedomwere adjusted using the Kenward-Roger method [36].”
MethodThe design and analysis of multi-phase plant breeding experiments
Smith, A. B., Lim, P., & Cullis, B. R. (2006) · The Journal of Agricultural Science · doi:10.1017/s0021859606006319 · Cited by 116
“In these cases, the adjustments of Kenward & Roger (1997) may be used.”
Method
Metadata
- Authors
- Kenward, M. G., & Roger, J. H.
- Affiliations
- University of Kent (GB)
- Venue
- Biometrics
- Publisher
- JSTOR
- Published
- 1997-09
- Volume
- 53
- Issue
- 3
- Pages
- 983
- ISSN
- 0006-341X (ISSN-L)
- OpenAlex
- W1981747359
- Cited by
- 4,552
- References
- 23
- Citations by year
- 2026: 217 · 2025: 196 · 2024: 216 · 2023: 213 · 2022: 261 · 2021: 299 · 2020: 287 · 2019: 270 · 2018: 247 · 2017: 240 · 2016: 286 · 2015: 256 · 2014: 217 · 2013: 189 · 2012: 218
- FWCI
- 7.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrics (publishedVersion)
- PubMed (publishedVersion)
- Rothamsted Repository (Rothamsted Repository) (submittedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Applied Mathematics › Statistical and numerical algorithms
- Keywords
- restricted maximum likelihood, small sample inference, fixed effects, Wald statistic, covariance matrix estimation, small sample bias, Hotelling's T2 statistic, linear mixed models, linear Gaussian models
- MeSH terms
- Analysis of Variance, Biometry, Clinical Trials as Topic, Humans, Intermittent Claudication, Nitrogen Fixation, Plants, Probability, Regression Analysis, Research Design, Rhizobium, Soil Microbiology, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Cross-Over Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.