Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Restricted maximum likelihood (REML) is now well established as a method for estimating the parameters of the general Gaussian linear model with a structured covariance matrix, in particular for mixed linear models. Conventionally, estimates of precision and inference for fixed effects are based on their asymptotic distribution, which is known to be inadequate for some small-sample problems. In this paper, we present a scaled Wald statistic, together with an F approximation to its sampling distribution, that is shown to perform well in a range of small sample settings. The statistic uses an adjusted estimator of the covariance matrix that has reduced small sample bias. This approach has the advantage that it reproduces both the statistics and F distributions in those settings where the latter is exact, namely for Hotelling T2 type statistics and for analysis of variance F-ratios. The performance of the modified statistics is assessed through simulation studies of four different REML analyses and the methods are illustrated using three examples.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Accurate model parameter estimations were ensured by use of the residual maximum likelihood approach with the Kenward and Roger, 1997 approximation of degrees of freedom.”
From Longitudinal association between social media use and psychological distress among adolescents (2020) · cites this paper as Kenward and Roger, 1997
In the citing paper
“Not only does the data in the motivating example have different flowering days for each genotype (see for example Fig. 2), but the proposed methodology also accounts for the different strata in the Wald test by approximating the denominator degrees of freedom (Kenward and Roger, 1997).”
From Incorporating environmental covariates to explore genotype × environment × management (G × E × M) interactions: A one-stage predictive model (2023) · cites this paper as Kenward and Roger, 1997
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Random-Effects Models for Longitudinal Data
Laird, N. M., & Ware, J. H. (1982) · Biometrics · doi:10.2307/2529876 · Cited by 8,923
Recovery of inter-block information when block sizes are unequal
PATTERSON, H. D., & THOMPSON, R. (1971) · Biometrika · doi:10.1093/biomet/58.3.545 · Cited by 4,058
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667 · cited 1×
“Restricted (or residual) maximum likelihood (REML) has become well established for the estimation of the parameters of Z (Patterson and Thompson, 1971; Harville, 1977; Thompson, 1980; Robinson, 1991) and the corresponding estimator of 3, 3 say, has a generalized least squares form that uses the REML estimate of Z.”
BackgroundThat BLUP is a Good Thing: The Estimation of Random Effects
Robinson, G. K. (1991) · Statistical Science · doi:10.1214/ss/1177011926 · Cited by 1,686
Gilmour, A. R., Thompson, R., & Cullis, B. R. (1995) · Biometrics · doi:10.2307/2533274 · Cited by 915
15 Design and Analysis of Cross-Over Trials
Kenward, M. G., & Jones, B. (2007) · Handbook of Statistics · doi:10.1016/s0169-7161(07)27015-4 · Cited by 871
Self-citationPrinciples of Multivariate Analysis
Ling, R. F. (1990) · Technometrics · doi:10.1080/00401706.1990.10484645 · Cited by 812
Satterthwaite, F. E. (1941) · Psychometrika · doi:10.1007/bf02288586 · Cited by 809
Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477102 · Cited by 445
Cited by
Items 401–500 of 1,051
Sample Size Determination for Hierarchical Longitudinal Designs with Differential Attrition Rates
Roy, A., Bhaumik, D. K., Aryal, S., & Gibbons, R. D. (2007) · Biometrics · doi:10.1111/j.1541-0420.2007.00769.x · Cited by 87
“Kenward and Roger (1997) suggested a scaled F distribution as a finite sample approximation to the null distribution of an FGLS test.”
BackgroundPlant Population and Fungicide Economically Reduced Winter Wheat Yield Gap in Kansas
Jaenisch, B. R., de Oliveira Silva, A., DeWolf, E., Ruiz‐Diaz, D. A., & Lollato, R. P. (2019) · Agronomy Journal · doi:10.2134/agronj2018.03.0223 · Cited by 87
Human-resource subsidies alter the dietary preferences of a mammalian top predator
Newsome, T. M., Ballard, G., Fleming, P. J. S., van de Ven, R., Story, G. L., & Dickman, C. R. (2014) · Oecologia · doi:10.1007/s00442-014-2889-7 · Cited by 99
“…asreml residual maximum likelihood for linear mixed models (gilmour et al. 2002) in the package asreml (Butler 2009), with adjusted Wald statistics (Kenward and roger 1997). use of anthropogenic food resources to assess the significance of any trends, all site × trip combinations were subject to…”
Effects of undersown crops on soil mineral N and grain yield of spring barley
Känkänen, H., & Eriksson, C. (2007) · European Journal of Agronomy · doi:10.1016/j.eja.2007.01.010 · Cited by 86
Wessels, A. M., Siemers, E. R., Yu, P., Andersen, S. W., Holdridge, K. C., Sims, J. R., Sundell, K., Stern, Y., Rentz, D. M., Dubois, B., et al. (2015) · The Journal of Prevention of Alzheimer's Disease · doi:10.14283/jpad.2015.82 · Cited by 86
“The Kenward-Roger approximation (18) was used to estimate denominator degrees of freedom.”
MethodThivel, D., Tremblay, M. S., Katzmarzyk, P. T., Fogelholm, M., Hu, G., Maher, C., Maia, J., Olds, T., Sarmiento, O. L., Standage, M., et al. (2019) · Preventive Medicine · doi:10.1016/j.ypmed.2018.10.025 · Cited by 86
“The Kenward and Roger approximation method(25) was used to calculate denominator degrees of freedom for statistical tests pertaining to fixed effects.”
MethodGenome-wide meta-analyses of stratified depression in Generation Scotland and UK Biobank
Hall, L. S., Adams, M. J., Arnau-Soler, A., Clarke, T.-K., Howard, D. M., Zeng, Y., Davies, G., Hagenaars, S. P., Maria Fernandez-Pujals, A., Gibson, J., et al. (2018) · Translational Psychiatry · doi:10.1038/s41398-017-0034-1 · Cited by 86
Bottomley, C., Ooko, M., Gasparrini, A., & Keogh, R. H. (2023) · Statistics in Medicine · doi:10.1002/sim.9669 · Cited by 118
Dreher, T., Thomason, P., Švehlík, M., Döderlein, L., Wolf, S. I., Putz, C., Uehlein, O., Chia, K., Steinwender, G., Sangeux, M., et al. (2017) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.13618 · Cited by 94
“All fixed effects in the linear mixed model were tested using the conditional Wald F-test with Kenward and Roger adjustment.(20) Non-significant fixed effects were removed and a reduced model was fitted.”
MethodAhles, T. A., Li, Y., McDonald, B. C., Schwartz, G. N., Kaufman, P. A., Tsongalis, G. J., Moore, J. H., & Saykin, A. J. (2014) · Psycho-Oncology · doi:10.1002/pon.3545 · Cited by 85
“As a precaution to control for potentially inflated pvalues in slightly unbalanced group sizes and small samples in some sub-groups, the Kenward-Roger adjustment [24-25] was applied as part of the F-statistic for fixed effects.”
MethodRiesch, R., Martin, R. A., & Langerhans, R. B. (2012) · The American Naturalist · doi:10.1086/668597 · Cited by 97
“…L for population nested within predation regime, and F-test using restricted maximum likelihood and the Kenward-Roger degrees of freedom adjustment (Kenward and Roger 1997) for predation regime to appropriately test this fixed effect while treating population as a random term (i.e., effectively…”
MethodHasegawa, S., Macdonald, C. A., & Power, S. A. (2016) · Global Change Biology · doi:10.1111/gcb.13147 · Cited by 99
“Kenward–Roger Degrees of Freedom Approximation was employed to estimate residual degrees of freedom for F tests in LMMs using the ‘LMERTEST’ package in R 3.1.2 (Kenward & Roger, 1997; Kuznetsova et al., 2014).”
MethodInfluentialCorrelates of objectively measured sedentary time and self-reported screen time in Canadian children
LeBlanc, A. G., Broyles, S. T., Chaput, J.-P., Leduc, G., Boyer, C., Borghese, M. M., & Tremblay, M. S. (2015) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-015-0197-1 · Cited by 84
“The Kenward-Roger approximation (DDFM= KR) was used to calculate degrees of freedom [25].”
MethodConditional sex allocation in the Red Mason bee, Osmia rufa
Seidelmann, K., Ulbrich, K., & Mielenz, N. (2009) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-009-0850-2 · Cited by 84
“Due to the unbalanced nature of the design, the method of Kenward and Roger (1997) was used for estimating the degrees of freedom for”
MethodSingh, P., Heikkinen, J., Ketoja, E., Nuutinen, V., Palojärvi, A., Sheehy, J., Esala, M., Mitra, S., Alakukku, L., & Regina, K. (2015) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2015.03.027 · Cited by 104
“The statistical significances of the fixed effects were determined through F-tests using the Kenward–Roger adjustment to the degrees of freedom (Kenward and Roger, 1997).”
MethodLascano, G. J., & Heinrichs, A. J. (2009) · Livestock Science · doi:10.1016/j.livsci.2008.12.007 · Cited by 108
“All denominator degrees of freedom for F-tests were calculated according to Kenward and Roger (1997). Repeated measurements were performed for ruminal parameters such as NH3-N, VFA and pH, and the samemodel was used except that hour was added and the interaction with diet and YC was included.”
MethodPerformance of the Kenward–Roger Method when the Covariance Structure is Selected Using AIC and BIC
Gomez, E. V., Schaalje, G. B., & Fellingham, G. W. (2005) · Communications in Statistics - Simulation and Computation · doi:10.1081/sac-200055719 · Cited by 95
“Small sample properties can be improved, however, by the use of approximate Satterthwaite-type denominator degrees of freedom (Fai and Cornelius, 1996), and further improved by adjustment of the test statistic combined with the degrees of freedom adjustment (Kenward and Roger, 1997).”
MethodInfluentialSkrip, M. M., Bauchinger, U., Goymann, W., Fusani, L., Cardinale, M., Alan, R. R., & McWilliams, S. R. (2015) · Ecology and Evolution · doi:10.1002/ece3.1601 · Cited by 84
“We then used this injection mass, the molar masses of D 2 O (20 g/mol) and unlabeled H 2 O (18 g/mol), and the deuterium enrichment of distilled blood water as determined by spectrophotometry to calculate estimated water space (Karasov and Pin-show 1998; McWilliams and Whitman 2013).”
MethodHodgins, K. A., & Barrett, S. C. H. (2008) · Evolution · doi:10.1111/j.1558-5646.2008.00404.x · Cited by 96
Sex-specific patterns of yolk androgen allocation depend on maternal diet in the zebra finch
Rutstein, A. N., Gilbert, L., Slater, P. J. B., & Graves, J. A. (2004) · Behavioral Ecology · doi:10.1093/beheco/arh123 · Cited by 84
Espin, C., Wallace, T., Lembke, E., Campbell, H., & Long, J. D. (2010) · Learning Disabilities Research & Practice · doi:10.1111/j.1540-5826.2010.00304.x · Cited by 83
“Restricted maximum likelihood was used for parameter estimation, and degrees of freedom ( df ) for the t tests of the parameter estimates were estimated using the method of Kenward and Roger (1997).”
MethodBarthel, M., Sauppe, S., Levinson, S. C., & Meyer, A. S. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01858 · Cited by 83
MEASURING LONGITUDINAL WRITING DEVELOPMENT USING INDICES OF SYNTACTIC COMPLEXITY AND SOPHISTICATION
Kyle, K., Crossley, S., & Verspoor, M. (2021) · Studies in Second Language Acquisition · doi:10.1017/s0272263120000546 · Cited by 83
“In cases in which the Satterthwaite method failed to converge, the Kenward – Roger method (Kenward & Roger, 1997) was used.”
MethodFamily Functioning and the Course of Adolescent Bipolar Disorder
Sullivan, A. E., Judd, C. M., Axelson, D. A., & Miklowitz, D. J. (2012) · Behavior Therapy · doi:10.1016/j.beth.2012.04.005 · Cited by 80
“Degrees of freedom used in model denominators were determined as outlined by Kenward and Roger (1997).”
MethodStroup, W. W. (2002) · Journal of Agricultural Biological and Environmental Statistics · doi:10.1198/108571102780 · Cited by 83
“mixed model ¸ may have to be approximated, for example, using the Satterthwaite or the Kenward and Roger (1997) procedure.”
MethodSafety and utility of acute electroconvulsive therapy for agitation and aggression in dementia
Acharya, D., Harper, D. G., Achtyes, E. D., Seiner, S. J., Mahdasian, J. A., Nykamp, L. J., Adkison, L. E., White, L. V. D. S., McClintock, S. M., Ujkaj, M., et al. (2014) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.4137 · Cited by 87
“Hypothesis tests were conducted using the Kenward–Rogers adjustment (Kenward and Roger, 1997; Kowalchuk et al., 2004).”
MethodFrom eggs to fledging: negative impact of urban habitat on reproduction in two tit species
Bailly, J., Scheifler, R., Berthe, S., Clément-Demange, V.-A., Leblond, M., Pasteur, B., & Faivre, B. (2015) · Journal für Ornithologie · doi:10.1007/s10336-015-1293-3 · Cited by 97
“The models containing a random factor were analysed with the package lme4 (Bates 2010) and the pbkrtest package for the estimation of degrees of freedom for GLMMs (Kenward and Roger 1997).”
Methodde Resende, M. D. V., & Alves, R. S. (2020) · Functional Plant Breeding Journal · doi:10.35418/2526-4117/v2n2a1 · Cited by 122
“Hypothesis tests for fixed effects Under REML estimation, Wald’s W statistic has been recommended for testing fixed effects (Kenward and Roger, 1997).”
MethodMorehouse, N. I., & Rutowski, R. L. (2010) · The American Naturalist · doi:10.1086/657043 · Cited by 85
Natural and management-induced dynamics of hydraulic conductivity along a cover-cropped field slope
Bodner, G., Loiskandl, W., Buchan, G. D., & Kaul, H. (2008) · Geoderma · doi:10.1016/j.geoderma.2008.06.012 · Cited by 82
Newbold, C. J., van Zijderveld, S. M., Hulshof, R. B. A., Fokkink, W. B., Leng, R. A., Terêncio, P. H., Powers, W. J., van Adrichem, P. W. M., Paton, N. D., & Perdok, H. B. (2014) · Journal of Animal Science · doi:10.2527/jas.2014-7677 · Cited by 82
“The degrees of freedom for tests of significance were calculated using the method described by Kenward and Roger (1997) using the MIXED procedure of SAS.”
MethodDaily Stressors, Past Depression, and Metabolic Responses to High-Fat Meals: A Novel Path to Obesity
Kiecolt-Glaser, J. K., Habash, D. L., Fagundes, C. P., Andridge, R., Peng, J., Malarkey, W. B., & Belury, M. A. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.05.018 · Cited by 82
“The Kenward-Roger adjustment to the degrees of freedom was used to control type I error (37), and no adjustments were made to p values to account for multiple testing.”
MethodUsing eye tracking to test for individual differences in attention to attractive faces
Valuch, C., Pflüger, L. S., Wallner, B., Laeng, B., & Ansorge, U. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00042 · Cited by 95
“We applied the Satterthwaite (1946) approximation for computing p-values for t-statistics and the Kenward and Roger (1997) approximation for F-statistics, as implemented in the lmerTest package version 2.0-6 (Kuznetsova et al., 2014).”
MethodInfluentialTherapeutic bond judgments: Congruence and incongruence.
Atzil–Slonim, D., Bar‐Kalifa, E., Rafaeli, E., Lutz, W., Rubel, J. A., Schiefele, A.-K., & Peri, T. (2015) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000015 · Cited by 97
Acceptance and commitment therapy universal prevention program for adolescents: a feasibility study.
Burckhardt, R., Manicavasagar, V., Batterham, P. J., Hadzi-Pavlovic, D., & Shand, F. (2017) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-017-0164-5 · Cited by 81
“The parameters of the model were estimated using restricted maximum likelihood as the evidence suggests it is preferable over maximum likelihood when the sample is small [55].”
MethodNieminen, M., Ketoja, E., Mikola, J., Terhivuo, J., Sirén, T., & Nuutinen, V. (2011) · Ecological Applications · doi:10.1890/10-1801.1 · Cited by 92
Heinritz, S. N., Weiss, E., Eklund, M., Aumiller, T., Heyer, C. M. E., Messner, S., Rings, A., Louis, S., Bischoff, S., & Mosenthin, R. (2016) · Nutrients · doi:10.3390/nu8050317 · Cited by 93
“Furthermore, multiple comparisons among sampling dates within treatments were performed using a t-test, with degrees of freedom determined by the Kenward–Roger method [42].”
MethodBell, M. L., & Rabe, B. A. (2020) · Trials · doi:10.1186/s13063-020-4114-9 · Cited by 99
Piessens, K., Adriaens, D., Jacquemyn, H., & Honnay, O. (2008) · Oecologia · doi:10.1007/s00442-008-1204-x · Cited by 97
“df were estimated using the Kenward–Roger method (Kenward and Roger 1997).”
MethodHarms, M. P., Wang, L., Campanella, C., Aldridge, K., Moffitt, A. J., Kuelper, J., Ratnanather, J. T., Miller, M. I., Barch, D. M., & Csernansky, J. G. (2010) · British Journal of Psychiatry · doi:10.1192/bjp.bp.109.067314 · Cited by 81
Hébert, F., Krause, C., Plourde, P.-Y., Achim, A., Prégent, G., & Ménétrier, J. (2016) · Forests · doi:10.3390/f7110276 · Cited by 81
“We used the Kenward-Rogers method of degrees of freedom approximation, as it performs better than other testing procedures under small sample conditions [42].”
MethodNext Speakers Plan Their Turn Early and Speak after Turn-Final “Go-Signals”
Barthel, M., Meyer, A. S., & Levinson, S. C. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00393 · Cited by 80
“Statistical significance was assessed with F-tests with Kenward-Roger approximations of degrees of freedom (Kenward and Roger, 1997; Fox and Weisberg, 2011; Halekoh and Højsgaard, 2014).”
MethodHow Deliberation Happens: Enabling Deliberative Reason
Niemeyer, S., Veri, F., Dryzek, J. S., & Bächtiger, A. (2023) · American Political Science Review · doi:10.1017/s0003055423000023 · Cited by 139
“We corrected for potential loss of power due to relatively small sample size (19 cases involving 387 survey respondents among 722 deliberating individuals) 23 using Restricted Maximum Likelihood (REML) estimation (Luke 2017) with the degrees of freedom estimated following Kenward and Roger (1997).”
Gan, Y., Campbell, C. A., Liu, L., Basnyat, P., & McDonald, C. L. (2008) · Agricultural Water Management · doi:10.1016/j.agwat.2008.08.012 · Cited by 80
Manyanga, T., Tremblay, M. S., Chaput, J., Katzmarzyk, P. T., Fogelholm, M., Hu, G., Kuriyan, R., Kurpad, A. V., Lambert, E. V., Maher, C. A., et al. (2017) · BMC Public Health · doi:10.1186/s12889-017-4383-8 · Cited by 101
“Missing values for income were multiply-imputed (5 imputations) using fully conditional specification (FCS) methods, under missing at random (MAR) assumptions [28] and using SAS version 9.”
MethodIncreased sequencing depth does not increase captured diversity of arbuscular mycorrhizal fungi
Vasar, M., Andreson, R., Davison, J., Jairus, T., Moora, M., Remm, M., Young, J. P. W., Zobel, M., & Öpik, M. (2017) · Mycorrhiza · doi:10.1007/s00572-017-0791-y · Cited by 91
Effects of the stress field induced by a running tyre on the soil pore system
Berisso, F. E., Schjønning, P., Lamandé, M., Weisskopf, P., Stettler, M., & Keller, T. (2013) · Soil and Tillage Research · doi:10.1016/j.still.2013.03.005 · Cited by 79
Moose and vole browsing patterns in experimentally assembled pure and mixed forest stands
Vehviläinen, H., & Koricheva, J. (2006) · Ecography · doi:10.1111/j.0906-7590.2006.04457.x · Cited by 79
“as unstructured and the denominator (= error) degrees of freedom were calculated using the Kenward-Roger method (Kenward and Roger 1997).”
MethodCareau, V., Buttemer, W. A., & Buchanan, K. L. (2014) · Integrative and Comparative Biology · doi:10.1093/icb/icu095 · Cited by 87
“Significance of fixed effects was tested with a conditional Wald F-statistic and the denominator degrees of freedom (df) were determined following Kenward and Roger (1997).”
MethodSex differences in the neural correlates of autonomic arousal: a pilot PET study
Nugent, A. C., Bain, E. E., Thayer, J. F., Sollers, J. J., & Drevets, W. C. (2011) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2011.03.001 · Cited by 78
“Degrees of freedom were calculated using the Kenward-Roger method, a method designed to produce accurate inferences on small samples (Kenward and Roger, 1997), and shown to be robust to violations of normality (Kowalchuk et al., 2004).”
MethodPandey, A. K., Kamarajan, C., Manz, N., Chorlian, D. B., Stimus, A., & Porjesz, B. (2016) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2015.10.002 · Cited by 76
“Further exploration of main and interaction effects was performed using Wald's tests (Kenward and Roger, 1997) for pairwise comparisons and the significance levels were adjusted with Bonferroni correction for multiple comparisons.”
The Use of Two‐Way Linear Mixed Models in Multitreatment Meta‐Analysis
Piepho, H. P., Williams, E. R., & Madden, L. V. (2012) · Biometrics · doi:10.1111/j.1541-0420.2012.01786.x · Cited by 78
“…component in the sec- aResults are identical for both analyses. bAdjusted means followed by a common letter are not significantly different at α = 5% according to a t-test using the Kenward and Roger (1997) method for degrees of freedom and variance adjustments. ond stage are shown in Table 4.”
MethodZöllner, A., Ganeles, J., Korostoff, J. M., Guerra, F., Krafft, T., & Brägger, U. (2008) · Clinical Oral Implants Research · doi:10.1111/j.1600-0501.2007.01517.x · Cited by 78
McNeish, D. (2016) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998615621299 · Cited by 101
“SPSS is able to calculate Satterthwaite degrees of freedom (Satterthwaite, 1946) and SAS allows users to select from five different options including the oft recommended Kenward–Roger (Kenward & Roger, 1997, 2009).”
MethodAtkinson, S. D., Prakash, A., Zhang, Q., Pangallo, B. A., Bangs, M. E., Emslie, G. J., & March, J. S. (2014) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2013.0146 · Cited by 80
“A Kenward–Roger correction (Kenward and Roger 1997) was used to estimate de-nominator degrees of freedom.”
McNeish, D. (2026) · Psychological Methods · doi:10.1037/met0000620 · Cited by 66
Janelsins, M. C., Roscoe, J. A., Berg, M. J., Thompson, B. D., Gallagher, M. J., Morrow, G. R., Heckler, C. E., Jean‐Pierre, P., Opanashuk, L. A., & Gross, R. A. (2009) · Cancer Investigation · doi:10.3109/07357900903405942 · Cited by 88
“Hypothesis testing of the fixed effects was performed using F tests with the Kenward–Roger denominator degrees of freedom method (19).”
MethodTHE EFFECTS OF NEUROFEEDBACK TRAINING IN THE COGNITIVE DIVISION OF THE ANTERIOR CINGULATE GYRUS
CANNON, R., LUBAR, J., CONGEDO, M., THORNTON, K., TOWLER, K., & HUTCHENS, T. (2007) · International Journal of Neuroscience · doi:10.1080/00207450500514003 · Cited by 77
Kopala-Sibley, D. C., Zuroff, D. C., Russell, J. J., Moskowitz, D. S., & Paris, J. (2012) · Journal of Abnormal Psychology · doi:10.1037/a0028513 · Cited by 77
Henningsen‐Schomers, M. R., Kirilina, E. P., Weigand, A., Bajbouj, M., & Pulvermüller, F. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhu274 · Cited by 96
Analysis of cluster randomised trials with an assessment of outcome at baseline
Hooper, R., Forbes, A. B., Hemming, K., Takeda, A., & Beresford, L. (2018) · BMJ · doi:10.1136/bmj.k1121 · Cited by 95
Missing data: mechanisms, methods, and messages
Nakagawa, S. (2015) · Ecological Statistics · doi:10.1093/acprof:oso/9780199672547.003.0005 · Cited by 76
Aboab, J., Sebille, V., Jourdain, M., Mangalaboyi, J., Gharbi, M., Mansart, A., & Annane, D. (2011) · Intensive Care Medicine · doi:10.1007/s00134-011-2236-y · Cited by 76
“Effects of time and treatment as well as their interaction were included and tested as appropriate (using the Kenward and Roger adjustment for degrees-of-freedom calculation [ 24 ]) in the model along with corresponding baseline measurements.”
MethodCrowdsourcing image analysis for plant phenomics to generate ground truth data for machine learning
Zhou, N., Siegel, Z. D., Zarecor, S., Lee, N., Campbell, D. A., Andorf, C. M., Nettleton, D., Lawrence-Dill, C. J., Ganapathysubramanian, B., Kelly, J. W., et al. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006337 · Cited by 76
Forced choices reveal a trade-off between cognitive effort and physical pain
Vogel, T. A., Savelson, Z. M., Otto, A. R., & Roy, M. (2020) · eLife · doi:10.7554/elife.59410 · Cited by 96
The complex social environment of female house mice ( Mus domesticus )
König, B., & Lindholm, A. K. (2012) · Evolution of the House Mouse · doi:10.1017/cbo9781139044547.007 · Cited by 76
Uytvanck, J. V., Maes, D., Vandenhaute, D., & Hoffmann, M. (2007) · Biological Conservation · doi:10.1016/j.biocon.2007.09.001 · Cited by 76
The effect of helminth co-infection on malaria in mice: A meta-analysis
Knowles, S. C. L. (2011) · International Journal for Parasitology · doi:10.1016/j.ijpara.2011.05.009 · Cited by 96
“The Kenward and Roger (1997) method for estimating parameter estimate standard errors and denominator degrees of freedom is specifically designed for models with random effects and increases the accuracy of significance tests.”
MethodVariations of Weight Loss Following Gastric Bypass and Gastric Band
Puzziferri, N., Nakonezny, P. A., Livingston, E. H., Carmody, T. J., Provost, D. A., & Rush, A. J. (2008) · Annals of Surgery · doi:10.1097/sla.0b013e3181820cbc · Cited by 76
Mammal pollinators lured by the scent of a parasitic plant
Johnson, S. D., Burgoyne, P. M., Harder, L. D., & Dötterl, S. (2011) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2010.2175 · Cited by 76
Mate attractiveness and primary resource allocation in the zebra finch
Rutstein, A. N., Gilbert, L., Slater, P. J. B., & Graves, J. A. (2004) · Animal Behaviour · doi:10.1016/j.anbehav.2004.02.011 · Cited by 85
Population size, habitat fragmentation, and the nature of adaptive variation in a stream fish
Fraser, D. J., Debes, P. V., Bernatchez, L., & Hutchings, J. A. (2014) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2014.0370 · Cited by 88
“Significances of the fixed terms were tested using F-tests with denominator degrees of freedom (ddf) adjusted according to [33].”
MethodBrommer, J. E., & Kluen, E. (2012) · Ecology and Evolution · doi:10.1002/ece3.412 · Cited by 81
“Fixed effects were tested using a Wald F-tests with the residual degree of freedom numerically derived following Kenward and Roger (1997).”
MethodVendramini, T. H. A., Takiya, C. S., da Silva, T. H., Zanferari, F., Rentas, M. F., Bertoni, J. C., Consentini, C. E. C., Gardinal, R., Acedo, T. S., & Rennó, F. P. (2016) · Animal Feed Science and Technology · doi:10.1016/j.anifeedsci.2016.01.015 · Cited by 86
Danielson, T. M., Rivera‐Monroy, V. H., Castañeda‐Moya, E., Briceño, H. O., Travieso, R., Marx, B. D., Gaiser, E. E., & Farfán, L. M. (2017) · Forest Ecology and Management · doi:10.1016/j.foreco.2017.08.009 · Cited by 77
Monk, B. J., Huang, H. Q., Burger, R. A., Mannel, R. S., Homesley, H. D., Fowler, J. M., Greer, B. E., Boente, M. P., Liang, S. X., & Wenzel, L. B. (2012) · Gynecologic Oncology · doi:10.1016/j.ygyno.2012.11.038 · Cited by 78
Changes in limiting resources determine spatio-temporal variability in tree–grass interactions
Gea‐Izquierdo, G., Montero, G., & Cañellas, I. (2009) · Agroforestry Systems · doi:10.1007/s10457-009-9211-4 · Cited by 74
“To test for differences within fixed effects we used contrasts compared to an F distribution, with degrees of freedom calculated following Kenward and Roger (1997).”
Schlatter, M. C., & Cameron, L. D. (2010) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9204-6 · Cited by 74
“The models included the intercept, with degrees of freedom calculated using the Kenward–Roger method [71].”
MethodTobin, D., Kause, A., Mäntysaari, E. A., Martín, S., Houlihan, D. F., Dobly, A., Kiessling, A., Rungruangsak-Torrissen, K., Ritola, O., & Ruohonen, K. (2006) · Aquaculture · doi:10.1016/j.aquaculture.2006.07.023 · Cited by 90
“To take into account the data structure, the method of Kenward and Roger (1997) was used to calculate correct standard errors of diet means, F-tests and their degrees of freedom for the fixed effects.”
MethodOgutu, J. O. (2013) · The Open Conservation Biology Journal · doi:10.2174/1874839201307010011 · Cited by 74
Hernández‐Sánchez, J., Visscher, P. M., Plastow, G. S., & Haley, C. (2003) · Genetics · doi:10.1093/genetics/164.2.637 · Cited by 84
Discovery of beneficial haplotypes for complex traits in maize landraces
Mayer, M., Hölker, A. C., González‐Segovia, E., Bauer, E., Presterl, T., Ouzunova, M., Melchinger, A. E., & Schön, C. (2020) · Nature Communications · doi:10.1038/s41467-020-18683-3 · Cited by 90
Banerjee, A., Camps, J., Zacur, E., Andrews, C. M., Rudy, Y., Choudhury, R. P., Rodríguez, B., & Grau, V. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0257 · Cited by 80
“corrections over all 20 subjects, we have adopted the linear mixed model analysis, with Kenward and Roger’s method applied for the degrees of freedom of the t-statistic [40].”
MethodWestbrook, J. W., Resende, M. F. R., Muńoz, P., Walker, A. R., Wegrzyn, J., Nelson, C. D., Neale, D. B., Kirst, M., Huber, D. A., Gezan, S. A., et al. (2013) · New Phytologist · doi:10.1111/nph.12240 · Cited by 80
“The effects of fixed factors, which included l (the overall trait means), s (site effects), and r (the effects of replicate within site), were assessed with approximate F-tests (Kenward & Roger, 1997).”
MethodInsomnia Moderates Outcome of Serotonin-Selective Reuptake Inhibitor Treatment in Depressed Youth
Emslie, G. J., Kennard, B. D., Mayes, T. L., Nakonezny, P. A., Zhu, L., Tao, R., Hughes, C., & Croarkin, P. (2012) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2011.0096 · Cited by 73
“Restricted maximum likelihood estimation and Type 3 tests of fixed effects were used, with the Kenward–Roger correction (Kenward and Roger 1997) applied to the spatial-power covariance structure for unequally spaced repeated measures.”
MethodKinematics and wing shape across flight speed in the bat,Leptonycteris yerbabuenae
von Busse, R., Hedenström, A., Winter, Y., & Johansson, L. (2012) · Biology Open · doi:10.1242/bio.20122964 · Cited by 73
“For all statistical tests we used mixed linear models using the GLM procedure in JMP 8.0 (SAS Institute, Cary, North Carolina, USA) with reduced maximum likelihood estimates (REML) of the variance, which produce unbiased estimates of variance and covariance parameters (Kenward and Roger, 1997).”
BackgroundMethodInfluentialGabel, V., Reichert, C. F., Maire, M., Schmidt, C., Schlangen, L. J. M., Kolodyazhniy, V., Garbazza, C., Cajochen, C., & Viola, A. U. (2017) · Scientific Reports · doi:10.1038/s41598-017-07060-8 · Cited by 72
One hundred years of the design of experiments on and off the pages of Biometrika
Atkinson, A. C. (2001) · Biometrika · doi:10.1093/biomet/88.1.53 · Cited by 86
McKie, B. G., Schindler, M. H., Gessner, M. O., & Malmqvist, B. (2009) · Oecologia · doi:10.1007/s00442-009-1336-7 · Cited by 88
“…on reliable solutions when species richness was included in these models, reflecting the very restricted variance around some mean estimates and the small sample size of the three-species treatment (both potentially undermining REML estimation; see Kenward and Roger 1997; Searle et al. 1992).”
BackgroundRahman, L., Whitelaw-Weckert, M. A., & Orchard, B. A. (2014) · Soil Research · doi:10.1071/sr14041 · Cited by 79
Emotional Eating, Health Behaviours, and Obesity in Children: A 12-Country Cross-Sectional Study
Jalo, E., Konttinen, H. M., Vepsäläinen, H., Chaput, J., Hu, G., Maher, C. A., Maia, J. A. R., Sarmiento, O. L., Standage, M., Tudor‐Locke, C., et al. (2019) · Nutrients · doi:10.3390/nu11020351 · Cited by 88
“The denominator degrees of freedom for statistical tests pertaining to fixed effects were calculated using the Kenward and Roger approximation [47].”
MethodHydrolyzable microplastics in soil—low biodegradation but formation of a specific microbial habitat?
Schöpfer, L., Schnepf, U., Marhan, S., Brümmer, F., Kandeler, E., & Pagel, H. (2022) · Biology and Fertility of Soils · doi:10.1007/s00374-022-01638-9 · Cited by 78
“Next, we conducted ANOVAs, approximating degrees of freedom with the Kenward-Roger method (Kenward and Roger 1997) provided in the lmerTest 3.1–2 package (Kuznetsova et al.”
MethodInfluentialReynolds, L., Cavadino, A., Chin, S., Little, Z., Akroyd, A., Tennant, G., Dobson, R., Broom, R. J., & Gautier, A. (2022) · BMC Cancer · doi:10.1186/s12885-021-09081-z · Cited by 85
“The Kenward and Roger adjustment was also used, which can control type I errors in smaller samples [35].”
MethodAttention Training for School-Aged Children With ADHD: Results of an Open Trial
Tamm, L., Hughes, C., Ames, L., Pickering, J., Silver, C. H., Stavinoha, P., Castillo, C. L., Rintelmann, J., Moore, J., Foxwell, A., et al. (2010) · Journal of Attention Disorders · doi:10.1177/1087054709347446 · Cited by 72
“Restricted maximum likelihood estimation and Type III tests of fixed effects were used, with the Kenward–Roger correction (Kenward & Roger, 1997) applied to the unstructured covariance model.”
MethodQuist, M. C., Guy, C. S., Bernot, R. J., & Stephen, J. L. (2002) · Journal of Fish Biology · doi:10.1006/jfbi.2002.2130 · Cited by 72
Comparison of SGA Oral Medications and a Long-Acting Injectable SGA: The PROACTIVE Study
Buckley, P. F., Schooler, N. R., Goff, D. C., Hsiao, J., Kopelowicz, A., Lauriello, J., Manschreck, T., Mendelowitz, A. J., Miller, D. D., Severe, J. B., et al. (2015) · Schizophrenia Bulletin · doi:10.1093/schbul/sbu067 · Cited by 72
“Although age at first episode (23) and number of psychiatric hospitalizations (11) were comparable, LAI-R subjects had been hospitalized more recently (mean 22.”
Geostatistical mapping of geomorphic variables in the presence of trend
Lark, R. M., & Webster, R. (2006) · Earth Surface Processes and Landforms · doi:10.1002/esp.1296 · Cited by 72
“The number of degrees of freedom in the denominator was computed by the procedure of Kenward and Roger (1997), which allows for correlation of the error terms and computes an adjustment for the Wald statistic.”
MethodGuillory, J., Chang, P., Henderson, C. R., Shengelia, R., Lama, S., Warmington, M., Jowza, M., Waldman, S., Gay, G., & Reid, M. C. (2015) · The Clinical Journal of Pain · doi:10.1097/ajp.0000000000000193 · Cited by 72
Cutforth, H. W., Angadi, S. V., McConkey, B. G., Miller, P. R., Ulrich, D., Gulden, R. H., Volkmar, K. M., Entz, M. H., & Brandt, S. A. (2013) · Canadian Journal of Soil Science · doi:10.4141/cjss2012-081 · Cited by 87
Experimental test of parasitism hypothesis for population cycles of a forest lepidopteran
Klemola, N., Andersson, T., Ruohomäki, K., & Klemola, T. (2010) · Ecology · doi:10.1890/09-2076.1 · Cited by 75
Metadata
- Authors
- Kenward, M. G., & Roger, J. H.
- Affiliations
- University of Kent (GB)
- Venue
- Biometrics
- Publisher
- JSTOR
- Published
- 1997-09
- Volume
- 53
- Issue
- 3
- Pages
- 983
- ISSN
- 0006-341X (ISSN-L)
- OpenAlex
- W1981747359
- Cited by
- 4,552
- References
- 23
- Citations by year
- 2026: 217 · 2025: 196 · 2024: 216 · 2023: 213 · 2022: 261 · 2021: 299 · 2020: 287 · 2019: 270 · 2018: 247 · 2017: 240 · 2016: 286 · 2015: 256 · 2014: 217 · 2013: 189 · 2012: 218
- FWCI
- 7.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrics (publishedVersion)
- PubMed (publishedVersion)
- Rothamsted Repository (Rothamsted Repository) (submittedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Applied Mathematics › Statistical and numerical algorithms
- Keywords
- restricted maximum likelihood, small sample inference, fixed effects, Wald statistic, covariance matrix estimation, small sample bias, Hotelling's T2 statistic, linear mixed models, linear Gaussian models
- MeSH terms
- Analysis of Variance, Biometry, Clinical Trials as Topic, Humans, Intermittent Claudication, Nitrogen Fixation, Plants, Probability, Regression Analysis, Research Design, Rhizobium, Soil Microbiology, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Cross-Over Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.