Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Restricted maximum likelihood (REML) is now well established as a method for estimating the parameters of the general Gaussian linear model with a structured covariance matrix, in particular for mixed linear models. Conventionally, estimates of precision and inference for fixed effects are based on their asymptotic distribution, which is known to be inadequate for some small-sample problems. In this paper, we present a scaled Wald statistic, together with an F approximation to its sampling distribution, that is shown to perform well in a range of small sample settings. The statistic uses an adjusted estimator of the covariance matrix that has reduced small sample bias. This approach has the advantage that it reproduces both the statistics and F distributions in those settings where the latter is exact, namely for Hotelling T2 type statistics and for analysis of variance F-ratios. The performance of the modified statistics is assessed through simulation studies of four different REML analyses and the methods are illustrated using three examples.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Accurate model parameter estimations were ensured by use of the residual maximum likelihood approach with the Kenward and Roger, 1997 approximation of degrees of freedom.”
From Longitudinal association between social media use and psychological distress among adolescents (2020) · cites this paper as Kenward and Roger, 1997
In the citing paper
“Not only does the data in the motivating example have different flowering days for each genotype (see for example Fig. 2), but the proposed methodology also accounts for the different strata in the Wald test by approximating the denominator degrees of freedom (Kenward and Roger, 1997).”
From Incorporating environmental covariates to explore genotype × environment × management (G × E × M) interactions: A one-stage predictive model (2023) · cites this paper as Kenward and Roger, 1997
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Random-Effects Models for Longitudinal Data
Laird, N. M., & Ware, J. H. (1982) · Biometrics · doi:10.2307/2529876 · Cited by 8,923
Recovery of inter-block information when block sizes are unequal
PATTERSON, H. D., & THOMPSON, R. (1971) · Biometrika · doi:10.1093/biomet/58.3.545 · Cited by 4,058
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667 · cited 1×
“Restricted (or residual) maximum likelihood (REML) has become well established for the estimation of the parameters of Z (Patterson and Thompson, 1971; Harville, 1977; Thompson, 1980; Robinson, 1991) and the corresponding estimator of 3, 3 say, has a generalized least squares form that uses the REML estimate of Z.”
BackgroundThat BLUP is a Good Thing: The Estimation of Random Effects
Robinson, G. K. (1991) · Statistical Science · doi:10.1214/ss/1177011926 · Cited by 1,686
Gilmour, A. R., Thompson, R., & Cullis, B. R. (1995) · Biometrics · doi:10.2307/2533274 · Cited by 915
15 Design and Analysis of Cross-Over Trials
Kenward, M. G., & Jones, B. (2007) · Handbook of Statistics · doi:10.1016/s0169-7161(07)27015-4 · Cited by 871
Self-citationPrinciples of Multivariate Analysis
Ling, R. F. (1990) · Technometrics · doi:10.1080/00401706.1990.10484645 · Cited by 812
Satterthwaite, F. E. (1941) · Psychometrika · doi:10.1007/bf02288586 · Cited by 809
Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models
Kackar, R. N., & Harville, D. A. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10477102 · Cited by 445
Cited by
Items 501–600 of 1,051
Anderson, S. R., Gianola, M., Perry, J. M., & Losin, E. A. R. (2020) · Pain Medicine · doi:10.1093/pm/pnaa258 · Cited by 74
“Planned pairwise mean differences were calculated for significant interaction effects using estimated marginal means [58, 59].”
MethodAn assessment of statistical methods for nonindependent data in ecological meta‐analyses
Song, C., Peacor, S. D., Osenberg, C. W., & Bence, J. R. (2020) · Ecology · doi:10.1002/ecy.3184 · Cited by 71
“In the general mixed-effect model literature, this issue is addressed by adjusting the degrees of freedom for a t or F test (Kenward and Roger 1997).”
MethodPER3 polymorphism and cardiac autonomic control: effects of sleep debt and circadian phase
Viola, A. U., James, L. M., Archer, S. N., & Dijk, D.-J. (2008) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00662.2008 · Cited by 71
Thivierge, S. C., Jean, L., & Simard, M. (2013) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2013.03.008 · Cited by 87
Randomized Dose-Ranging Trial of Tamoxifen at Low Doses in Hormone Replacement Therapy Users
DeCensi, A., Gandini, S., Serrano, D., Cazzaniga, M., Pizzamiglio, M., Maffini, F. A., Pelosi, G., Daldoss, C., Omodei, U., Johansson, H. A., et al. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.09.4318 · Cited by 73
Missing data in clinical trials: control‐based mean imputation and sensitivity analysis
Mehrotra, D. V., Liu, F., & Permutt, T. (2017) · Pharmaceutical Statistics · doi:10.1002/pst.1817 · Cited by 72
Goyarts, T., Dänicke, S., Rothkötter, H. ‐., Spilke, J., Tiemann, U., & Schollenberger, M. (2005) · Zentralblatt für Veterinärmedizin Reihe A · doi:10.1111/j.1439-0442.2005.00734.x · Cited by 71
Zocchi, A., Girlanda, E., Varnier, G., Sartori, I., Zanetti, L., Wildish, G. A., Lennon, M., Mugnaini, M., & Heidbreder, C. A. (2003) · Synapse · doi:10.1002/syn.10271 · Cited by 71
Vuorio, K. M., Lagus, A., Lehtimäki, J. M., Suomela, J., & Helminen, H. (2005) · Journal of Experimental Marine Biology and Ecology · doi:10.1016/j.jembe.2005.02.006 · Cited by 80
Kiecolt-Glaser, J. K., Jaremka, L., Andridge, R., Peng, J., Habash, D., Fagundes, C. P., Glaser, R., Malarkey, W. B., & Belury, M. A. (2015) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2014.11.018 · Cited by 70
Sleep patterns and sugar-sweetened beverage consumption among children from around the world
Chaput, J., Tremblay, M. S., Katzmarzyk, P. T., Fogelholm, M., Hu, G., Maher, C. A., Maia, J. A. R., Olds, T., Onywera, V. O., Sarmiento, O. L., et al. (2018) · Public Health Nutrition · doi:10.1017/s1368980018000976 · Cited by 82
“The denominator degrees of freedom for statistical tests pertaining to fixed effects were calculated using the Kenward and Roger approximation((23)).”
MethodExperimental evidence that thermal selection shapes mitochondrial genome evolution
Lajbner, Z., Pnini, R., Camus, M. F., Miller, J., & Dowling, D. K. (2018) · Scientific Reports · doi:10.1038/s41598-018-27805-3 · Cited by 97
Mixed‐Model Analysis of Crossover Genotype–Environment Interactions
Yang, R. (2007) · Crop Science · doi:10.2135/cropsci2006.09.0611 · Cited by 68
“The statistic in Eq. [11] is approximately t distributed, with the approximate degrees of freedom being estimated according to McLean and Sanders (1988) and Kenward and Roger (1997) .”
BackgroundMethodInfluentialOn Efficiency of Constrained Longitudinal Data Analysis Versus Longitudinal Analysis of Covariance
Lu, K. (2010) · Biometrics · doi:10.1111/j.1541-0420.2009.01332.x · Cited by 69
“Kenward and Roger (1997) provide an approximation that involves inflating the estimated variance–covariance matrix of the mean parameter estimates and computing Satterthwaitetype degrees of freedom.”
BackgroundThe effect of skewness and kurtosis on the robustness of linear mixed models
Arnau, J., Bendayan, R., Blanca, M. J., & Bono, R. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0306-x · Cited by 69
“However, small sample properties can be improved by procedures that adjust the degrees of freedom—for example, the method developed by Kenward and Roger (1997). The KR procedure provides an adjusted estimator of the covariance matrix of β that has reduced the bias for small sample inference when the asymptotic covaria…”
Oral curcumin for radiation dermatitis: a URCC NCORP study of 686 breast cancer patients
Wolf, J. R., Heckler, C. E., Guido, J. J., Peoples, A. R., Gewandter, J. S., Ling, M. N., Vinciguerra, V. P., Anderson, T., Evans, L. S., Wade, J. L., et al. (2017) · Supportive Care in Cancer · doi:10.1007/s00520-017-3957-4 · Cited by 77
“Restricted Maximum Likelihood estimation was used, and the KenwardRoger method was used for the F tests [25].”
MethodEffects of methylmercury exposure on the immune function of juvenile common loons (Gavia immer)
Kenow, K. P., Grasman, K. A., Hines, R. K., Meyer, M. W., Gendron‐Fitzpatrick, A., Spalding, M. G., & Gray, B. R. (2007) · Environmental Toxicology and Chemistry · doi:10.1897/06-442r.1 · Cited by 81
Moseby, K. E., & Hill, B. M. (2011) · Wildlife Research · doi:10.1071/wr10235 · Cited by 69
“The significance of fixed effects was determined using Wald’s statistic (Kenward and Roger 1997).”
MethodCornwallis, C. K., & O’Connor, E. (2009) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2009.0807 · Cited by 69
“The Kenward & Roger (1997) method for calculating denominator degrees of freedom was used, which is specifically designed for analysing unbalanced repeated measures data with models that contain multiple random effects (Kenward & Roger 1997; Littell et al. 2006).”
MethodCroarkin, P. E., Nakonezny, P. A., Wall, C. A., Murphy, L. L., Sampson, S. M., Frye, M. A., & Port, J. D. (2016) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2015.11.005 · Cited by 69
Long-Term Experiments with cropping systems: Case studies on data analysis
Onofri, A., Seddaiu, G., & Piepho, H. (2016) · European Journal of Agronomy · doi:10.1016/j.eja.2016.02.005 · Cited by 69
Schütz, K. E., Clark, K., Cox, N., Lr, M., & Tucker, C. B. (2010) · Animal Welfare · doi:10.1017/s0962728600001858 · Cited by 69
Multilevel modeling of single-case data: A comparison of maximum likelihood and Bayesian estimation.
Moeyaert, M., Rindskopf, D., Onghena, P., & Van den Noortgate, W. (2017) · Psychological Methods · doi:10.1037/met0000136 · Cited by 68
A SIMULATION STUDY TO EVALUATE PROC MIXED ANALYSIS OF REPEATED MEASURES DATA
Guerin, L., & Stroup, W. W. (2000) · Conference on Applied Statistics in Agriculture · doi:10.4148/2475-7772.1249 · Cited by 69
“Kenward and Roger (1997) obtained a bias correction for standard errors and test statistics that is implemented in SAS Version 8.”
Multilevel Modeling and Ordinary Least Squares Regression: How Comparable Are They?
Huang, F. L. (2017) · The Journal of Experimental Education · doi:10.1080/00220973.2016.1277339 · Cited by 83
Barch, D. M., Belden, A. C., Tillman, R., Whalen, D., & Luby, J. L. (2018) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2017.12.011 · Cited by 69
“Degreesof-freedom calculations used the method of Kenward and Roger.52 We used similar growth curve models to ask whether externalizing symptoms (as internalizing symptoms were not significant; see below) and IFG connectivity predicted by ACEs covaried across scan waves.”
MethodShippam, W., Preston, R., Douglas, J., Taylor, J., Albert, A., & Chau, A. M. T. (2019) · Anaesthesia · doi:10.1111/anae.14567 · Cited by 73
“If a significant interaction was found, we followed it up with pairwise tests of the estimated marginal means between the groups at each time-point (e.g. HFNO baseline vs. control baseline) using the Kenward–Roger method to estimate confidence intervals, and controlling for multiple comparisons using Tukey’smethod [16…”
MethodAdrian, M., McCauley, E., Berk, M. S., Asarnow, J. R., Korslund, K., Avina, C., Gallop, R., & Linehan, M. M. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13099 · Cited by 68
“(Agresti, 1990; Kenward & Roger, 1997) Outcomes analyses adjusted for site and assessed for differential treatment effects across site by including a site-by-treatment interaction.”
MethodMarcato, F., van den Brand, H., Kemp, B., Engel, B., Wolthuis-Fillerup, M., & van Reenen, K. (2020) · Journal of Dairy Science · doi:10.3168/jds.2019-17445 · Cited by 62
Neonatal Adiposity and Childhood Obesity
Moore, B. F., Harrall, K. K., Sauder, K. A., Glueck, D. H., & Dabelea, D. (2020) · PEDIATRICS · doi:10.1542/peds.2020-0737 · Cited by 77
Cognitive exposure versus avoidance in patients with chronic pain: Adherence matters
Nicholas, M. K., Asghari, A., Sharpe, L., Brnabic, A., Wood, B. M., Overton, S., Tonkin, L., de Sousa, M., Finniss, D., Beeston, L., et al. (2014) · European Journal of Pain · doi:10.1002/j.1532-2149.2013.00383.x · Cited by 68
“An unstructured covariance matrix was used to estimate within-patient errors, and the method of Kenward and Roger (1997) was used to estimate denominator degrees of freedom.”
MethodTemporal dynamics for soil aggregates determined using X-ray CT scanning
Garbout, A., Munkholm, L. J., & Hansen, S. B. (2013) · Geoderma · doi:10.1016/j.geoderma.2013.04.004 · Cited by 68
TESTING HYPOTHESES ASSOCIATED WITH BIRD RESPONSES TO WILDFIRE
Lindenmayer, D. B., Wood, J. T., Cunningham, R. B., MacGregor, C. I., Crane, M. J., Michael, D., Montague‐Drake, R., Brown, D., Muntz, R., & Gill, A. M. (2008) · Ecological Applications · doi:10.1890/07-1943.1 · Cited by 79
Predictors of Drug Shortages and Association with Generic Drug Prices: A Retrospective Cohort Study
Dave, C. V., Pawar, A. M., Fox, E. R., Brill, G. E., & Kesselheim, A. S. (2018) · Value in Health · doi:10.1016/j.jval.2018.04.1826 · Cited by 71
Wittenborn, A. K., Liu, T., Ridenour, T. A., Lachmar, E. M., Mitchell, E. A., & Seedall, R. B. (2018) · Journal of Marital and Family Therapy · doi:10.1111/jmft.12350 · Cited by 80
VILLALPANDO, S. N., WILLIAMS, R. S., & NORBY, R. J. (2009) · Global Change Biology · doi:10.1111/j.1365-2486.2008.01721.x · Cited by 67
“Denominator degrees of freedom were estimated using the r 2009 The Authors Journal compilation r 2009 Blackwell Publishing Ltd, Global Change Biology, 15, 930–942 Kenward–Roger method (Kenward & Roger, 1997).”
MethodBerthelot, N., Paccalet, T., Gilbert, E., Moreau, I., Mérette, C., Gingras, N., Rouleau, N., & Maziade, M. (2015) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.140211 · Cited by 83
Factors determining the home ranges of pet cats: A meta-analysis
Hall, C. M., Bryant, K. A., Haskard, K., Major, T., Bruce, S., & Calver, M. C. (2016) · Biological Conservation · doi:10.1016/j.biocon.2016.09.029 · Cited by 67
“There is no universally agreed way of calculating the denominator degrees of freedom (DDF) for small sample inference in mixed effects models using REML (Kenward and Roger, 1997; Schaalje et al., 2002).”
MethodStored root carbohydrates can maintain root respiration for extended periods
Aubrey, D. P., & Teskey, R. O. (2017) · New Phytologist · doi:10.1111/nph.14972 · Cited by 73
Kuiper, J. R., Stapleton, H. M., Wills-Karp, M. A., Wang, X., Burd, I. D., & Buckley, J. P. (2020) · Environmental Health · doi:10.1186/s12940-020-00610-0 · Cited by 69
Greenhouse–Geisser Adjustment and the ANOVA-Type Statistic: Cousins or Twins?
Bathke, A. C., Schabenberger, O., Tobias, R. D., & Madden, L. V. (2009) · The American Statistician · doi:10.1198/tast.2009.08187 · Cited by 67
Cardona-Olarte, P., Twilley, R. R., Krauss, K. W., & Rivera‐Monroy, V. H. (2006) · Hydrobiologia · doi:10.1007/s10750-006-0140-1 · Cited by 67
“This methodology requires a different approach than the traditional mean square methods to calculate the degrees of freedom and estimate treatment effects (Kenward & Roger, 1997).”
MethodAnalysing repeated measurements in soil monitoring and experimentation
Webster, R., & Payne, R. W. (2002) · European Journal of Soil Science · doi:10.1046/j.1365-2389.2002.00408.x · Cited by 67
“Kenward & Roger (1997) point out that small samples and missing data are common in experiments with repeated measurements and that appropriate adjustments are desirable.”
BackgroundTillage and Fertilizer Effects on Crop Yield and Soil Properties over 45 Years in Southern Illinois
Cook, R. L., & Trlica, A. (2016) · Agronomy Journal · doi:10.2134/agronj2015.0397 · Cited by 75
Is telomere length a molecular marker of past thermal stress in wild fish?
Debes, P. V., Visse, M., Panda, B., Ilmonen, P., & Vasemägi, A. (2016) · Molecular Ecology · doi:10.1111/mec.13856 · Cited by 72
Harriman, V. B., Dawson, R. D., Bortolotti, L. E., & Clark, R. G. (2017) · Ecology and Evolution · doi:10.1002/ece3.2815 · Cited by 67
Geremia, E. V., Crestani, S., Mascheroni, J. D. C., Carnevalli, R. A., Mourão, G. B., & da Silva, S. C. (2018) · Livestock Science · doi:10.1016/j.livsci.2018.03.020 · Cited by 67
“The correlation/covariance matrix was chosen using the Schwarz's Bayesian Criterion (SBC or BIC in SAS) (Littell et al., 2000), and the correction for degrees of freedom made using the Kenward and Rogers (1997) method (SAS command: DDFM=KR).”
MethodAfter-effects of 10 Hz tACS over the prefrontal cortex on phonological word decisions
Moliadze, V., Sierau, L., Lyzhko, E., Stenner, T., Werchowski, M., Siniatchkin, M., & Hartwigsen, G. (2019) · Brain stimulation · doi:10.1016/j.brs.2019.06.021 · Cited by 74
“001), but no significant interaction (F(4, 42) = 0.”
Inflorescence architecture and wind pollination in six grass species
Friedman, J., & Harder, L. D. (2004) · Functional Ecology · doi:10.1111/j.0269-8463.2004.00921.x · Cited by 74
Novello, S., Kaiser, R., Mellemgaard, A., Douillard, J.-Y., Orlov, S., Krzakowski, M., von Pawel, J., Gottfried, M., Bondarenko, I., Liao, M., et al. (2015) · European Journal of Cancer · doi:10.1016/j.ejca.2014.11.015 · Cited by 66
“The treatment effect was estimated as the average difference between the treatment group scores, together with the 95% CIs and associated p-values based on a t-statistic with degrees of freedom calculated using the Kenward–Roger method [26].”
Sorg, D., Difford, G. F., Mühlbach, S., Kuhla, B., Swalve, H. H., Lassen, J., Strabel, T., & Pszczoła, M. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.08.024 · Cited by 72
Campbell, M. S., Ouyang, A., Krishnakumar, I. M., Charnigo, R. J., Westgate, P. M., & Fleenor, B. S. (2019) · Nutrition · doi:10.1016/j.nut.2019.01.002 · Cited by 74
Raczka, N. C., Piñeiro, J., Tfaily, M., Chu, R., Lipton, M. S. W., Paša‐Tolić, L., Morrissey, E. M., & Brzostek, E. (2021) · Scientific Reports · doi:10.1038/s41598-021-97942-9 · Cited by 76
Efficacy and Safety of Roluperidone for the Treatment of Negative Symptoms of Schizophrenia
Davidson, M., Saoud, J. B., Staner, C., Noël, N., Werner, S., Luthringer, E., Walling, D. P., Weiser, M., Harvey, P. D., Strauss, G. P., et al. (2022) · Schizophrenia Bulletin · doi:10.1093/schbul/sbac013 · Cited by 68
Tillage Effects on Soil Carbon Balance in a Semiarid Agroecosystem
Bono, A., Álvarez, R., Buschiazzo, D. E., & Cantet, R. J. C. (2008) · Soil Science Society of America Journal · doi:10.2136/sssaj2007.0250 · Cited by 76
Möller, C. M., Karlgren, L., Sandell, A., Falkenström, F., & Philips, B. (2016) · Psychotherapy Research · doi:10.1080/10503307.2016.1158433 · Cited by 79
“Kenward, M. G., & Roger, J. H. (2009).”
MethodAccommodating Small Sample Sizes in Three-Level Models When the Third Level is Incidental
McNeish, D., & Wentzel, K. R. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2016.1262236 · Cited by 73
“Studies have found that the Kenward-Roger (KR) correction (Kenward & Roger, 1997; 2009) can address both of these issues (Bell et al., 2014; Ferron et al., 2009; McNeish & Stapleton, 2016b).”
BackgroundRepeatability of glucocorticoid hormones in vertebrates: a meta-analysis
Schoenemann, K. L., & Bonier, F. (2018) · PeerJ · doi:10.7717/peerj.4398 · Cited by 70
Manyanga, T., Barnes, J. D., Chaput, J., Katzmarzyk, P. T., Prista, A., & Tremblay, M. S. (2019) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-019-0861-y · Cited by 81
“Denominator degrees of freedom pertaining to fixed effects were calculated using the Kenward Roger approximation (DDFM=KR) [37].”
MethodMOORE, D. J. P., Aref, S., Ho, M. R., Pippen, J. S., Hamilton, J. G., & DeLucia, E. H. (2006) · Global Change Biology · doi:10.1111/j.1365-2486.2006.01189.x · Cited by 74
“A dominance hierarchy has established among pines and individual crowns were classified as either emergent, codominant or suppressed based on their position relative to the average canopy height in each plot (Naidu et al., 1998).”
Spatial smoothing of autocorrelations to control the degrees of freedom in fMRI analysis
Worsley, K. J. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.02.007 · Cited by 66
“Note that with modest smoothing, the effective df increases, the resulting P = 0.05 threshold decreases, and roughly twice as much activation is detected. theory uses the same techniques as Kenward & Roger (1997) and Kiebel et al. (2003).”
InfluentialResults of the citalopram to enhance cognition in Huntington disease trial
Beglinger, L. J., Adams, W. H., Langbehn, D., Fiedorowicz, J. G., Jorge, R., Biglan, K., Caviness, J., Olson, B., Robinson, R. G., Kieburtz, K., et al. (2014) · Movement Disorders · doi:10.1002/mds.25750 · Cited by 65
“We used a Kenward-Roger correction of degrees of freedom in the t statistic.(31)”
MethodReisig, D., & Godfrey, L. D. (2007) · Environmental Entomology · doi:10.1603/0046-225x(2007)36[1466:srocah]2.0.co;2 · Cited by 73
Stability of hand force production. II. Ascending and descending synergies
Reschechtko, S., & Latash, M. L. (2018) · Journal of Neurophysiology · doi:10.1152/jn.00045.2018 · Cited by 66
“F-test degrees of freedom were computed using the Kenward-Roger method (Kenward and Roger 1997), as implemented by SAS 9.4.”
MethodInfluentialNitrogen storage dynamics are affected by masting events in Fagus crenata
Han, Q., Kabeya, D., Iio, A., Inagaki, Y., & Kakubari, Y. (2013) · Oecologia · doi:10.1007/s00442-013-2824-3 · Cited by 65
Huber, J. E., Darling, M., Francis, E. J., & Zhang, D. (2012) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2012/11-0059 · Cited by 65
The consequences of monoecy and protogyny for mating in wind‐pollinatedCarex
Friedman, J., & Barrett, S. C. H. (2008) · New Phytologist · doi:10.1111/j.1469-8137.2008.02664.x · Cited by 65
Robinson, K. E., Kaizar, E. E., Catroppa, C., Godfrey, C., & Yeates, K. O. (2014) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsu031 · Cited by 77
“We used the KenwardRoger adjusted F-test to test for an effect of participant type (Kenward & Roger, 1997).”
MethodSerotonin 2c receptor RNA editing in major depression and suicide
Lyddon, R., Dwork, A. J., Keddache, M. A., Siever, L. J., & Dracheva, S. M. (2012) · The World Journal of Biological Psychiatry · doi:10.3109/15622975.2011.630406 · Cited by 63
Peippo, J., Vartia, K., Kananen-Anttila, K., Räty, M., Korhonen, K., Hurme, T., Myllymäki, H., Sairanen, A., & Mäki‐Tanila, A. (2008) · Animal Reproduction Science · doi:10.1016/j.anireprosci.2008.02.002 · Cited by 64
“Analyses of Experiment 2 were conducted with the MIXED procedure of SAS 9 (SAS Institute, Cary, NC, USA) using REML and degrees of freedom were calculated according to Kenward and Roger (1997). Model assumptions were checked graphically.”
BackgroundMethodMaziade, M., Rouleau, N., Mérette, C., Cellard, C., Battaglia, M., Marino, C., Jomphe, V., Gilbert, E., Achim, A. M., Bouchard, R.-H., et al. (2010) · Schizophrenia Bulletin · doi:10.1093/schbul/sbq026 · Cited by 72
Pino, F., & Heinrichs, A. J. (2016) · Journal of Dairy Science · doi:10.3168/jds.2015-10034 · Cited by 63
“All denominator degrees of freedom for F-tests were calculated according to Kenward and Roger (1997). The model used was”
MethodFlight, L., Allison, A., Dimairo, M., Lee, E., Mandefield, L., & Walters, S. J. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0249-5 · Cited by 64
“A number of methods which include Satterthwaite [20] and Kenward-Roger [21] approximations have been proposed to correct degrees of freedom.”
MethodDreimüller, N., Lieb, K., Tadić, A., Engelmann, J., Wollschläger, D., & Wagner, S. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.06.067 · Cited by 69
Lukey, P. T., Coello, C. S., Gunn, R. N., Parker, C. A., Wilson, F. J., Saleem, A., Garman, N., Costa, M. J., Kendrick, S. F. W., Onega, M., et al. (2019) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-019-04586-z · Cited by 68
“The probability ratio > 1 and probability ratio > 2 were also computed using the calculated t statistic and Kenward-Roger degrees of freedom [29] from the model.”
MethodLindenmayer, D. B., Knight, E., McBurney, L., Michael, D. D., & Banks, S. C. (2010) · Forest Ecology and Management · doi:10.1016/j.foreco.2010.08.047 · Cited by 63
Cunningham, R. B., Lindenmayer, D. B., Crane, M. J., Michael, D., Barton, P. S., Gibbons, P., Okada, S., Ikin, K., & Stein, J. (2013) · Diversity and Distributions · doi:10.1111/ddi.12145 · Cited by 72
“We assessed statistical significance of effects by calculating adjusted Wald statistics (Kenward & Rogers, 1997).”
MethodStatistical analysis of two arm randomized pre-post designs with one post-treatment measurement
Wan, F. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01323-9 · Cited by 82
Using cluster-robust standard errors when analyzing group-randomized trials with few clusters
Huang, F. L., & Li, X. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01627-0 · Cited by 91
An adjusted coefficient of determination (R2) for generalized linear mixed models in one go
Piepho, H. (2023) · Biometrical Journal · doi:10.1002/bimj.202200290 · Cited by 85
Watt, L. J., Clark, C., Krebs, G. L., Petzel, C. E., Nielsen, S. G., & Utsumi, S. A. (2015) · Journal of Dairy Science · doi:10.3168/jds.2015-9463 · Cited by 66
Fluctuating Asymmetry and Environmental Stress: Understanding the Role of Trait History
Coster, G. D., Dongen, S. V., Malaki, P., Muchai, M., Alcántara-Exposito, A., Matheve, H., & Lens, L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057966 · Cited by 74
“Degrees of freedom were estimated following Kenward and Roger [49] and a restricted maximum likelihood (REML) procedure was applied in SAS 9.”
MethodInfluentialKause, A., Tobin, D., Dobly, A., Houlihan, D. F., Martin, S., Mäntysaari, E. A., Ritola, O., & Ruohonen, K. (2006) · Genetics Selection Evolution · doi:10.1186/1297-9686-38-4-389 · Cited by 63
“The method of Kenward and Roger [22] was used to calculate correct F-tests and their degrees of freedom for the fixed effects.”
MethodInfluentialNutrient Utilization of Differing Forage-to-Concentrate Ratios by Growing Holstein Heifers
Moody, M. L., Zanton, G. I., Daubert, J., & Heinrichs, A. J. (2007) · Journal of Dairy Science · doi:10.3168/jds.2007-0570 · Cited by 66
High-field proton magnetic resonance spectroscopy reveals metabolic effects of normal brain aging
Harris, J. L., Yeh, H., Swerdlow, R. H., Choi, I.-Y., Lee, P., & Brooks, W. M. (2014) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2014.01.018 · Cited by 70
“Considering the small sample size, the degrees of freedom were calculated by the Kenward-Roger method (Kenward and Roger, 1997).”
MethodGenetic and environmental effects influencing fruit colour and QTL analysis in raspberry
McCallum, S., Woodhead, M., Hackett, C. A., Kassim, A. S. M., Paterson, A., & Graham, J. (2010) · Theoretical and Applied Genetics · doi:10.1007/s00122-010-1334-5 · Cited by 75
“A marker close to each QTL and with fewest missing values was identified, and environment, marker and their interaction were fitted as fixed effects and their significance evaluated using an F statistic (Kenward and Roger 1997).”
MethodCroft, H. A., Pomara, N., Gommoll, C., Chen, D., Nunez, R., & Mathews, M. (2014) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.14m08992 · Cited by 61
Garbacz, S. A., Sheridan, S. M., Koziol, N. A., Kwon, K., & Holmes, S. R. (2015) · School Psychology Review · doi:10.17105/spr-14-0035.1 · Cited by 73
“For all analyses, standard errors and denominator degrees of freedom were computed using the correction of Kenward and Roger (1997), and variance components were estimated using residual maximum likelihood.”
MethodUsing an epiphytic moss to identify previously unknown sources of atmospheric cadmium pollution
Donovan, G. H., Jovan, S., Gatziolis, D., Burstyn, I., Michael, Y. L., Amacher, M. C., & Monleón, V. J. (2016) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2016.03.182 · Cited by 64
“Spatial dependence means that observations are not independent, so we calculated degrees of freedom using the Kenward–Rogers approximation (Kenward and Roger, 1997).”
Gonçalves, M., Gourley, K. M., Dritz, S. S., Tokach, M. D., Bello, N. M., DeRouchey, J. M., Woodworth, J. C., & Goodband, R. D. (2016) · Journal of Animal Science · doi:10.2527/jas.2015-0087 · Cited by 63
“Degrees of freedom were estimated using the Kenward–Rogers approach (Kenward and Roger, 1997).”
MethodCarrico, C., Chelette, K., Westgate, P., Powell, E. S., Nichols, L., Fleischer, A., & Sawaki, L. (2016) · Stroke · doi:10.1161/strokeaha.116.012671 · Cited by 59
Katzmarzyk, P. T., Broyles, S. T., Champagne, C. M., Chaput, J., Fogelholm, M., Hu, G., Kuriyan, R., Kurpad, A. V., Lambert, E. V., Maia, J. A. R., et al. (2016) · Nutrients · doi:10.3390/nu8120770 · Cited by 80
“The denominator degrees of freedom for statistical tests pertaining to fixed effects were calculated using the Kenward and Roger approximation [30].”
MethodMoore, M. P., Lis, C., Gherghel, I., & Martin, R. A. (2019) · Ecology Letters · doi:10.1111/ele.13200 · Cited by 61
Hoppe, J. E., Kasi, A. S., Pittman, J. E., Jensen, R., Thia, L. P., Robinson, P. J., Tirakitsoontorn, P., Ramsey, B. W., Mall, M. A., Taylor‐Cousar, J. L., et al. (2025) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(24)00407-7 · Cited by 83
Contribution of arbuscular mycorrhiza to soil quality in contrasting cropping systems
Kahiluoto, H., Ketoja, E., & Vestberg, M. V. (2009) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2009.05.016 · Cited by 76
“The significances of the fixed effects were determined by an F-test, and the method described by Kenward and Roger (1997) was used to calculate degrees of freedom.”
MethodCritchfield, J. W., Young, D. H., Hayes, T. L., Braun, J. V., Garcia, J. C., Pollard, R. B., & Shacklett, B. L. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003577 · Cited by 67
“The model was estimated using restricted maximum likelihood, and degrees of freedom were calculated using the Kenward and Roger adjustment [46].”
MethodExternal influences on children’s self-served portions at meals
Fisher, J. O., Birch, L. L., Zhang, J., Grusak, M. A., & Hughes, S. O. (2013) · International Journal of Obesity · doi:10.1038/ijo.2012.216 · Cited by 75
Does exercise improve self-reported sleep quality in non-remitted major depressive disorder?
Rethorst, C. D., Sunderajan, P., Greer, T. L., Grannemann, B. D., Nakonezny, P. A., Carmody, T. J., & Trivedi, M. H. (2013) · Psychological Medicine · doi:10.1017/s0033291712001675 · Cited by 62
“Restricted maximum likelihood estimation and Type 3 tests of fixed effects were used, with the Kenward–Roger correction (Kenward & Roger, 1997) applied to the first-order autoregressive covariance structure.”
MethodFarokhnia, M., Schwandt, M. L., Lee, M. R., Bollinger, J. W., Farinelli, L. A., Amodio, J. P., Sewell, L., Lionetti, T. A., Spero, D. E., & Leggio, L. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.71 · Cited by 62
Marshall, W. L., McCrea, J. B., Macha, S., Menzel, K., Liu, F., van Schanke, A., de Haes, J. I. U., Hussaini, A., Jordan, H. R., Drexel, M., et al. (2018) · The Journal of Clinical Pharmacology · doi:10.1002/jcph.1094 · Cited by 65
Metadata
- Authors
- Kenward, M. G., & Roger, J. H.
- Affiliations
- University of Kent (GB)
- Venue
- Biometrics
- Publisher
- JSTOR
- Published
- 1997-09
- Volume
- 53
- Issue
- 3
- Pages
- 983
- ISSN
- 0006-341X (ISSN-L)
- OpenAlex
- W1981747359
- Cited by
- 4,552
- References
- 23
- Citations by year
- 2026: 217 · 2025: 196 · 2024: 216 · 2023: 213 · 2022: 261 · 2021: 299 · 2020: 287 · 2019: 270 · 2018: 247 · 2017: 240 · 2016: 286 · 2015: 256 · 2014: 217 · 2013: 189 · 2012: 218
- FWCI
- 7.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrics (publishedVersion)
- PubMed (publishedVersion)
- Rothamsted Repository (Rothamsted Repository) (submittedVersion)
- Kent Academic Repository (University of Kent) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Mathematics › Applied Mathematics › Statistical and numerical algorithms
- Keywords
- restricted maximum likelihood, small sample inference, fixed effects, Wald statistic, covariance matrix estimation, small sample bias, Hotelling's T2 statistic, linear mixed models, linear Gaussian models
- MeSH terms
- Analysis of Variance, Biometry, Clinical Trials as Topic, Humans, Intermittent Claudication, Nitrogen Fixation, Plants, Probability, Regression Analysis, Research Design, Rhizobium, Soil Microbiology, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Cross-Over Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.