The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Model averaging and muddled multimodel inferences
Abstract
Three flawed practices associated with model averaging coefficients for predictor variables in regression models commonly occur when making multimodel inferences in analyses of ecological data. Model‐averaged regression coefficients based on Akaike information criterion (AIC) weights have been recommended for addressing model uncertainty but they are not valid, interpretable estimates of partial effects for individual predictors when there is multicollinearity among the predictor variables. Multicollinearity implies that the scaling of units in the denominators of the regression coefficients may change across models such that neither the parameters nor their estimates have common scales, therefore averaging them makes no sense. The associated sums of AIC model weights recommended to assess relative importance of individual predictors are really a measure of relative importance of models, with little information about contributions by individual predictors compared to other measures of relative importance based on effects size or variance reduction. Sometimes the model‐averaged regression coefficients for predictor variables are incorrectly used to make model‐averaged predictions of the response variable when the models are not linear in the parameters. I demonstrate the issues with the first two practices using the college grade point average example extensively analyzed by Burnham and Anderson. I show how partial standard deviations of the predictor variables can be used to detect changing scales of their estimates with multicollinearity. Standardizing estimates based on partial standard deviations for their variables can be used to make the scaling of the estimates commensurate across models, a necessary but not sufficient condition for model averaging of the estimates to be sensible. A unimodal distribution of estimates and valid interpretation of individual parameters are additional requisite conditions. The standardized estimates or equivalently the t statistics on unstandardized estimates also can be used to provide more informative measures of relative importance than sums of AIC weights. Finally, I illustrate how seriously compromised statistical interpretations and predictions can be for all three of these flawed practices by critiquing their use in a recent species distribution modeling technique developed for predicting Greater Sage‐Grouse ( Centrocercus urophasianus ) distribution in Colorado, USA. These model averaging issues are common in other ecological literature and ought to be discontinued if we are to make effective scientific contributions to ecological knowledge and conservation of natural resources.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
Multimodel Inference: Understanding AIC and BIC in Model Selection
Burnham, K. P., & Anderson, D. R. (2004) · Sociological Methods & Research · doi:10.1177/0049124104268644 · Cited by 11,648 · cited 3×
“…parameters in model gj, Ii (gj) is an indicator function taking the value 1 if predictor Xi is in model gj (0 otherwise), and wþ(i ) is the sum of AIC weights for all models in the set where the parameter for ith predictor variable is estimated (Burnham and Anderson 2002, 2004, Lukacs et al. 2010).”
BackgroundMethodInfluentialDormann, C. F., Elith, J., Bacher, S., Buchmann, C., Carl, G., Carré, G., Marquéz, J. R. G., Gruber, B., Lafourcade, B., Leitão, P. J., et al. (2013) · Ecography · doi:10.1111/j.1600-0587.2012.07348.x · Cited by 9,998
Harrell, F. E. (2001) · Springer Series in Statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,195
Species Distribution Models: Ecological Explanation and Prediction Across Space and Time
Elith, J., & Leathwick, J. R. (2009) · Annual Review of Ecology, Evolution, and Systematics · doi:10.1146/annurev.ecolsys.110308.120159 · Cited by 6,707
Applied Linear Statistical Models.
Lorenz, F., Neter, J., Wasserman, W., & Kutner, M. H. (1986) · Journal of the American Statistical Association · doi:10.2307/2289108 · Cited by 5,565
The Statistical Analysis of Compositional Data
Aitchison, J. (1986) · doi:10.1007/978-94-009-4109-0 · Cited by 4,383
Model Selection and Inference: A Practical Information-Theoretic Approach
Strawderman, R. L., Burnham, K. P., & Anderson, D. R. (2000) · Journal of the American Statistical Association · doi:10.2307/2669574 · Cited by 4,194
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999) · Statistical Science · doi:10.1214/ss/1009212519 · Cited by 4,118
Burnham, K. P., Anderson, D. R., & Huyvaert, K. P. (2011) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1029-6 · Cited by 4,104
Model selection in ecology and evolution
Johnson, J. B., & Omland, K. S. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2003.10.013 · Cited by 3,838
The Statistical Analysis of Compositional Data
Piepel, G. F. (1988) · Technometrics · doi:10.1080/00401706.1988.10488337 · Cited by 3,418
Burnham, K. P., & Anderson, D. R. (1998) · doi:10.1007/978-1-4757-2917-7 · Cited by 3,387
CONFRONTING MULTICOLLINEARITY IN ECOLOGICAL MULTIPLE REGRESSION
Graham, M. H. (2003) · Ecology · doi:10.1890/02-3114 · Cited by 2,536
Multimodel inference in ecology and evolution: challenges and solutions
GRUEBER, C. E., NAKAGAWA, S., LAWS, R. J., & JAMIESON, I. G. (2011) · Journal of Evolutionary Biology · doi:10.1111/j.1420-9101.2010.02210.x · Cited by 2,263
Generalized Collinearity Diagnostics
Fox, J., & Monette, G. (1992) · Journal of the American Statistical Association · doi:10.1080/01621459.1992.10475190 · Cited by 1,881
Sur les fonctions convexes et les inégalités entre les valeurs moyennes
Jensen, J. L. W. V. (1906) · Acta Mathematica · doi:10.1007/bf02418571 · Cited by 1,811
(2010) · Cited by 1,762
Model selection for ecologists: the worldviews of AIC and BIC
Aho, K., Derryberry, D., & Peterson, T. (2014) · Ecology · doi:10.1890/13-1452.1 · Cited by 1,165 · cited 1×
“The initial enthusiasm for the informationtheoretic approach as an alternative paradigm for statistical inference is now being tempered by increased recognition of its direct relationship with other likelihood based statistics (Aho et al. 2014, Murtaugh 2014a, b, Spanos 2014).”
BackgroundChevan, A., & Sutherland, M. (1991) · The American Statistician · doi:10.1080/00031305.1991.10475776 · Cited by 1,163
Chamberlin, T. C. (1965) · Science · doi:10.1126/science.148.3671.754 · Cited by 1,069
Mac Nally, R. (2000) · Biodiversity & Conservation · doi:10.1023/a:1008985925162 · Cited by 938
Partial Time Regressions as Compared with Individual Trends
Frisch, R., & Waugh, F. V. (1933) · Econometrica · doi:10.2307/1907330 · Cited by 629
Regression, Prediction and Shrinkage
Copas, J. B. (1983) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1983.tb01258.x · Cited by 612
Estimators of Relative Importance in Linear Regression Based on Variance Decomposition
Grömping, U. (2007) · The American Statistician · doi:10.1198/000313007x188252 · Cited by 612
How to Standardize Regression Coefficients
Bring, J. (1994) · The American Statistician · doi:10.1080/00031305.1994.10476059 · Cited by 519
Seasonal Adjustment of Economic Time Series and Multiple Regression Analysis
Lovell, M. C. (1963) · Journal of the American Statistical Association · doi:10.1080/01621459.1963.10480682 · Cited by 443
Species distribution models in conservation biogeography: developments and challenges
Franklin, J. (2013) · Diversity and Distributions · doi:10.1111/ddi.12125 · Cited by 410
Alternatives To Statistical Hypothesis Testing In Ecology: A Guide To Self Teaching
Hobbs, N. T., & Hilborn, R. (2006) · Ecological Applications · doi:10.1890/04-0645 · Cited by 384
Comparison of model building and selection strategies
Doherty, P. F., White, G. C., & Burnham, K. P. (2012) · Journal of Ornithology · doi:10.1007/s10336-010-0598-5 · Cited by 379
Compositional Data Analysis: Where Are We and Where Should We Be Heading?
Aitchison, J., & J. Egozcue, J. (2005) · Mathematical Geology · doi:10.1007/s11004-005-7383-7 · Cited by 379
The Method of Multiple Working Hypotheses
Chamberlin, T. C. (1998) · Maritime Archaeology · doi:10.1007/978-1-4899-0084-5_20 · Cited by 377
Freckleton, R. P. (2011) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1045-6 · Cited by 355
Model selection bias and Freedman’s paradox
Lukacs, P. M., Burnham, K. P., & Anderson, D. R. (2010) · Annals of the Institute of Statistical Mathematics · doi:10.1007/s10463-009-0234-4 · Cited by 321
Murtaugh, P. A. (2014) · Ecology · doi:10.1890/13-0590.1 · Cited by 317
Methods to quantify variable importance: implications for the analysis of noisy ecological data
Murray, K., & Conner, M. M. (2009) · Ecology · doi:10.1890/07-1929.1 · Cited by 275
Linear regression with compositional explanatory variables
Hron, K., Filzmoser, P., & Thompson, K. (2012) · Journal of Applied Statistics · doi:10.1080/02664763.2011.644268 · Cited by 185
Galipaud, M., Gillingham, M. A. F., David, M., & Dechaume‐Moncharmont, F. (2014) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12251 · Cited by 181
P values are only an index to evidence: 20th‐ vs. 21st‐century statistical science
Burnham, K. P., & Anderson, D. R. (2014) · Ecology · doi:10.1890/13-1066.1 · Cited by 165
Blums, P., Nichols, J. D., Hines, J. E., Lindberg, M. S., & Mednis, A. (2005) · Oecologia · doi:10.1007/s00442-004-1794-x · Cited by 162
Concepts of Relative Importance in Recent Scientific Literature
Kruskal, W., & Majors, R. (1989) · The American Statistician · doi:10.1080/00031305.1989.10475596 · Cited by 115
Giudice, J. H., Fieberg, J., & Lenarz, M. S. (2012) · Journal of Wildlife Management · doi:10.1002/jwmg.213 · Cited by 73
Recurring controversies aboutPvalues and confidence intervals revisited
Spanos, A. (2014) · Ecology · doi:10.1890/13-1291.1 · Cited by 53
A note on model uncertainty in linear regression
Candolo, C., Davison, A. C., & Demétrio, C. G. B. (2003) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00349 · Cited by 31
Murtaugh, P. A. (2014) · Ecology · doi:10.1890/13-1858.1 · Cited by 10
Cited by
A brief introduction to mixed effects modelling and multi-model inference in ecology
Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018) · PeerJ · doi:10.7717/peerj.4794 · Cited by 2,094
Zhao, K., Wulder, M. A., Hu, T., Bright, R., Wu, Q., Qin, H., Li, Y., Toman, E., Mallick, B., Zhang, X., et al. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.04.034 · Cited by 514
Cross validation for model selection: A review with examples from ecology
Yates, L. A., Aandahl, Z., Richards, S. A., & Brook, B. W. (2023) · Ecological Monographs · doi:10.1002/ecm.1557 · Cited by 459
Doses of Neighborhood Nature: The Benefits for Mental Health of Living with Nature
Cox, D. T. C., Shanahan, D. F., Hudson, H. L., Plummer, K. E., Siriwardena, G. M., Fuller, R. A., Anderson, K., Hancock, S., & Gaston, K. J. (2017) · BioScience · doi:10.1093/biosci/biw173 · Cited by 401
Dormann, C. F., Calabrese, J. M., Guillera‐Arroita, G., Matechou, E., Bahn, V., Bartoń, K., Beale, C. M., Ciuti, S., Elith, J., Gerstner, K., et al. (2018) · Ecological Monographs · doi:10.1002/ecm.1309 · Cited by 400
Metadata
- Authors
- Cade, B. S.
- Venue
- Ecology
- Publisher
- Wiley
- Published
- 2015-09
- Volume
- 96
- Issue
- 9
- Pages
- 2370-2382
- ISSN
- 0012-9658 (ISSN-L)
- 1939-9170
- OpenAlex
- W1981295458
- Cited by
- 622
- References
- 46
- FWCI
- 60.44
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.