The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
On model selection criteria in multimodel analysis
Abstract
Hydrologic systems are open and complex, rendering them prone to multiple conceptualizations and mathematical descriptions. There has been a growing tendency to postulate several alternative hydrologic models for a site and use model selection criteria to (1) rank these models, (2) eliminate some of them, and/or (3) weigh and average predictions and statistics generated by multiple models. This has led to some debate among hydrogeologists about the merits and demerits of common model selection (also known as model discrimination or information) criteria such as AIC,AICc,BIC, and KIC and some lack of clarity about the proper interpretation and mathematical representation of each criterion. We examine the model selection literature to find that (1) all published rigorous derivations of AIC and AICc require that the (true) model having generated the observational data be in the set of candidate models; (2) though BIC and KIC were originally derived by assuming that such a model is in the set,BIC has been rederived by Cavanaugh and Neath (1999) without the need for such an assumption; and (3)KIC reduces to BIC as the number of observations becomes large relative to the number of adjustable model parameters, implying that it likewise does not require the existence of a true model in the set of alternatives. We explain why KIC is the only criterion accounting validly for the likelihood of prior parameter estimates, elucidate the unique role that the Fisher information matrix plays in KIC, and demonstrate through an example that it imbues KIC with desirable model selection properties not shared by AIC,AICc, or BIC. Our example appears to provide the first comprehensive test of how AIC,AICc,BIC, and KIC weigh and rank alternative models in light of the models' predictive performance under cross validation with real hydrologic data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
Probability, Random Variables, and Stochastic Processes.
Abrahams, J., & Papoulis, A. (1984) · Journal of the American Statistical Association · doi:10.2307/2288754 · Cited by 16,226
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Multimodel Inference: Understanding AIC and BIC in Model Selection
Burnham, K. P., & Anderson, D. R. (2004) · Sociological Methods & Research · doi:10.1177/0049124104268644 · Cited by 11,648
Regression and time series model selection in small samples
HURVICH, C. M., & TSAI, C.-L. (1989) · Biometrika · doi:10.1093/biomet/76.2.297 · Cited by 6,586
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
Bozdogan, H. (1987) · Psychometrika · doi:10.1007/bf02294361 · Cited by 4,468
Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999) · Statistical Science · doi:10.1214/ss/1009212519 · Cited by 4,118
On the mathematical foundations of theoretical statistics
Fisher, R. A. (1922) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1922.0009 · Cited by 4,039
GSLIB: Geostatistical Software Library and User's Guide
Ziegel, E. R., Deutsch, C., & Journel, A. (1995) · Technometrics · doi:10.2307/1269177 · Cited by 3,097
(2016) · Time Series with Mixed Spectra · doi:10.1201/b15154-8 · Cited by 2,739
A manifesto for the equifinality thesis
Beven, K. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.07.007 · Cited by 2,581
Seber, G. A. F., & Wild, C. J. (1989) · Wiley Series in Probability and Statistics · doi:10.1002/0471725315 · Cited by 1,962
The Selection of Prior Distributions by Formal Rules
Kass, R. E., & Wasserman, L. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10477003 · Cited by 1,308
Seber, G. A. F., & Lee, A. J. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/9780471722199 · Cited by 1,135
EFRON, B., & HINKLEY, D. V. (1978) · Biometrika · doi:10.1093/biomet/65.3.457 · Cited by 952
Carrera, J., & Neuman, S. P. (1986) · Water Resources Research · doi:10.1029/wr022i002p00199 · Cited by 871
Self-citationLinhart, H., & Zucchini, W. (1986) · Cited by 869
On entropy maximization principle
Akaike, H. (1977) · Medical Entomology and Zoology · Cited by 634
A framework for dealing with uncertainty due to model structure error
Refsgaard, J. C., van der Sluijs, J. P., Brown, J., & van der Keur, P. (2006) · Advances in Water Resources · doi:10.1016/j.advwatres.2005.11.013 · Cited by 535
An Introduction to Model Selection
Zucchini, W. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1276 · Cited by 465
Maximum likelihood Bayesian averaging of uncertain model predictions
Neuman, S. P. (2003) · Stochastic Environmental Research and Risk Assessment (SERRA) · doi:10.1007/s00477-003-0151-7 · Cited by 459
Self-citationGeneralised information criteria in model selection
Konishi, S. (1996) · Biometrika · doi:10.1093/biomet/83.4.875 · Cited by 433
Information Criteria for Discriminating Among Alternative Regression Models
Sawa, T. (1978) · Econometrica · doi:10.2307/1913828 · Cited by 391
Optimal Choice of AR and MA Parts in Autoregressive Moving Average Models
Kashyap, R. L. (1982) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1982.4767213 · Cited by 329
J., J. E., & Krishnaiah, P. R. (1978) · Technometrics · doi:10.2307/1268148 · Cited by 309
Multimodel Ranking and Inference in Ground Water Modeling
Poeter, E., & Anderson, D. (2005) · Groundwater · doi:10.1111/j.1745-6584.2005.0061.x · Cited by 231
Poeter, E. P., Hill, M. C., Banta, E. R., et al. (2005) · Techniques and methods · doi:10.3133/tm6a11 · Cited by 226
Maximum likelihood Bayesian averaging of spatial variability models in unsaturated fractured tuff
Ye, M., Neuman, S. P., & Meyer, P. D. (2004) · Water Resources Research · doi:10.1029/2003wr002557 · Cited by 211
Self-citationZio, E., & Apostolakis, G. (1996) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(96)00078-6 · Cited by 199
Kass, R. E., & Vaidyanathan, S. K. (1992) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1992.tb01868.x · Cited by 168
Cooley, R. L. (1982) · Water Resources Research · doi:10.1029/wr018i004p00965 · Cited by 139
Generalizing the derivation of the schwarz information criterion
Cavanaugh, J. E., & Neath, A. A. (1999) · Communication in Statistics- Theory and Methods · doi:10.1080/03610929908832282 · Cited by 138
Ye, M., Neuman, S. P., Meyer, P. D., & Pohlmann, K. F. (2005) · Water Resources Research · doi:10.1029/2005wr004260 · Cited by 113
Self-citationInformational complexity criteria for regression models
Bozdogan, H., & Haughton, D. (1998) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(98)00025-5 · Cited by 90
Testing Alternative Ground Water Models Using Cross‐Validation and Other Methods
Foglia, L., Mehl, S. W., Hill, M. C., Perona, P., & Burlando, P. (2007) · Ground Water · doi:10.1111/j.1745-6584.2007.00341.x · Cited by 84
Regression and time series model selection using variants of the schwarz information criterion
Neath, A. A., & Cavanaugh, J. E. (1997) · Communication in Statistics- Theory and Methods · doi:10.1080/03610929708831934 · Cited by 83
Samper, J., & Neuman, S. P. (1989) · Water Resources Research · doi:10.1029/wr025i003p00351 · Cited by 74
Self-citationMMA, A Computer Code for Multi-Model Analysis
Poeter, E. P., & Hill, M. C. (2007) · Techniques and methods · doi:10.3133/tm6e3 · Cited by 68
Meyer, P. D., Ye, M., Rockhold, M., Neuman, S. P., & Cantrell, K. J. (2007) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/921264 · Cited by 57
Self-citation
Cited by
Song, X., Zhang, J., Zhan, C., Xuan, Y., Ye, M., & Xu, C. (2015) · Journal of Hydrology · doi:10.1016/j.jhydrol.2015.02.013 · Cited by 654
Self-citationTowards a comprehensive assessment of model structural adequacy
Gupta, H. V., Clark, M. P., Vrugt, J. A., Abramowitz, G., & Ye, M. (2012) · Water Resources Research · doi:10.1029/2011wr011044 · Cited by 466
Self-citationEvaluating uncertainty in integrated environmental models: A review of concepts and tools
Matott, L. S., Babendreier, J. E., & Purucker, S. T. (2009) · Water Resources Research · doi:10.1029/2008wr007301 · Cited by 286
Wöhling, T., & Vrugt, J. A. (2008) · Water Resources Research · doi:10.1029/2008wr007154 · Cited by 86
An analytical framework to assist decision makers in the use of forest ecosystem model predictions
Larocque, G. R., Bhatti, J. S., Ascough, J. C., Liu, J., Luckai, N., Mailly, D., Archambault, L., & Gordon, A. M. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.03.009 · Cited by 23
Metadata
- Authors
- Ye, M., Meyer, P. D., & Neuman, S. P.
- Venue
- Water Resources Research
- Publisher
- American Geophysical Union (AGU)
- Published
- 2008-03
- Volume
- 44
- Issue
- 3
- ISSN
- 0043-1397 (ISSN-L)
- 1944-7973
- OpenAlex
- W1932382878
- Cited by
- 262
- References
- 42
- FWCI
- 19.33
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.