The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
In defense of P values
Abstract
Statistical hypothesis testing has been widely criticized by ecologists in recent years. I review some of the more persistent criticisms of P values and argue that most stem from misunderstandings or incorrect interpretations, rather than from intrinsic shortcomings of the P value. I show that P values are intimately linked to confidence intervals and to differences in Akaike's information criterion (ΔAIC), two metrics that have been advocated as replacements for the P value. The choice of a threshold value of ΔAIC that breaks ties among competing models is as arbitrary as the choice of the probability of a Type I error in hypothesis testing, and several other criticisms of the P value apply equally to ΔAIC. Since P values, confidence intervals, and ΔAIC are based on the same statistical information, all have their places in modern statistical practice. The choice of which to use should be stylistic, dictated by details of the application rather than by dogmatic, a priori considerations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Pseudoreplication and the Design of Ecological Field Experiments
Hurlbert, S. H. (1984) · Ecological Monographs · doi:10.2307/1942661 · Cited by 8,377 · cited 1×
“In my opinion, it would benefit the science if more time and effort were spent on designing effective studies with adequate replication (Hurlbert 1984), and less on advocacy for particular tools to be used in summarizing the data.”
BackgroundBurnham, K. P., Anderson, D. R., & Huyvaert, K. P. (2011) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-010-1029-6 · Cited by 4,104
Effect size, confidence interval and statistical significance: a practical guide for biologists
Nakagawa, S., & Cuthill, I. C. (2007) · Biological Reviews · doi:10.1111/j.1469-185x.2007.00027.x · Cited by 3,835
Null Hypothesis Testing: Problems, Prevalence, and an Alternative
Anderson, D. R., Burnham, K. P., & Thompson, W. L. (2000) · The Journal of Wildlife Management · doi:10.2307/3803199 · Cited by 1,930
Akaike Information Criterion Statistics.
Grove, D., Sakamoto, Y., Ishiguro, M., & Kitagawa, G. (1988) · The Statistician · doi:10.2307/2348776 · Cited by 1,184
The statistical sleuth : a course in methods of data analysis
Ramsey, F. L., & Schafer, D. W. (2002) · Cited by 1,166
The Insignificance of Statistical Significance Testing
Johnson, D. H. (1999) · The Journal of Wildlife Management · doi:10.2307/3802789 · Cited by 1,104
Statistical Evidence: A Likelihood Paradigm
Reeves, J. H., & Royall, R. M. (1998) · Journal of the American Statistical Association · doi:10.2307/2669869 · Cited by 814
Cited by
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
A brief introduction to mixed effects modelling and multi-model inference in ecology
Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018) · PeerJ · doi:10.7717/peerj.4794 · Cited by 2,094
Model selection for ecologists: the worldviews of AIC and BIC
Aho, K., Derryberry, D., & Peterson, T. (2014) · Ecology · doi:10.1890/13-1452.1 · Cited by 1,165
“Murtaugh (2014) revealed an important mathematical connection between DAIC and P values for a comparison of two models (one nested in the other).”
BackgroundThe fickle P value generates irreproducible results
Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015) · Nature Methods · doi:10.1038/nmeth.3288 · Cited by 765
Sensitivity and specificity of information criteria
(2019) · doi:10.1093/bib/bbz016 · Cited by 718
Rewriting results sections in the language of evidence
Muff, S., Nilsen, E. B., O’Hara, R. B., & Nater, C. R. (2022) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2021.10.009 · Cited by 636
Model averaging and muddled multimodel inferences
Cade, B. S. (2015) · Ecology · doi:10.1890/14-1639.1 · Cited by 622
On doing better science: From thrill of discovery to policy implications
Antonakis, J. (2017) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.01.006 · Cited by 475
Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test
Midway, S., Robertson, M., Flinn, S., & Kaller, M. (2020) · PeerJ · doi:10.7717/peerj.10387 · Cited by 430
Halsey, L. G. (2019) · Biology Letters · doi:10.1098/rsbl.2019.0174 · Cited by 357
The relative performance of AIC, AICCand BIC in the presence of unobserved heterogeneity
Brewer, M. J., Butler, A., & Cooksley, S. L. (2016) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12541 · Cited by 342
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
The human sex ratio from conception to birth
Orzack, S. H., Stubblefield, J. W., Akmaev, V. R., Colls, P., Munné, S., Scholl, T., Steinsaltz, D., & Zuckerman, J. E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1416546112 · Cited by 283
Biodiversity–multifunctionality relationships depend on identity and number of measured functions
Meyer, S. T., Ptacnik, R., Hillebrand, H., Bessler, H., Buchmann, N., Ebeling, A., Eisenhauer, N., Engels, C., Fischer, M., Halle, S., et al. (2017) · Nature Ecology & Evolution · doi:10.1038/s41559-017-0391-4 · Cited by 265
Transparency in Ecology and Evolution: Real Problems, Real Solutions
Parker, T. H., Forstmeier, W., Koricheva, J., Fidler, F., Hadfield, J. D., Chee, Y. E., Kelly, C. D., Gurevitch, J., & Nakagawa, S. (2016) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2016.07.002 · Cited by 201
Ito, H. C., Shiraishi, H., Nakagawa, M., & Takamura, N. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229052 · Cited by 42
Metadata
- Authors
- Murtaugh, P. A.
- Venue
- Ecology
- Publisher
- Wiley
- Published
- 2014-03
- Volume
- 95
- Issue
- 3
- Pages
- 611-617
- ISSN
- 0012-9658 (ISSN-L)
- 1939-9170
- OpenAlex
- W2037592609
- Cited by
- 317
- References
- 33
- FWCI
- 33.68
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.