The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Commentary: P Values and Statistical Practice
Abstract
Sander Greenland and Charles Poole1 accept that P values are here to stay but recognize that some of their most common interpretations have problems. The casual view of the P value as posterior probability of the truth of the null hypothesis is false and not even close to valid under any reasonable model, yet this misunderstanding persists even in high-stakes settings (as discussed, for example, by Greenland in 2011).2 The formal view of the P value as a probability conditional on the null is mathematically correct but typically irrelevant to research goals (hence, the popularity of alternative—if wrong—interpretations). A Bayesian interpretation based on a spike-and-slab model makes little sense in applied contexts in epidemiology, political science, and other fields in which true effects are typically nonzero and bounded (thus violating both the “spike” and the “slab” parts of the model). I find Greenland and Poole’s1 perspective to be valuable: it is important to go beyond criticism and to understand what information is actually contained in a P value. These authors discuss some connections between P values and Bayesian posterior probabilities. I am not so optimistic about the practical value of these connections. Conditional on the continuing omnipresence of P values in applications, however, these are important results that should be generally understood. Greenland and Poole1 make two points. First, they describe how P values approximate posterior probabilities under prior distributions that contain little information relative to the data: This misuse [of P values] may be lessened by recognizing correct Bayesian interpretations. For example, under weak priors, 95% confidence intervals approximate 95% posterior probability intervals, one-sided P values approximate directional posterior probabilities, and point estimates approximate posterior medians. I used to think this way, too (see many examples in our books), but in recent years have moved to the position that I do not trust such direct posterior probabilities. Unfortunately, I think we cannot avoid informative priors if we wish to make reasonable unconditional probability statements. To put it another way, I agree with the mathematical truth of the quotation above, but I think it can mislead in practice because of serious problems with apparently noninformative or weak priors. Second, the main proposal made by Greenland and Poole is to interpret P values as bounds on posterior probabilities: [U]nder certain conditions, a one-sided P value for a prior median provides an approximate lower bound on the posterior probability that the point estimate is on the wrong side of that median. This is fine, but when sample sizes are moderate or small (as is common in epidemiology and social science), posterior probabilities will depend strongly on the prior distribution. Although I do not see much direct value in a lower bound, I am intrigued by Greenland and Poole’s1 point that “if one uses an informative prior to derive the posterior probability of the point estimate being in the wrong direction, P0/2 provides a reference point indicating how much the prior information influenced that posterior probability.” This connection could be useful to researchers working in an environment in which P values are central to communication of statistical results. In presenting my view of the limitations of Greenland and Poole’s1 points, I am leaning heavily on their own work, in particular on their emphasis that, in real problems, prior information is always available and is often strong enough to have an appreciable impact on inferences. Before explaining my position, I will briefly summarize how I view classical P values and my experiences. For more background, I recommend the discussion by Krantz3 of null hypothesis testing in psychology research. WHAT IS A P VALUE IN PRACTICE? The P value is a measure of discrepancy of the fit of a model or “null hypothesis” H to data y. Mathematically, it is defined as Pr(T(yrep)>T(y)|H), where yrep represents a hypothetical replication under the null hypothesis and T is a test statistic (ie, a summary of the data, perhaps tailored to be sensitive to departures of interest from the model). In a model with free parameters (a “composite null hypothesis”), the P value can depend on these parameters, and there are various ways to get around this, by plugging in point estimates, averaging over a posterior distribution, or adjusting for the estimation process. I do not go into these complexities further, bringing them up here only to make the point that the construction of P values is not always a simple or direct process. (Even something as simple as the classical chi-square test has complexities to be discovered; see the article by Perkins et al4). In theory, the P value is a continuous measure of evidence, but in practice it is typically trichotomized approximately into strong evidence, weak evidence, and no evidence (these can also be l
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893
Research works printed but not linked: 6
- Greenland, 2013, Living with P-values: resurrecting a Bayesian perspective on frequentist statistics., Epidemiology, 24, 62, 10.1097/EDE.0b013e3182785741 · DOI
- Greenland, 2011, Null misinterpretation in statistical testing and its impact on health risk assessment., Prev Med, 53, 225, 10.1016/j.ypmed.2011.08.010 · DOI
- Krantz, 1999, The null hypothesis testing controversy in psychology., J Am Stat Assoc, 94, 1372, 10.1080/01621459.1999.10473888 · DOI
- Perkins, 2011, Computing the confidence levels for a root-mean- square test of goodness-of-fit., Appl Math Comput, 217, 9072, 10.1016/j.amc.2011.03.124 · DOI
- Gelman, 2009, Of beauty, sex, and power: statistical challenges in estimating small effects., Am Sci, 97, 310, 10.1511/2009.79.310 · DOI
- Kanazawa, 2007, Beautiful parents have more daughters: a further implication of the generalized Trivers-Willard hypothesis (gTWH)., J Theor Biol, 244, 133, 10.1016/j.jtbi.2006.07.017 · DOI
Cited by
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors
Gelman, A., & Carlin, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551642 · Cited by 1,343
Self-citationPrins, N., & Kingdom, F. A. A. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01250 · Cited by 429
Evolution of Reporting P Values in the Biomedical Literature, 1990-2015
Chavalarias, D., Wallach, J. D., Li, A. H. T., & Ioannidis, J. P. A. (2016) · JAMA · doi:10.1001/jama.2016.1952 · Cited by 370
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376
Metadata
- Authors
- Gelman, A.
- Venue
- Epidemiology
- Published
- 2013-01
- Volume
- 24
- Issue
- 1
- Pages
- 69-72
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2049642512
- Cited by
- 215
- References
- 2
- FWCI
- 8.99
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Commentary: P Values and Statistical Practice. https://app.cochat.ai/papers/10.1097/ede.0b013e31827886f7
- The paper
- Gelman, A. (2013). Commentary: P Values and Statistical Practice. Epidemiology. https://doi.org/10.1097/ede.0b013e31827886f7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.