Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 215

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Commentary: P Values and Statistical Practice

Gelman, Andrew · 2013 · Epidemiology

Abstract

Sander Greenland and Charles Poole1 accept that P values are here to stay but recognize that some of their most common interpretations have problems. The casual view of the P value as posterior probability of the truth of the null hypothesis is false and not even close to valid under any reasonable model, yet this misunderstanding persists even in high-stakes settings (as discussed, for example, by Greenland in 2011).2 The formal view of the P value as a probability conditional on the null is mathematically correct but typically irrelevant to research goals (hence, the popularity of alternative—if wrong—interpretations). A Bayesian interpretation based on a spike-and-slab model makes little sense in applied contexts in epidemiology, political science, and other fields in which true effects are typically nonzero and bounded (thus violating both the “spike” and the “slab” parts of the model). I find Greenland and Poole’s1 perspective to be valuable: it is important to go beyond criticism and to understand what information is actually contained in a P value. These authors discuss some connections between P values and Bayesian posterior probabilities. I am not so optimistic about the practical value of these connections. Conditional on the continuing omnipresence of P values in applications, however, these are important results that should be generally understood. Greenland and Poole1 make two points. First, they describe how P values approximate posterior probabilities under prior distributions that contain little information relative to the data: This misuse [of P values] may be lessened by recognizing correct Bayesian interpretations. For example, under weak priors, 95% confidence intervals approximate 95% posterior probability intervals, one-sided P values approximate directional posterior probabilities, and point estimates approximate posterior medians. I used to think this way, too (see many examples in our books), but in recent years have moved to the position that I do not trust such direct posterior probabilities. Unfortunately, I think we cannot avoid informative priors if we wish to make reasonable unconditional probability statements. To put it another way, I agree with the mathematical truth of the quotation above, but I think it can mislead in practice because of serious problems with apparently noninformative or weak priors. Second, the main proposal made by Greenland and Poole is to interpret P values as bounds on posterior probabilities: [U]nder certain conditions, a one-sided P value for a prior median provides an approximate lower bound on the posterior probability that the point estimate is on the wrong side of that median. This is fine, but when sample sizes are moderate or small (as is common in epidemiology and social science), posterior probabilities will depend strongly on the prior distribution. Although I do not see much direct value in a lower bound, I am intrigued by Greenland and Poole’s1 point that “if one uses an informative prior to derive the posterior probability of the point estimate being in the wrong direction, P0/2 provides a reference point indicating how much the prior information influenced that posterior probability.” This connection could be useful to researchers working in an environment in which P values are central to communication of statistical results. In presenting my view of the limitations of Greenland and Poole’s1 points, I am leaning heavily on their own work, in particular on their emphasis that, in real problems, prior information is always available and is often strong enough to have an appreciable impact on inferences. Before explaining my position, I will briefly summarize how I view classical P values and my experiences. For more background, I recommend the discussion by Krantz3 of null hypothesis testing in psychology research. WHAT IS A P VALUE IN PRACTICE? The P value is a measure of discrepancy of the fit of a model or “null hypothesis” H to data y. Mathematically, it is defined as Pr(T(yrep)>T(y)|H), where yrep represents a hypothetical replication under the null hypothesis and T is a test statistic (ie, a summary of the data, perhaps tailored to be sensitive to departures of interest from the model). In a model with free parameters (a “composite null hypothesis”), the P value can depend on these parameters, and there are various ways to get around this, by plugging in point estimates, averaging over a posterior distribution, or adjusting for the estimation process. I do not go into these complexities further, bringing them up here only to make the point that the construction of P values is not always a simple or direct process. (Even something as simple as the classical chi-square test has complexities to be discovered; see the article by Perkins et al4). In theory, the P value is a continuous measure of evidence, but in practice it is typically trichotomized approximately into strong evidence, weak evidence, and no evidence (these can also be l

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1097/ede.0b013e31827886f7

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 6

  1. Greenland, 2013, Living with P-values: resurrecting a Bayesian perspective on frequentist statistics., Epidemiology, 24, 62, 10.1097/EDE.0b013e3182785741 · DOI
  2. Greenland, 2011, Null misinterpretation in statistical testing and its impact on health risk assessment., Prev Med, 53, 225, 10.1016/j.ypmed.2011.08.010 · DOI
  3. Krantz, 1999, The null hypothesis testing controversy in psychology., J Am Stat Assoc, 94, 1372, 10.1080/01621459.1999.10473888 · DOI
  4. Perkins, 2011, Computing the confidence levels for a root-mean- square test of goodness-of-fit., Appl Math Comput, 217, 9072, 10.1016/j.amc.2011.03.124 · DOI
  5. Gelman, 2009, Of beauty, sex, and power: statistical challenges in estimating small effects., Am Sci, 97, 310, 10.1511/2009.79.310 · DOI
  6. Kanazawa, 2007, Beautiful parents have more daughters: a further implication of the generalized Trivers-Willard hypothesis (gTWH)., J Theor Biol, 244, 133, 10.1016/j.jtbi.2006.07.017 · DOI

Cited by

See all 215

Metadata

Authors
Gelman, A.
Venue
Epidemiology
Published
2013-01
Volume
24
Issue
1
Pages
69-72
ISSN
1044-3983 (ISSN-L)
OpenAlex
W2049642512
Cited by
215
References
2
FWCI
8.99
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Commentary: P Values and Statistical Practice. https://app.cochat.ai/papers/10.1097/ede.0b013e31827886f7
The paper
Gelman, A. (2013). Commentary: P Values and Statistical Practice. Epidemiology. https://doi.org/10.1097/ede.0b013e31827886f7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.