CoChat

It’s Time to Rehabilitate the P-Value

Weinberg, Clarice R. · 2001 · Epidemiology
Cited by 66

The DOI matches its Crossref record.

Abstract

Any biostatistician worth her salt has had extensive training in hypothesis testing. Those of us who then find ourselves working in epidemiology have been frustrated when we try to use our tools. The P-value has become politically incorrect, at least among a circle of vocal and influential epidemiologists. This journal has taken a leading role in discouraging the use of P- values and hypothesis tests, in favor of confidence intervals and estimation. 1 This view has produced a near-complete prohibition, and the few of us intrepid diehards who have secured the former editor’s permission to publish an actual P- value in these pages wear that achievement as a badge of courage. The P-value quantifies the discrepancy between a given data set and the null hypothesis, as the probability of results being as discrepant or more so, given the null hypothesis. It thus assesses the degree of inconsistency between the stated null and the observed data. A fuller description of this simple idea can be found in many introductory textbooks. 2 How has the P-value come to be so despised? There are many reasons. Certainly, less abstract ways to summarize study results are appealing. I like to see papers that stay close to the data by showing graphs and point estimates where possible. Moreover, the point estimate for a risk ratio can speak more directly to the possible public health implications than can a P-value, which inherently confounds sample size and magnitude of effect. Furthermore, the P-value is easy to misinterpret. (No, it is not the probability that a null hypothesis is true, nor is it the probability that we are making a certain error.) Mindless efforts to dichotomize results into significant vs nonsignificant, based on some cutpoint, such as 0.05, get us into trouble: results were significant for males but not for females, and therefore? One of the most pernicious abuses of automated decision making occurs when clinical treatments are asserted to be equivalent, based on a nonsignificant P-value for the observed difference. 3 Finally, we have all read literature reviews where silly claims are made that the research on a given question has been “inconsistent,” because some were “positive” (statistically significant) and others “negative” (not statistically significant) - when in fact there is no inconsistency among the findings. Such abuses remain painfully common in epidemiology. On the other hand, the propensity to misuse or misunderstand a tool should not necessarily lead us to prohibit its use. The theory of estimation is also often misunderstood. How many epidemiologists can explain the meaning of their 95% confidence interval? There are other simple concepts susceptible to fuzzy thinking. I once quizzed a class of epidemiology students and discovered that most had only a foggy notion of what is meant by the word “bias.” Should we then abandon all discussion of bias, and dumb down the field to the point where no subtleties need trouble us? There is no need to choose one set of techniques to the dogmatic exclusion of another. The challenge is to have a healthy respect for the uses and limitations of the available methods. Sometimes estimation alone is insufficient and hypothesis testing should play an important supplemental role. Here are some examples. Consider a case-control study in which an exposure is divided according to quintiles for analysis. This creates a four-dimensional contrast parameter in which the four estimates are not statistically independent. The appropriate confidence region is not four separate confidence intervals (one per odds ratio), but a curved region in four-dimensional space. Showing such results graphically is problematic. But the separate evaluation of the marginal confidence interval for each odds ratio is not a satisfactory alternative. My first question in evaluating the effect of a multi-level factor will always be: Is there overall evidence for an influence of this exposure on this outcome? We could go right to estimation, and there are good methods for estimating an exposure-response curve, eg, using splines or generalized additive models (GAMs). However, empirical estimates, whether based on category-based odds ratios or spline fits or GAM approaches, will rarely give a flat estimated exposure-response, even when the truth is flat. I want to avoid being deceived by the wiggles in the estimated exposure-response, wiggles that might only reflect the random noise present in all finite data sets. Is there much evidence for a difference in risk across the five exposure categories, or across the range of measured exposures? If a test suggests there is, I can allow myself to become fascinated by the wiggles in the estimated exposure-response. If not, then I should wonder if trying to find inferential meaning in the curve, however interesting, is more akin to reading tea leaves than science. Another context where the estimate is naturally multivariate is when assessing possible heter

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1097/00001648-200105000-00004

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cited by

See all 66

Metadata

Authors
Weinberg, C. R.
Venue
Epidemiology
Published
2001-05
Volume
12
Issue
3
Pages
288-290
ISSN
1044-3983 (ISSN-L)
OpenAlex
W1964897942
Cited by
66
References
0
FWCI
4.09
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: It’s Time to Rehabilitate the P-Value. https://app.cochat.ai/papers/10.1097/00001648-200105000-00004
The paper
Weinberg, C. R. (2001). It’s Time to Rehabilitate the P-Value. Epidemiology. https://doi.org/10.1097/00001648-200105000-00004

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.