It’s Time to Rehabilitate the P-Value
The DOI matches its Crossref record.
Abstract
Any biostatistician worth her salt has had extensive training in hypothesis testing. Those of us who then find ourselves working in epidemiology have been frustrated when we try to use our tools. The P-value has become politically incorrect, at least among a circle of vocal and influential epidemiologists. This journal has taken a leading role in discouraging the use of P- values and hypothesis tests, in favor of confidence intervals and estimation. 1 This view has produced a near-complete prohibition, and the few of us intrepid diehards who have secured the former editor’s permission to publish an actual P- value in these pages wear that achievement as a badge of courage. The P-value quantifies the discrepancy between a given data set and the null hypothesis, as the probability of results being as discrepant or more so, given the null hypothesis. It thus assesses the degree of inconsistency between the stated null and the observed data. A fuller description of this simple idea can be found in many introductory textbooks. 2 How has the P-value come to be so despised? There are many reasons. Certainly, less abstract ways to summarize study results are appealing. I like to see papers that stay close to the data by showing graphs and point estimates where possible. Moreover, the point estimate for a risk ratio can speak more directly to the possible public health implications than can a P-value, which inherently confounds sample size and magnitude of effect. Furthermore, the P-value is easy to misinterpret. (No, it is not the probability that a null hypothesis is true, nor is it the probability that we are making a certain error.) Mindless efforts to dichotomize results into significant vs nonsignificant, based on some cutpoint, such as 0.05, get us into trouble: results were significant for males but not for females, and therefore? One of the most pernicious abuses of automated decision making occurs when clinical treatments are asserted to be equivalent, based on a nonsignificant P-value for the observed difference. 3 Finally, we have all read literature reviews where silly claims are made that the research on a given question has been “inconsistent,” because some were “positive” (statistically significant) and others “negative” (not statistically significant) - when in fact there is no inconsistency among the findings. Such abuses remain painfully common in epidemiology. On the other hand, the propensity to misuse or misunderstand a tool should not necessarily lead us to prohibit its use. The theory of estimation is also often misunderstood. How many epidemiologists can explain the meaning of their 95% confidence interval? There are other simple concepts susceptible to fuzzy thinking. I once quizzed a class of epidemiology students and discovered that most had only a foggy notion of what is meant by the word “bias.” Should we then abandon all discussion of bias, and dumb down the field to the point where no subtleties need trouble us? There is no need to choose one set of techniques to the dogmatic exclusion of another. The challenge is to have a healthy respect for the uses and limitations of the available methods. Sometimes estimation alone is insufficient and hypothesis testing should play an important supplemental role. Here are some examples. Consider a case-control study in which an exposure is divided according to quintiles for analysis. This creates a four-dimensional contrast parameter in which the four estimates are not statistically independent. The appropriate confidence region is not four separate confidence intervals (one per odds ratio), but a curved region in four-dimensional space. Showing such results graphically is problematic. But the separate evaluation of the marginal confidence interval for each odds ratio is not a satisfactory alternative. My first question in evaluating the effect of a multi-level factor will always be: Is there overall evidence for an influence of this exposure on this outcome? We could go right to estimation, and there are good methods for estimating an exposure-response curve, eg, using splines or generalized additive models (GAMs). However, empirical estimates, whether based on category-based odds ratios or spline fits or GAM approaches, will rarely give a flat estimated exposure-response, even when the truth is flat. I want to avoid being deceived by the wiggles in the estimated exposure-response, wiggles that might only reflect the random noise present in all finite data sets. Is there much evidence for a difference in risk across the five exposure categories, or across the range of measured exposures? If a test suggests there is, I can allow myself to become fascinated by the wiggles in the estimated exposure-response. If not, then I should wonder if trying to find inferential meaning in the curve, however interesting, is more akin to reading tea leaves than science. Another context where the estimate is naturally multivariate is when assessing possible heter
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Wacholder, S., Chanock, S., Garcia-Closas, M., El ghormli, L., & Rothman, N. (2004) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djh075 · Cited by 1,754
Issues in the reporting of epidemiological studies: a survey of recent practice.
Pocock, S. J., Collier, T. J., Dandreo, K. J., de Stavola, B. L., Goldman, M. B., Kalish, L. A., Kasten, L. E., & McCormack, V. A. (2004) · BMJ · doi:10.1136/bmj.38250.571088.55 · Cited by 313
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
A method to automate probabilistic sensitivity analyses of misclassified binary variables
Fox, M. P., Lash, T. L., & Greenland, S. (2005) · International Journal of Epidemiology · doi:10.1093/ije/dyi184 · Cited by 288
Metadata
- Authors
- Weinberg, C. R.
- Venue
- Epidemiology
- Published
- 2001-05
- Volume
- 12
- Issue
- 3
- Pages
- 288-290
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W1964897942
- Cited by
- 66
- References
- 0
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: It’s Time to Rehabilitate the P-Value. https://app.cochat.ai/papers/10.1097/00001648-200105000-00004
- The paper
- Weinberg, C. R. (2001). It’s Time to Rehabilitate the P-Value. Epidemiology. https://doi.org/10.1097/00001648-200105000-00004
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.