The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Low P-Values or Narrow Confidence Intervals: Which Are More Durable?
Abstract
What should be the role of P-values and confidence intervals in the interpretation of scientific results? This question is not new 1 and our field of epidemiology is far from alone in struggling with it. 2,3 I have four suggestions for authors and readers. The first is quite broad, so I offer that one before describing current practices. I then turn to the other three. My remarks are confined to settings in which P-values and confidence intervals accompany estimates of effect measures, such as the relative risk. Briefly, here are my suggestions. One, we should work harder than ever to avoid strict or exact interpretations of P-values and confidence intervals in observational research, where these statistics lack a theoretical basis. Two, we should stop interpreting P-values and confidence intervals as though they measure the probability of hypotheses. Three, when we want to know the probability of hypotheses, we should use Bayesian methods, which are designed expressly for that purpose. Four, we should get serious about precision and look for narrow confidence intervals instead of low P-values to identify results that are least influenced by random error. Real Life Is Not Randomized When treatment or exposure is randomized, we have a solid theoretical basis, testable in simulations, for the probability models from which P-values, confidence intervals, and likelihoods are deduced. In observational research, all we can do is hope that the social, behavioral, and physical processes by which people become exposed to risk factors in the unrandomized real world do not differ too greatly from randomization. 4 Unfortunately, each time we find that risk factors are associated with each other in observational studies, we find evidence against that hope. We cannot remind ourselves too often of this fundamental problem. At the very least, it should cause us to avoid hairsplitting interpretations of probabilistic statistics in observational research, where they are intrinsically fuzzy. Contemporary Uses of P-Values and Confidence Intervals Significance testing unquestionably dominates epidemiology today. In attempting to refrain from this practice over the past 17 years, 5 I have often been expected, assumed, encouraged, and sometimes even forced to engage in it by editors, reviewers, colleagues, professors, students, funding sources, regulators, attorneys, and journalists. It is not easy to be a non-tester in a testing world. After Rothman’s highly influential 1978 essay, “A Show of Confidence,”6 an immense and easily documented shift in reporting style took place. 7 Whereas P-values or “S” (significant) and “NS” (not significant) once were reported exclusively, the reporting of confidence intervals has now become accepted practice, with or without P- value accompaniment. Confidence intervals have a survival advantage for the tiny non-testing minority to which I belong. They enable us to gauge the precision of estimates easily, but without depriving the established majority of its beloved tests. Epidemiologists who see no purpose to a confidence interval other than its use in significance testing sometimes wonder why this shift in reporting practice has occurred. The P-value provides the information they desire more efficiently and exactly. Some are vaguely aware that confidence intervals supposedly convey information that P-values do not, but are unsure what that extra information is and even less sure how it might be useful. The word “precision” seems to be used with increasing regularity nowadays, and confidence intervals are occasionally described as “wide,” but “wide” and “imprecise” often seem nothing more than code words for “includes the null value” and hence for “not statistically significant.” Improbable Observations Do Not Imply Improbable Hypotheses When we estimate a parameter such as the relative risk, each possible value of that parameter is the expected value under some hypothesis, and each hypothesis has a P-value. 8,9 What we call “the”P-value is the P-value for the null hypothesis. Approximately, each P-value is the probability of obtaining an estimate at least as far from a specified value as the estimate we have obtained, if that specified value were the true value. It follows that no P-value, for the null hypothesis or any other, is the probability that the specified hypothesis is true. As an obvious example, the hypothesis corresponding to the point estimate has a (two-sided) P-value of 1.0. However, we do not treat our point estimates as absolutely certain to be true. Neither is the point estimate, in general, the most probable value. For a given estimate, the 95% confidence interval is the set of all parameter values for which P ≥ 0.05. For the value at each limit of a 95% confidence interval, P = 0.05 (two-sided). Thus, if either of the 95% confidence limits for a relative risk estimate equals 1.0 (the null value of this parameter), we can infer that the null P-value is 0.05. From this link be
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
IX. On the problem of the most efficient tests of statistical hypotheses
Neyman, J., & Pearson, E. S. (1933) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1933.0009 · Cited by 2,341
Null Hypothesis Testing: Problems, Prevalence, and an Alternative
Anderson, D. R., Burnham, K. P., & Thompson, W. L. (2000) · The Journal of Wildlife Management · doi:10.2307/3803199 · Cited by 1,930
ON THE PROBLEM OF THE MOST EFFICIENT TESTS OF STATISTICAL HYPOTHESES
(1967) · Joint Statistical Papers · doi:10.1525/9780520339897-008 · Cited by 1,504
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,185
ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE
NEYMAN, J., & PEARSON, E. S. (1928) · Biometrika · doi:10.1093/biomet/20a.3-4.263 · Cited by 1,070
Toward Evidence-Based Medical Statistics. 2: The Bayes Factor
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00019 · Cited by 912
Calibration of ρ Values for Testing Precise Null Hypotheses
Sellke, T., Bayarri, M. J., & Berger, J. O. (2001) · The American Statistician · doi:10.1198/000313001300339950 · Cited by 854
Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence
Berger, J. O., & Sellke, T. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478397 · Cited by 775
Cited by
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Self-citationA Randomized Trial Comparing Radical Prostatectomy with Watchful Waiting in Early Prostate Cancer
Holmberg, L., Bill-Axelson, A., Helgesen, F., Salo, J. O., Folmerz, P., Häggman, M., Andersson, S.-O., Spångberg, A., Busch, C., Nordling, S., et al. (2002) · New England Journal of Medicine · doi:10.1056/nejmoa012794 · Cited by 817
Good practices for quantitative bias analysis
Lash, T. L., Fox, M. P., MacLehose, R. F., Maldonado, G., McCandless, L. C., & Greenland, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu149 · Cited by 646
Impact of Early-Life Bisphenol A Exposure on Behavior and Executive Function in Children
Braun, J. M., Kalkbrenner, A. E., Calafat, A. M., Yolton, K., Ye, X., Dietrich, K. N., & Lanphear, B. P. (2011) · Pediatrics · doi:10.1542/peds.2011-1335 · Cited by 561
Metadata
- Authors
- Poole, C.
- Venue
- Epidemiology
- Published
- 2001-05
- Volume
- 12
- Issue
- 3
- Pages
- 291-294
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2151129876
- Cited by
- 350
- References
- 27
- FWCI
- 12.69
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Low P-Values or Narrow Confidence Intervals: Which Are More Durable? https://app.cochat.ai/papers/10.1097/00001648-200105000-00005
- The paper
- Poole, C. (2001). Low P-Values or Narrow Confidence Intervals: Which Are More Durable? Epidemiology. https://doi.org/10.1097/00001648-200105000-00005
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.