Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

On judging the significance of differences by examining the overlap between confidence intervals.

Schenker, Nathaniel, Gentleman, Jane F·2001·The American Statistician

Abstract

To judge whether the difference between two point estimates is statistically significant, data analysts often examine the overlap between the two associated confidence intervals. We compare this technique to the standard method of testing significance under the common assumptions of consistency, asymptotic normality, and asymptotic independence of the estimates. Rejection of the null hypothesis by the method of examining overlap implies rejection by the standard method, whereas failure to reject by the method of examining overlap does not imply failure to reject by the standard method. As a consequence, the method of examining overlap is more conservative (i.e., rejects the null hypothesis less often) than the standard method when the null hypothesis is true, and it mistakenly fails to reject the null hypothesis more frequently than does the standard method when the null hypothesis is false. Although the method of examining overlap is simple and especially convenient when lists or graphs of confidence intervals have been presented, we conclude that it should not be used for formal significance testing unless the data analyst is aware of its deficiencies and unless the information needed to carry out a more appropriate procedure is unavailable.

Overview

Integrity signals

Registry

Crossref · 10.1198/000313001317097960

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 33 references

Research works printed but not linked: 2

  1. Eising E. G., 1998, Journal of Nuclear Medicine, 39, 1062
  2. Martinez M. E., 1999, Cancer Research, 59, 5181

Cited by

See all 1,218

Metadata

Authors
Schenker, N., & Gentleman, J. F.
Affiliations
National Center for Health Statistics (US)
Venue
The American Statistician
Published
2001-08
Volume
55
Issue
3
Pages
182-186
ISSN
  • 0003-1305 (ISSN-L)
  • 1537-2731
OpenAlex
W2036002592
Cited by
1,218
References
33
Citations by year
2026: 57 · 2025: 63 · 2024: 85 · 2023: 77 · 2022: 105 · 2021: 75 · 2020: 63 · 2019: 61 · 2018: 64 · 2017: 64 · 2016: 53 · 2015: 50 · 2014: 63 · 2013: 51 · 2012: 39
FWCI
19.30
Type
journal article
Language
en
Open access
closed
Locations
The American Statistician (publishedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
Keywords
confidence intervals, significance testing, null hypothesis rejection, conservative tests, asymptotic normality, asymptotic independence, consistency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: On judging the significance of differences by examining the overlap between confidence intervals. https://app.cochat.ai/papers/10.1198/000313001317097960
The paper
Schenker, N., & Gentleman, J. F. (2001). On judging the significance of differences by examining the overlap between confidence intervals. The American Statistician. https://doi.org/10.1198/000313001317097960

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.