The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Inference by eye: Reading the overlap of independent confidence intervals
Abstract
When 95 per cent confidence intervals (CIs) on independent means do not overlap, the two‐tailed p‐value is less than 0.05 and there is a statistically significant difference between the means. However, p for non‐overlapping 95 per cent CIs is actually considerably smaller than 0.05: If the two CIs just touch, p is about 0.01, and the intervals can overlap by as much as about half the length of one CI arm before p becomes as large as 0.05. Keeping in mind this rule—that overlap of half the length of one arm corresponds approximately to statistical significance at p = 0.05—can be helpful for a quick appreciation of figures that display CIs, especially if precise p‐values are not reported. The author investigated the robustness of this and similar rules, and found them sufficiently accurate when sample sizes are at least 10, and the two intervals do not differ in width by more than a factor of 2. The author reviewed previous discussions of CI overlap and extended the investigation to p‐values other than 0.05 and 0.01. He also studied 95 per cent CIs on two proportions, and on two Pearson correlations, and found similar rules apply to overlap of these asymmetric CIs, for a very broad range of cases. Wider use of figures with 95 per cent CIs is desirable, and these rules may assist easy and appropriate understanding of such figures. Copyright © 2008 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Methods for Psychology.
Gudgeon, A. C., & Howell, D. C. (1994) · The Statistician · doi:10.2307/2348956 · Cited by 8,278
Two-sided confidence intervals for the single proportion: comparison of seven methods
Newcombe, R. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8<857::aid-sim777>3.0.co;2-e · Cited by 5,107
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Interval estimation for the difference between independent proportions: comparison of eleven methods
Newcombe, R. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8<873::aid-sim779>3.0.co;2-i · Cited by 1,851
THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN TWO MEANS WHEN THE POPULATION VARIANCES ARE UNEQUAL
WELCH, B. L. (1938) · Biometrika · doi:10.1093/biomet/29.3-4.350 · Cited by 1,640
Publication Manual of the American Psychological Association: BOOK REVIEW
Yao, R. (2011) · Family and Consumer Sciences Research Journal · doi:10.1111/j.1552-3934.2011.02081.x · Cited by 1,451
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449 · cited 12×
“An informal adjustment towards conservatism, to protect against inflated Type 1 error rates [ 5 ], may be to use p 5 ] recommended a range of ways to think about CIs without invoking p values.”
BackgroundMethodInfluentialSelf-citationOn judging the significance of differences by examining the overlap between confidence intervals.
Schenker, N., & Gentleman, J. F. (2001) · The American Statistician · doi:10.1198/000313001317097960 · Cited by 1,218
Payton, M. E., Greenstone, M. H., & Schenker, N. (2003) · Journal of Insect Science · doi:10.1093/jis/3.1.34 · Cited by 856
Payton, M. E., Greenstone, M. H., & Schenker, N. (2003) · Journal of Insect Science · doi:10.1673/031.003.3401 · Cited by 699
Jmp Start Statistics: A Guide to Statistical and Data Analysis Using Jmp and Jmp in Software
Sall, J., Creighton, L., & Lehman, A. (1999) · Medical Entomology and Zoology · Cited by 644
The graphical presentation of a collection of means
Goldstein, H., & Healy, M. J. R. (1995) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983411 · Cited by 639
(2006) · Educational Researcher · doi:10.3102/0013189x035006033 · Cited by 543
Cumming, G. (2008) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2008.00079.x · Cited by 451
Self-citationComparing median lethal concentration values using confidence interval overlap or ratio tests
Wheeler, M. W., Park, R. M., & Bailer, A. J. (2006) · Environmental Toxicology and Chemistry · doi:10.1897/05-320r.1 · Cited by 409
Research Methods in Psychology
Basavanna, M. (2015) · Psychology for Nurses · doi:10.5005/jp/books/12408_3 · Cited by 399
Tryon, W. W. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.371 · Cited by 391
Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.389 · Cited by 390
Self-citationA brief note on overlapping confidence intervals
Austin, P. C., & Hux, J. E. (2002) · Journal of Vascular Surgery · doi:10.1067/mva.2002.125015 · Cited by 386
Uniform Requirements for Manuscripts Submitted to Biomedical Journals
(1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-1-199701010-00006 · Cited by 247
Julious, S. A. (2004) · Pharmaceutical Statistics · doi:10.1002/pst.126 · Cited by 225
Statistics with Confidence. Confidence Intervals and Statistical Guidelines
Newcombe, R. (1990) · Journal of Medical Genetics · doi:10.1136/jmg.27.1.71 · Cited by 181
Bulpitt, C. J. (1987) · The Lancet · doi:10.1016/s0140-6736(87)92100-3 · Cited by 162
Schneider, B. (2006) · Cited by 149
Testing statistical hypotheses using standard error bars and confidence intervals
Payton, M. E., Miller, A. E., & Raun, W. R. (2000) · Communications in Soil Science and Plant Analysis · doi:10.1080/00103620009370458 · Cited by 143
If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2.
Wolfe, R. S., & Hanley, J. A. (2002) · PubMed · Cited by 123
Statistical Reform in Psychology: Is Anything Changing?
Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., Lo, J., McMenamin, N., & Wilson, S. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01881.x · Cited by 122
Self-citationStatistical reform in medicine, psychology and ecology
Fidler, F. M., Cumming, G., Burgman, M. A., & Thomason, N. R. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.035 · Cited by 95
Self-citationHate Speech and Academic Freedom in the Academy
Tierney, W. G. (2006) · Educational Researcher · doi:10.3102/0013189x035003033 · Cited by 88
Understanding and evaluating clinical trials
Bigby, M. E., & Gadenne, A. S. (1996) · Journal of the American Academy of Dermatology · doi:10.1016/s0190-9622(96)80053-3 · Cited by 71
Basic statistics and the inconsistency of multiple comparison procedures.
Saville, D. J. (2003) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0087423 · Cited by 65
Evaluating Overlapping Confidence Intervals
Nelson, L. S. (1989) · Journal of Quality Technology · doi:10.1080/00224065.1989.11979156 · Cited by 39
Using Confidence Intervals to Test Hypotheses
Barr, D. R. (1969) · Journal of Quality Technology · doi:10.1080/00224065.1969.11980385 · Cited by 27
Overlapping confidence intervals
Cole, S. R., & Blair, R. C. (1999) · Journal of the American Academy of Dermatology · doi:10.1016/s0190-9622(99)70281-1 · Cited by 20
Ryan, G. W., & Leadbetter, S. (2002) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1036109580 · Cited by 17
Understanding and evaluating clinical trials
Rahlfs, V. W. (1997) · Journal of the American Academy of Dermatology · doi:10.1016/s0190-9622(97)70126-9 · Cited by 7
Bigby, M. E., & Gadenne, A. S. (1997) · Journal of the American Academy of Dermatology · doi:10.1016/s0190-9622(97)70127-0 · Cited by 3
Cited by
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Self-citationTrust and team performance: A meta-analysis of main effects, moderators, and covariates.
De Jong, B. A., Dirks, K. T., & Gillespie, N. (2016) · Journal of Applied Psychology · doi:10.1037/apl0000110 · Cited by 616
The frequencies and disparities of adverse childhood experiences in the U.S.
Giano, Z., Wheeler, D. L., & Hubach, R. D. (2020) · BMC Public Health · doi:10.1186/s12889-020-09411-z · Cited by 463
Plant-soil feedback and the maintenance of diversity in Mediterranean-climate shrublands
Teste, F. P., Kardol, P., Turner, B. L., Wardle, D. A., Zemunik, G., Renton, M., & Laliberté, E. (2017) · Science · doi:10.1126/science.aai8291 · Cited by 424
Impacts of the COVID-19 pandemic on rural America
Mueller, J. T., McConnell, K., Burow, P. B., Pofahl, K., Merdjanoff, A. A., & Farrell, J. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2019378118 · Cited by 391
Metadata
- Authors
- Cumming, G.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2008-11-07
- Volume
- 28
- Issue
- 2
- Pages
- 205-220
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W1966165134
- Cited by
- 670
- References
- 37
- FWCI
- 7.76
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.