The DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Abstract
Little is known about researchers' understanding of confidence intervals (CIs) and standard error (SE) bars. Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited to visit a Web site where they adjusted a figure until they judged 2 means, with error bars, to be just statistically significantly different (p < .05). Results from 473 respondents suggest that many leading researchers have severe misconceptions about how error bars relate to statistical significance, do not adequately distinguish CIs and SE bars, and do not appreciate the importance of whether the 2 means are independent or come from a repeated measures design. Better guidelines for researchers and less ambiguous graphical conventions are needed before the advantages of CIs for research communication can be realized.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Through estimating statistically significant differences and 95% CIs among studies97, in comparison to interventions for stress, our findings suggest that breathwork might be associated with similar—and non-significantly different—effects.”
From Effect of breathwork on stress and mental health: A meta-analysis of randomised-controlled trials (2023) · cites this paper as [97]
In the citing paper
“However, CIs are not immune to incorrect interpretations either (Belia et al., 2005; Hoekstra et al., 2014; Miller and Ulrich, 2016).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Belia et al., 2005
Citations
Cites
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Things I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449 · cited 20×
“In fact, when sample sizes are similar and not small, and CI widths are similar, then if 95% CIs on independent means just touch, the two-tail p value is about .006 ( Cumming & Finch, 2005; Payton, Greenstone, & Schenker, 2003), not .05 as many believe.”
BackgroundMethodInfluentialSelf-citationStatistics with Confidence: Confidence Intervals and Statistical Guidelines.
JOLLIFFE, F. R., Gardner, M. J., & Altman, D. G. (1990) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982808 · Cited by 1,319
On judging the significance of differences by examining the overlap between confidence intervals.
Schenker, N., & Gentleman, J. F. (2001) · The American Statistician · doi:10.1198/000313001317097960 · Cited by 1,218
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,044
Beyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034
Payton, M. E., Greenstone, M. H., & Nathaniel Schenker, A. (2003) · Journal of Insect Science · doi:10.1093/jis/3.1.34 · Cited by 858
Research works printed but not linked: 3
- 2001, Publication manual of the American Psychological Association
- 10.1177/00131640121971374 · DOI · Paper page
- Smithson, 2002, Confidence intervals.
Cited by
Sorted by citations
D'Ignazio, C., & Klein, L. F. (2020) · doi:10.7551/mitpress/11805.001.0001 · Cited by 1,738
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449
Self-citationVariables associated with achievement in higher education: A systematic review of meta-analyses.
Schneider, M., & Preckel, F. (2017) · Psychological Bulletin · doi:10.1037/bul0000098 · Cited by 1,347
Best Practices for Estimating, Interpreting, and Presenting Nonlinear Interaction Effects
Mize, T. D. (2019) · Sociological Science · doi:10.15195/v6.a4 · Cited by 1,019
Error bars in experimental biology
Cumming, G., Fidler, F., & Vaux, D. L. (2007) · The Journal of Cell Biology · doi:10.1083/jcb.200611141 · Cited by 940
Self-citation
Metadata
- Authors
- Belia, S., Fidler, F., Williams, J., & Cumming, G.
- Affiliations
- La Trobe University (AU)
- Venue
- Psychological Methods
- Published
- 2005
- Volume
- 10
- Issue
- 4
- Pages
- 389-396
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2110405340
- Cited by
- 390
- References
- 27
- Citations by year
- 2026: 15 · 2025: 20 · 2024: 27 · 2023: 30 · 2022: 35 · 2021: 23 · 2020: 21 · 2019: 24 · 2018: 43 · 2017: 19 · 2016: 18 · 2015: 15 · 2014: 21 · 2013: 12 · 2012: 11
- FWCI
- 8.98
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- confidence intervals, error bars, statistical significance, repeated measures designs, scholarly communication
- MeSH terms
- Data Interpretation, Statistical, Humans, Research, Confidence Intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
