The DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Abstract
Little is known about researchers' understanding of confidence intervals (CIs) and standard error (SE) bars. Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited to visit a Web site where they adjusted a figure until they judged 2 means, with error bars, to be just statistically significantly different (p < .05). Results from 473 respondents suggest that many leading researchers have severe misconceptions about how error bars relate to statistical significance, do not adequately distinguish CIs and SE bars, and do not appreciate the importance of whether the 2 means are independent or come from a repeated measures design. Better guidelines for researchers and less ambiguous graphical conventions are needed before the advantages of CIs for research communication can be realized.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Through estimating statistically significant differences and 95% CIs among studies97, in comparison to interventions for stress, our findings suggest that breathwork might be associated with similar—and non-significantly different—effects.”
From Effect of breathwork on stress and mental health: A meta-analysis of randomised-controlled trials (2023) · cites this paper as [97]
In the citing paper
“However, CIs are not immune to incorrect interpretations either (Belia et al., 2005; Hoekstra et al., 2014; Miller and Ulrich, 2016).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Belia et al., 2005
Citations
Cites
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Things I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449 · cited 20×
“In fact, when sample sizes are similar and not small, and CI widths are similar, then if 95% CIs on independent means just touch, the two-tail p value is about .006 ( Cumming & Finch, 2005; Payton, Greenstone, & Schenker, 2003), not .05 as many believe.”
BackgroundMethodInfluentialSelf-citationStatistics with Confidence: Confidence Intervals and Statistical Guidelines.
JOLLIFFE, F. R., Gardner, M. J., & Altman, D. G. (1990) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982808 · Cited by 1,319
On judging the significance of differences by examining the overlap between confidence intervals.
Schenker, N., & Gentleman, J. F. (2001) · The American Statistician · doi:10.1198/000313001317097960 · Cited by 1,218
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,044
Beyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034
Payton, M. E., Greenstone, M. H., & Nathaniel Schenker, A. (2003) · Journal of Insect Science · doi:10.1093/jis/3.1.34 · Cited by 858
Research works printed but not linked: 3
- 2001, Publication manual of the American Psychological Association
- 10.1177/00131640121971374 · DOI · Paper page
- Smithson, 2002, Confidence intervals.
Cited by
Sorted by citations
Items 101–106 of 106
Misconceptions of the p-value among Chilean and Italian Academic Psychologists
Badenes-Ribera, L., Frias-Navarro, D., Iotti, B., Bonilla-Campos, A., & Longobardi, C. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01247 · Cited by 35
“However, CIs are not immune to incorrect interpretations either (Belia et al., 2005; Hoekstra et al., 2014; Miller and Ulrich, 2016).”
BackgroundZhang, S., Heck, P. R., Meyer, M. N., Chabris, C. F., Goldstein, D. G., & Hofman, J. M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2302491120 · Cited by 30
Experimentally disambiguating models of sensory cue integration
Scarfe, P. (2022) · Journal of Vision · doi:10.1167/jov.22.1.5 · Cited by 17
Xu, Z., Song, T., & Lee, Y.-C. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103455 · Cited by 18
Visualizing Uncertainty to Promote Clinicians’ Understanding of Measurement Error
Frans, N., Hummelen, B., Albers, C. J., & Paap, M. C. S. (2023) · Assessment · doi:10.1177/10731911221147042 · Cited by 5
“Several studies show that such abstract metrics can be difficult to interpret (Charter & Feldt, 2002; Hildon et al., 2012; McManus, 2012; Plebani et al., 2018; Simpkin & Armstrong, 2019), even for people who have received rigorous training (Belia et al., 2005; Kalinowski et al., 2018).”
BackgroundHeltne, A., Frans, N., Hummelen, B., Falkum, E., Germans Selvik, S., & Paap, M. C. S. (2023) · Scandinavian Journal of Psychology · doi:10.1111/sjop.12918 · Cited by 5
Metadata
- Authors
- Belia, S., Fidler, F., Williams, J., & Cumming, G.
- Affiliations
- La Trobe University (AU)
- Venue
- Psychological Methods
- Published
- 2005
- Volume
- 10
- Issue
- 4
- Pages
- 389-396
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2110405340
- Cited by
- 390
- References
- 27
- Citations by year
- 2026: 15 · 2025: 20 · 2024: 27 · 2023: 30 · 2022: 35 · 2021: 23 · 2020: 21 · 2019: 24 · 2018: 43 · 2017: 19 · 2016: 18 · 2015: 15 · 2014: 21 · 2013: 12 · 2012: 11
- FWCI
- 8.98
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- confidence intervals, error bars, statistical significance, repeated measures designs, scholarly communication
- MeSH terms
- Data Interpretation, Statistical, Humans, Research, Confidence Intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
