The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Abstract
Wider use in psychology of confidence intervals (CIs), especially as error bars in figures, is a desirable development. However, psychologists seldom use CIs and may not understand them well. The authors discuss the interpretation of figures with error bars and analyze the relationship between CIs and statistical significance testing. They propose 7 rules of eye to guide the inferential use of figures with error bars. These include general principles: Seek bars that relate directly to effects of interest, be sensitive to experimental design, and interpret the intervals. They also include guidelines for inferential interpretation of the overlap of CIs on independent group means. Wider use of interval estimation in psychology has the potential to improve research communication substantially.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Mcgill, R., Tukey, J. W., & Larsen, W. A. (1978) · The American Statistician · doi:10.1080/00031305.1978.10479236 · Cited by 2,618
Using confidence intervals in within-subject designs
Loftus, G. R., & Masson, M. E. J. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03210951 · Cited by 2,433
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
Edgington, E. S., Edgington, E., & Onghena, P. (1964) · Springer eBooks · doi:10.1002/0471667196.ess2169 · Cited by 2,052
Confidence intervals rather than P values: estimation rather than hypothesis testing.
Gardner, M. J., & Altman, D. G. (1986) · BMJ · doi:10.1136/bmj.292.6522.746 · Cited by 2,014
Smithson, M. J. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778 · cited 1×
“Distinguish practical or clinical significance from statistical significance (Kendall, 1999; Kirk, 1996).”
BackgroundStevens' Handbook of Experimental Psychology
Pashler, H. (2002) · Wiley eBooks · doi:10.1002/0471214426 · Cited by 1,713
THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN TWO MEANS WHEN THE POPULATION VARIANCES ARE UNEQUAL
Welch, B. L. (1938) · Biometrika · doi:10.1093/biomet/29.3-4.350 · Cited by 1,506
Cleveland, W. S. (1985) · Medical Entomology and Zoology · Cited by 1,425
Statistics with Confidence: Confidence Intervals and Statistical Guidelines.
JOLLIFFE, F. R., Gardner, M. J., & Altman, D. G. (1990) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982808 · Cited by 1,319
On judging the significance of differences by examining the overlap between confidence intervals.
Schenker, N., & Gentleman, J. F. (2001) · The American Statistician · doi:10.1198/000313001317097960 · Cited by 1,218
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,044
Beyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034
Using confidence intervals for graphically based data interpretation.
Masson, M. E., & Loftus, G. (2003) · Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale · doi:10.1037/h0087426 · Cited by 819
What If There Were No Significance Tests?
(2013) · Psychology Press eBooks · doi:10.4324/9781315827353 · Cited by 675
McGill, R., Tukey, J. W., & Larsen, W. A. (1978) · The American Statistician · doi:10.2307/2683468 · Cited by 666
Steiger, J. H. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.2.164 · Cited by 592
Thompson, B. (2002) · Educational Researcher · doi:10.3102/0013189x031003025 · Cited by 546
What If There Were No Significance Tests?
(2016) · Routledge eBooks · doi:10.4324/9781315629049 · Cited by 543
Cumming, G., & Finch, S. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401614002 · Cited by 528
Self-citationNormative comparisons for the evaluation of clinical significance.
Kendall, P. C., Marrs, A. L., Nath, S. R., & Sheldrick, R. C. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.67.3.285 · Cited by 468
Tryon, W. (2001) · Psychological methods · doi:10.1037/1082-989x.6.4.371 · Cited by 423
Applying Contemporary Statistical Techniques
(2003) · Elsevier eBooks · doi:10.1016/b978-0-12-751541-0.x5021-4 · Cited by 421
Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.389 · Cited by 390
Self-citationStatistical Inference: A Commentary for the Social and Behavioural Sciences
Oakes, M. W. (1986) · Medical Entomology and Zoology · Cited by 388
The Significance of the Difference Between Two Means when the Population Variances are Unequal
Welch, B. L. (1938) · Biometrika · doi:10.2307/2332010 · Cited by 374
(1995) · Choice Reviews Online · doi:10.5860/choice.32-6272 · Cited by 310
How many discoveries have been lost by ignoring modern statistical methods?
Wilcox, R. R. (1998) · American Psychologist · doi:10.1037/0003-066x.53.3.300 · Cited by 295
The Hospital Admission Decision for Patients With Community-Acquired Pneumonia
Fine, M. J. (1997) · Archives of Internal Medicine · doi:10.1001/archinte.1997.00440220040006 · Cited by 291
Smithson, M. J. (2001) · Educational and Psychological Measurement · doi:10.1177/00131640121971392 · Cited by 275
Edgington, E. S. (1998) · Technometrics · doi:10.1080/00401706.1998.10485213 · Cited by 245
Applying Contemporary Statistical Techniques
Nembhard, H. B. (2004) · Technometrics · doi:10.1198/tech.2004.s233 · Cited by 187
Statistics with Confidence. Confidence Intervals and Statistical Guidelines
Newcombe, R. G. (1990) · Journal of Medical Genetics · doi:10.1136/jmg.27.1.71 · Cited by 181
Replication and Researchers' Understanding of Confidence Intervals and Standard Error Bars
Cumming, G., Williams, J., & Fidler, F. M. (2004) · Understanding Statistics · doi:10.1207/s15328031us0304_5 · Cited by 175
Self-citationThe Null Hypothesis Testing Controversy in Psychology
Krantz, D. H. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473888 · Cited by 158
If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2.
Wolfe, R. S., & Hanley, J. (2002) · PubMed · Cited by 122
Computing Correct Confidence Intervals for ANOVA Fixed- and Random-Effects Effect Sizes
Fidler, F. M., & Thompson, B. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401614003 · Cited by 107
Fidler, F. M. (2002) · Educational and Psychological Measurement · doi:10.1177/001316402236876 · Cited by 106
The earth is round (p < .05): Rejoinder.
Cohen, J. (1995) · American Psychologist · doi:10.1037/0003-066x.50.12.1103 · Cited by 100
Kendall, P. C. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.3.283 · Cited by 87
Confidence Intervals for Effect Sizes in Analysis of Variance
Bird, K. (2002) · Educational and Psychological Measurement · doi:10.1177/0013164402062002001 · Cited by 86
On the communication of information by displays of standard errors and confidence intervals
Estes, W. K. (1997) · Psychonomic Bulletin & Review · doi:10.3758/bf03210790 · Cited by 69
Basic statistics and the inconsistency of multiple comparison procedures.
Saville, D. J. (2003) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0087423 · Cited by 65
Analysis, Interpretation, and Visual Presentation of Experimental Data
Loftus, G. R. (2002) · doi:10.1002/0471214426.pas0409 · Cited by 56
Fine, M. J. (1997) · Archives of Internal Medicine · doi:10.1001/archinte.157.1.36 · Cited by 52
Past and Future American Psychological Association Guidelines for Statistical Practice
Finch, S. J., Thomason, N. R., & Cumming, G. (2002) · Theory & Psychology · doi:10.1177/0959354302126005 · Cited by 40
Self-citationReform of statistical inference in psychology: The case ofMemory & Cognition
Finch, S. J., Cumming, G., Williams, J., Palmer, L., Griffith, E., Alders, C., Anderson, J. L., & Goodman, O. (2004) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195577 · Cited by 39
Self-citationDisplaying your findings : a practical guide for creating figures, posters, and presentations
Nicol, A. A. M., & Pexman, P. M. (2003) · Cited by 35
The Null Hypothesis Testing Controversy in Psychology
Krantz, D. H. (1999) · Journal of the American Statistical Association · doi:10.2307/2669949 · Cited by 34
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (2016) · doi:10.4324/9781315629049-20 · Cited by 22
Vaux, D. L. (2004) · Nature · doi:10.1038/428799c · Cited by 16
Cumming, D. (2004) · doi:10.1049/ic:20040588 · Cited by 1
Research works printed but not linked: 6
- 2001, Publication manual of the American Psychological Association
- 10.1177/00131640121971374 · DOI · Paper page
- 10.1177/00131640121971383 · DOI
- Light, 1994, The visual presentation and interpretation of meta-analysis., 439
- Smithson, 2002, Confidence intervals.
- Steiger, 1997, Noncentrality interval estimation and the evaluation of statistical models., 221
Cited by
Sorted by citations
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303
“For excellent examples, see ESCI (Cumming and Finch, 2005), confidence interval software by Smithson (2001), and G ∗ Power (Faul et al., 2009).”
BackgroundMethodInfluentialDurlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011) · Child Development · doi:10.1111/j.1467-8624.2010.01564.x · Cited by 10,854
“Multiple papers evaluating the same intervention but containing different outcome data at post or follow-up for the same cohort were combined into a single study.”
BackgroundMethodseaborn: statistical data visualization
Waskom, M. (2021) · Journal of Open Source Software · doi:10.21105/joss.03021 · Cited by 7,310
“The confidence intervals are computed with a bootstrap algorithm, allowing them to generalize over many different statistics, and the default level allows the user to perform “inference by eye” (Cumming & Finch, 2005).”
MethodThe New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
“The best way to think of SE bars on a mean is usually to double the whole length—so the bars extend 2 SE above and 2 SE below the mean—and interpret them as being, approximately, the 95% CI, as Figure 2 illustrates (Cumming & Finch, 2005).”
Self-citationEllis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Metadata
- Authors
- Cumming, G., & Finch, S.
- Affiliations
- La Trobe University (AU)
- Venue
- American Psychologist
- Published
- 2005
- Volume
- 60
- Issue
- 2
- Pages
- 170-180
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2005534705
- Cited by
- 1,449
- References
- 59
- Citations by year
- 2026: 62 · 2025: 66 · 2024: 74 · 2023: 92 · 2022: 102 · 2021: 110 · 2020: 92 · 2019: 86 · 2018: 80 · 2017: 82 · 2016: 76 · 2015: 89 · 2014: 63 · 2013: 59 · 2012: 48
- FWCI
- 39.36
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Psychologist (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- confidence intervals, error bars, statistical significance testing, interval estimation, scholarly communication
- MeSH terms
- Humans, Confidence Intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
