Cited by 1,449Statistical Methods and Bayesian InferenceStatistics Education and MethodologiesMeta-analysis and systematic reviews

The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.

Inference by Eye: Confidence Intervals and How to Read Pictures of Data.

Cumming, Geoff, Finch, Sue·2005·American Psychologist

DOI 10.1037/0003-066x.60.2.170

Open this paper in CoChat — free account

Abstract

Wider use in psychology of confidence intervals (CIs), especially as error bars in figures, is a desirable development. However, psychologists seldom use CIs and may not understand them well. The authors discuss the interpretation of figures with error bars and analyze the relationship between CIs and statistical significance testing. They propose 7 rules of eye to guide the inferential use of figures with error bars. These include general principles: Seek bars that relate directly to effects of interest, be sensitive to experimental design, and interpret the intervals. They also include guidelines for inferential interpretation of the overlap of CIs on independent group means. Wider use of interval estimation in psychology has the potential to improve research communication substantially.

Integrity signals

Registry

Crossref · 10.1037/0003-066x.60.2.170

Cites

60 of 60 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
The earth is round (p < .05).
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Methods of Meta-Analysis
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Variations of Box Plots
Mcgill, R., Tukey, J. W., & Larsen, W. A. (1978) · The American Statistician · doi:10.1080/00031305.1978.10479236 · Cited by 2,618
Using confidence intervals in within-subject designs
Loftus, G. R., & Masson, M. E. J. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03210951 · Cited by 2,433
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
Randomization Tests
Edgington, E. S., Edgington, E., & Onghena, P. (1964) · Springer eBooks · doi:10.1002/0471667196.ess2169 · Cited by 2,052

Show all 59 references

Research works printed but not linked: 6

2001, Publication manual of the American Psychological Association
10.1177/00131640121971374 · DOI · Paper page
10.1177/00131640121971383 · DOI
Light, 1994, The visual presentation and interpretation of meta-analysis., 439
Smithson, 2002, Confidence intervals.
Steiger, 1997, Noncentrality interval estimation and the evaluation of statistical models., 221

Sorted by citations

Items 101–200 of 1,000

Data Visualization in the Neurosciences: Overcoming the Curse of Dimensionality
Allen, E. A., Erhardt, E. B., & Calhoun, V. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.05.001 · Cited by 146
Cronbach's Alpha: Review of Limitations and Associated Recommendations
Agbo, A. A. (2010) · Journal of Psychology in Africa · doi:10.1080/14330237.2010.10820371 · Cited by 168
“Hays (1973 cited in Cumming & Finch, 2005) defined CI as “an estimated range of values with a given high probability of covering the true population value” (p.171 of Cumming & Finch, 2005).”
Background
Near-infrared spectroscopy-derived muscle oxygen saturation on a 0% to 100% scale: reliability and validity of the Moxy Monitor
Feldmann, A., Schmitz, R. W., & Erlacher, D. (2019) · Journal of Biomedical Optics · doi:10.1117/1.jbo.24.11.115001 · Cited by 159
“Equivalency testing was used as the groundwork for statistical procedure based on the confidence interval (CI) comparisons and a priori equivalency intervals (EIs) to determine statistical equivalency and statistical difference in accordance with studies by Lakens(37) and Cumming and Finch.(38) The a priori determined…”
Method
A meta‐analysis of psychopathy‐, antisocial PD‐ and FFM associations
Decuyper, M., Pauw, S. S. W. D., Fruyt, F. D., Bolle, M. D., & Clercq, B. J. D. (2009) · European Journal of Personality · doi:10.1002/per.729 · Cited by 159
“…confidence intervals was used to identify substantive differences in the FFM associations of psychopathy and APD, with completely non-overlapping intervals reflecting a conservative significance level (approximately p .01) in light of the number of comparisons being made (Cumming & Finch, 2005).”
Method
Diagnosing the locus of trust: A temporal perspective for trustor, trustee, and dyadic influences on perceived trustworthiness.
Jones, S. L., & Shah, P. P. (2015) · Journal of Applied Psychology · doi:10.1037/apl0000041 · Cited by 160
“The confidence intervals provide a range of plausible values for the estimated differences (Cumming & Finch, 2005).”
High-Risk Sex Offenders May Not Be High Risk Forever
Hanson, R. K., Harris, A. J. R., Helmus, L. M., & Thornton, D. (2014) · Journal of Interpersonal Violence · doi:10.1177/0886260514526062 · Cited by 160
“In particular, structured methods for evaluating sexual offenders’ criminogenic needs have been demonstrated to be incremental to Static-99/R in the prediction of sexual recidivism for prison samples (Beggs & Grace, 2010; Knight & Thornton, 2007; Olver et al., 2007) and community samples (McGrath et al., 2012). Even i…”
fMRI reveals a preference for near viewing in the human parieto-occipital cortex
Quinlan, D. J., & Culham, J. C. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.02.029 · Cited by 155
“Moreover, error bars are often misunderstood by researchers (Belia et al., 2005; Cumming and Finch, 2005) and error bars based on repeated-measures variability can be complex to display and understand (Masson and Loftus, 2003).”
Background
Physical impairments and activity limitations in people with femoroacetabular impingement: a systematic review
Diamond, L. E., Dobson, F., Bennell, K. L., Wrigley, T. V., Hodges, P. W., & Hinman, R. S. (2014) · British Journal of Sports Medicine · doi:10.1136/bjsports-2013-093340 · Cited by 162
A meta-analysis of the effects of psychotherapy with adults sexually abused in childhood
Taylor, J. E., & Harvey, S. T. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.05.008 · Cited by 156
The effect of anticipation and the specificity of sex differences for amygdala and hippocampus function in emotional memory
Mackiewicz, K. L., Sarinopoulos, I., Cleven, K. L., & Nitschke, J. B. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0601648103 · Cited by 147
Adjusting the Warm-Glow Thermostat: How Incentivizing Participation in Voluntary Green Programs Moderates Their Impact on Service Satisfaction
Giebelhausen, M. D., Chun, H. H., Cronin, J. J., & Hult, G. T. M. (2016) · Journal of Marketing · doi:10.1509/jm.14.0497 · Cited by 166
“A margin of error overlap of 50% is approximate to a p-value of .05 (see Cumming and Finch 2005, p. 176).”
Background
The Reliability and Validity of the Inventory of Callous Unemotional Traits: A Meta-Analytic Review
Cardinale, E. M., & Marsh, A. A. (2017) · Assessment · doi:10.1177/1073191117747392 · Cited by 152
Entrepreneurs' Social Capital and the Economic Performance of Small Businesses: The Moderating Role of Competitive Intensity and Entrepreneurs' Experience
Hernández, C. E., Izquierdo, C. C., & Cillán, J. G. (2016) · Strategic Entrepreneurship Journal · doi:10.1002/sej.1228 · Cited by 158
Estimation for Better Inference in Neuroscience
Calin-Jageman, R. J., & Cumming, G. (2019) · eNeuro · doi:10.1523/eneuro.0205-19.2019 · Cited by 147
“This makes it clear “by eye” that there are many compatible effect sizes in common (Cumming and Finch, 2005), giving an intuitive sense that evidence for specificity is weak.”
BackgroundSelf-citation
Impact of Criticism of Null‐Hypothesis Significance Testing on Statistical Reporting Practices in Conservation Biology
Fidler, F. M., Burgman, M. A., Cumming, G., Buttrose, R., & Thomason, N. R. (2006) · Conservation Biology · doi:10.1111/j.1523-1739.2006.00525.x · Cited by 149
“Others argue for more general use of confidence intervals, particularly in graphical form (Cumming & Finch 2001, 2005; DiStefano 2003).”
BackgroundSelf-citation
Basolateral Amygdala Response to Food Cues in the Absence of Hunger Is Associated with Weight Gain Susceptibility
Sun, X., Kroemer, N. B., Veldhuizen, M. G., Babbs, A. E., de Araújo, I. E., Gitelman, D. R., Sherwin, R. S., Sinha, R., & Small, D. M. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3884-14.2015 · Cited by 147
“These confidence intervals can be used as expectations (or “priors”) for future studies and the overlap corresponds to p-values of the difference (Cumming and Finch, 2005).”
Method
Integrating Methods to Optimize Circumplex Description and Comparison of Groups
Wright, A. G. C., Pincus, A. L., Conroy, D. E., & Hilsenroth, M. J. (2009) · Journal of Personality Assessment · doi:10.1080/00223890902935696 · Cited by 146
“Finally, the 95% CI is conceptually identical to its linear counterpart and can be interpreted the same way (see Cumming & Finch, 2005 , for a review on how to interpret CIs).”
Background
Diet and ADHD, Reviewing the Evidence: A Systematic Review of Meta-Analyses of Double-Blind Placebo-Controlled Trials Evaluating the Efficacy of Diet Interventions on the Behavior of Children with ADHD
Pelsser, L. M., Frankena, K., Toorman, J., & Rodrigues Pereira, R. (2017) · PLOS ONE · doi:10.1371/journal.pone.0169277 · Cited by 145
Representing Error bars in within-subject designs in typical software packages
O’Brien, F., & Cousineau, D. (2014) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.10.1.p056 · Cited by 144
Aesthetic Pleasure versus Aesthetic Interest: The Two Routes to Aesthetic Liking
Graf, L. K. M., & Landwehr, J. R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00015 · Cited by 143
Comparing Visual-Interactive Labeling with Active Learning: An Experimental Study
Bernard, J., Hutter, M., Zeppelzauer, M., Fellner, D. W., & Sedlmair, M. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744818 · Cited by 149
Personality, cognitive ability, and academic performance: Differential associations across school subjects and school tracks
Brandt, N. D., Lechner, C. M., Tetzner, J., & Rammstedt, B. (2019) · Journal of Personality · doi:10.1111/jopy.12482 · Cited by 142
Depression and delinquency covariation in an accelerated longitudinal sample of adolescents.
Kofler, M. J., McCart, M. R., Zajac, K., Ruggiero, K. J., Saunders, B. E., & Kilpatrick, D. G. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0024108 · Cited by 152
“Differences in regression weights are significant at p .01 if their 95% confidence intervals do not overlap, and significant at p .05 if the proportion of confidence interval overlap is approximately .50 or less (Cumming & Finch, 2005).”
Background
The statistical recommendations of the American Psychological Association Publication Manual: Effect sizes, confidence intervals, and meta‐analysis
Cumming, G., Fidler, F. M., Kalinowski, P., & Lai, J. (2011) · Australian Journal of Psychology · doi:10.1111/j.1742-9536.2011.00037.x · Cited by 140
Self-citation
Biochar-based fertilizer effects on crop productivity: a meta-analysis
Melo, L. C. A., Lehmann, J. C., Carneiro, J. S. D. S., & Arbestain, M. C. (2022) · Plant and Soil · doi:10.1007/s11104-021-05276-2 · Cited by 146
“Groups within each sub-category were considered to differ when the 95% confidence intervals did not overlap between them (Cumming and Finch 2005).”
Background
Significant and meaningful effects in sports biomechanics research
Knudson, D. (2009) · Sports Biomechanics · doi:10.1080/14763140802629966 · Cited by 146
“The confidence intervals from several studies can be combined in figures to summarize concisely the size of likely effects (Batterham and Hopkins, 2005 ; Cumming and Finch, 2005 ).”
Background
Framing and Blame Attribution in Populist Rhetoric
Busby, E. C., Gubler, J. R., & Hawkins, K. A. (2019) · The Journal of Politics · doi:10.1086/701832 · Cited by 150
Alzheimers’ disease and caregiving: a meta-analytic review comparing the mental health of primary carers to controls
Ma, M., Dorstyn, D., Ward, L., & Prentice, S. (2018) · Aging & Mental Health · doi:10.1080/13607863.2017.1370689 · Cited by 133
The Correct Interpretation of Confidence Intervals
Tan, S. H., & Tan, S. B. (2010) · Proceedings of Singapore Healthcare · doi:10.1177/201010581001900316 · Cited by 129
Building psychosocial assets and wellbeing among adolescent girls: A randomized controlled trial
Leventhal, K. S., Gillham, J. E., DeMaria, L., Andrew, G., Peabody, J. W., & Leventhal, S. (2015) · Journal of Adolescence · doi:10.1016/j.adolescence.2015.09.011 · Cited by 138
“We then examined 95% Confidence Intervals (CIs) of score differences from T1 to T2 within conditions (for a review of CI interpretation, see Cumming & Finch, 2005).”
Method
Does population size affect genetic diversity? A test with sympatric lizard species
Hague, M. T. J., & Routman, E. J. (2015) · Heredity · doi:10.1038/hdy.2015.76 · Cited by 126
“…determining which evolutionary mechanisms are actually responsible for genetic diversity is facilitated by comparing more closely related species that differ in significant aspects of life history or ecology (Leffler et al., 2012; Cutter and Payseur, 2013), especially if the species are sympatric.”
Background
Estimating confidence intervals for eigenvalues in exploratory factor analysis
Larsen, R., & Warne, R. T. (2010) · Behavior Research Methods · doi:10.3758/brm.42.3.871 · Cited by 127
“However, it should be noted that this latter method is very conservative, with an approximate p value of .006 when both CIs are 95% CIs (Cumming & Finch, 2005, p. 176), whereas the previous two applications theoretically correspond to p .05.”
Increasing the Transparency of Research Papers with Explorable Multiverse Analyses
Dragicevic, P., Jansen, Y., Sarma, A., Kay, M., & Chevalier, F. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300295 · Cited by 144
“Animating the multiverse of bootstrap datasets allows the reader to appreciate the reliability of the different quantities, trends and patterns depicted by each plot and to carry out “inference by eye” [30]: a pattern that is stable across the multiverse is a good indication that it is reliable.”
A Systematic Meta-analysis of the Reliability and Validity of Subjective Cognitive Load Questionnaires in Experimental Multimedia Learning Research
Krieglstein, F., Beege, M., Rey, G. D., Ginns, P., Krell, M., & Schneider, S. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09683-4 · Cited by 167
“To compare the effect sizes (i.e., aggregated reliability and validity estimates) for significant differences, the 95% percent confidence intervals (CI) were consulted (Cumming & Finch, 2005).”
Method
Multiple regression and correlation techniques: Recent controversies and best practices.
Hoyt, W. T., Imel, Z. E., & Chan, F. H. (2008) · Rehabilitation Psychology · doi:10.1037/a0013021 · Cited by 134
Confidence intervals and replication: Where will the next mean fall?
Cumming, G., & Maillardet, R. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.3.217 · Cited by 122
“Cumming and Finch (2005) discussed that CI and noted that in the independent means case, it is almost certainly wider than the CI on either individual mean, whereas in the case of paired data, it is usually narrower than either but can have any width from zero to considerably larger than either…”
InfluentialSelf-citation
Evaluating the potential biases in carnivore capture–recapture studies associated with the use of lure and varying density estimation techniques using photographic‐sampling data of the Malagasy civet
Gerber, B. D., Karpanty, S. M., & Kelly, M. J. (2011) · Population Ecology · doi:10.1007/s10144-011-0276-3 · Cited by 122
“We defined a significant difference between methods when the 95% confidence intervals of two means overlapped no more than half the average margin of error; this is equivalent to a conservative hypothesis test at a = 0.05 (Cumming and Finch 2005).”
MethodInfluential
Statistical Reform in Psychology: Is Anything Changing?
Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., Lo, J., McMenamin, N., & Wilson, S. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01881.x · Cited by 122
Self-citation
Values and adult age: findings from two cohorts of the European Social Survey
Robinson, O. C. (2012) · European Journal of Ageing · doi:10.1007/s10433-012-0247-3 · Cited by 137
“Confidence intervals were compared using the aforementioned technique described by Cumming and Finch (2005).”
MethodInfluential
Conceptualizing personal web usage in work contexts: A preliminary framework
Kim, S. J., & Byrne, S. E. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2011.07.006 · Cited by 125
“2 displays 95% CIs of external causes, including groupnorms, physical distances from the boss/professor , anonymity , and convenient access to the internet.”
BackgroundMethodInfluential
Error bars in within-subject designs: a comment on Baguley (2012)
Cousineau, D., & O’Brien, F. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0441-z · Cited by 119
A meta-analysis of the effects of psychotherapy with sexually abused children and adolescents
Harvey, S. T., & Taylor, J. E. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.03.006 · Cited by 118
“Therefore, confidence intervals that overlapped by one quarter (mostly non-overlapping; see Cumming & Finch, 2005) were considered as potential moderators.”
Background
Standard errors and confidence intervals in within-subjects designs: Generalizing Loftus and Masson (1994) and avoiding the biases of alternative accounts
Franz, V. H., & Loftus, G. R. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0230-1 · Cited by 118
“The rules, which we will call the 2- and 3-SEM rules, respectively, are equivalent to Cumming and Finch’s (2005) Rules 6 and 7.”
Method
Competing core processes in attention-deficit/hyperactivity disorder (ADHD): Do working memory deficiencies underlie behavioral inhibition deficits
Alderson, R. M., Rapport, M. D., Hudec, K. L., Sarver, D. E., & Kofler, M. J. (2010) · Journal of Abnormal Child Psychology · doi:10.1007/s10802-010-9387-0 · Cited by 117
Meta-analysis of the relationship between academic achievement and broad abilities of the Cattell-horn-Carroll theory
Zaboski, B. A., Kranzler, J. H., & Gage, N. A. (2018) · Journal of School Psychology · doi:10.1016/j.jsp.2018.10.001 · Cited by 130
Peer Instruction in computing: The value of instructor intervention
Zingaro, D., & Porter, L. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.09.015 · Cited by 116
“Error bars roughly represent the 68% confidence interval; visually, significant differences between two conditions are represented by error bars that are separated by the average size of the two error bars in question (Cumming & Finch, 2005).”
Background
Workforce Capacity to Respond to Children Whose Parents have a Mental Illness
Maybery, D. J., & Reupert, A. (2006) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/j.1440-1614.2006.01865.x · Cited by 115
“Confidence Intervals do not overlap; for a detail exposition of the approach see [24]).”
Background
Cognitive Functioning under Stress: Evidence from Informal Caregivers of Palliative Patients
Mackenzie, C. S., Smith, M. C., Hasher, L., Leach, L. R., & Behl, P. (2007) · Journal of Palliative Medicine · doi:10.1089/jpm.2006.0171 · Cited by 115
Effect sizes and the interpretation of research results in international business
Ellis, P. (2010) · Journal of International Business Studies · doi:10.1057/jibs.2010.39 · Cited by 114
“A growing number of guidelines has recently emerged to help authors deal with the interpretation challenge (e.g., Cumming & Finch, 2005; Ellis, 2010a; Shaver, 2008).”
Background
The Effect of Visual Appearance on the Performance of Continuous Sliders and Visual Analogue Scales
Matejka, J., Glueck, M., Grossman, T., & Fitzmaurice, G. W. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858063 · Cited by 124
“Results Due to growing concerns in various research fields over the limitations of null hypothesis significance testing for interpreting and reporting experimental results [5,8,9,42] we base our analyses and discussions on effect sizes and confidence intervals [6].”
Background
Comparison of face‐to‐face and internet interventions for body image and eating problems in adult women: An RCT
Paxton, S. J., McLean, S. A., Gollings, E. K., Faulkner, C., & Wertheim, E. H. (2007) · International Journal of Eating Disorders · doi:10.1002/eat.20446 · Cited by 111
A sophisticated model for rating water quality
Uddin, M. G., Nash, S., Rahman, A., & Olbert, A. I. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.161614 · Cited by 111
Discrimination through Optimization: How Facebook's Ad Delivery Can Lead to Biased OutcomesPreprint
Ali, M., Sapieżyński, P., Bogen, M., Korolova, A., Mislove, A. E., & Rieke, A. (2019) · arXiv (Cornell University) · Cited by 116
A reassessment of the precision of carbonate clumped isotope measurements: implications for calibrations and paleoclimate reconstructions
Fernandez, A., Müller, I. A., Rodríguez‐Sanz, L., van Dijk, J., Looser, N., & Bernasconi, S. M. (2017) · Geochemistry, Geophysics, Geosystems · doi:10.1002/2017gc007106 · Cited by 101
“The difference is, in fact, highly significant with a statistical significance much larger than 95% (Blainey et al., 2014; Cumming & Finch, 2005).”
BackgroundInfluential
Predilections for narrative outcomes: The impact of story contexts and reader preferences
Rapp, D. N., & Gerrig, R. J. (2005) · Journal of Memory and Language · doi:10.1016/j.jml.2005.04.003 · Cited by 109
“Note that the triangle on each ﬂoating conﬁdence interval sig-niﬁes the association of the conﬁdence interval with the diﬀerence between the adjacent means (Cumming & Finch, 2005).”
Some Whys and Hows of Experiments in Human—Computer Interaction
Hornbæk, K. (2013) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000043 · Cited by 109
“, [29]), or to use di↵erent modes of inference (such as Bayesian, see [107]).”
Method
Responding to public disclosure of corporate social irresponsibility in host countries: Information control and ownership control
Wang, S. L., & Li, D. (2019) · Journal of International Business Studies · doi:10.1057/s41267-019-00224-w · Cited by 109
Visual Reasoning Strategies for Effect Size Judgments and Decisions
Kale, A., Kay, M., & Hullman, J. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3030335 · Cited by 109
A Meta-Analysis of the Impact of Universal and Indicated Preventive Technology-Delivered Interventions for Higher Education Students
Conley, C. S., Durlak, J. A., Shapiro, J. B., Kirsch, A. C., & Zahniser, E. (2016) · Prevention Science · doi:10.1007/s11121-016-0662-3 · Cited by 110
“We followed the guidelines of Cumming and Finch (2005), who emphasized that interpreting significance only when the CIs of means do not overlap is too conservative.”
Method
Communicating the Results of Criterion Referenced Prediction Measures: Risk Categories for the Static-99R and Static-2002R Sexual Offender Risk Assessment Tools
Hanson, R. K., Babchishin, K., Helmus, L., Thornton, D., & Phenix, A. (2017) · Psychological Assessment · doi:10.1037/pas0000371 · Cited by 93
The Trauma and Mental Health Impacts of Coercive Control: A Systematic Review and Meta-Analysis
Lohmann, S., Cowlishaw, S., Ney, L., O’Donnell, M., & Felmingham, K. (2024) · Trauma, Violence, & Abuse · doi:10.1177/15248380231162972 · Cited by 107
“Statistical significance of subgroup differences was inferred when the 95% CIs for point estimates for each sub-group did not overlap (Cumming & Finch, 2005).”
Background
Understanding the relationship between cognitive performance and function in daily life after traumatic brain injury
Wilson, L., Horton, L., Kunzmann, K., Sahakian, B. J., Newcombe, V., Stamatakis, E. A., von Steinbüchel, N., Cunitz, K., Čović, A., Maas, A. I. R., et al. (2020) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp-2020-324492 · Cited by 105
Effects of exogenous glucagon-like peptide-1 on gastric emptying and glucose absorption in the critically ill: Relationship to glycemia*
Deane, A. M., Chapman, M. J., Fraser, R. J. L., Summers, M. J., Zaknic, A. V., Storey, J. P., Jones, K. L., Rayner, C. K., & Horowitz, M. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181d9d87a · Cited by 104
Do Differing Types of Field Experiences Make a Difference in Teacher Candidates' Perceived Level of Competence?
Caprano, M. M., Caprano, R. M., & Helfeldt, J. P. (2010) · Teacher education quarterly (Claremont, Calif.) · Cited by 101
Patenting motives, technology strategies, and open innovation
Holgersson, M., & Granstrand, O. (2017) · Management Decision · doi:10.1108/md-04-2016-0233 · Cited by 85
“…of open innovation strategies is most notably related to increasing importance of motives related to attracting external financing, and the difference compared to closed innovation is significant when analyzing overlaps of confidence intervals (e.g., Cumming, 2009, Cumming and Finch, 2005).”
Background
Depression presentations, stigma, and mental health literacy: A critical review and YouTube content analysis
Devendorf, A. R., Bender, A. M., & Rottenberg, J. A. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101843 · Cited by 114
Criminal Recidivism in Sexual Homicide Perpetrators
Hill, A., Habermann, N., Klusmann, D., Berner, W., & Briken, P. (2007) · International Journal of Offender Therapy and Comparative Criminology · doi:10.1177/0306624x07307450 · Cited by 100
“Confidence intervals (95% CI) were calculated for the reported recidivism rates (Cumming & Finch, 2005).”
Method
Evidence-based treatment for adult women with child abuse-related Complex PTSD: a quantitative review
Dorrepaal, E., Thomaes, K., Hoogendoorn, A. W., Veltman, D. J., Draijer, N., & van Balkom, A. J. L. M. (2014) · European Journal of Psychotraumatology · doi:10.3402/ejpt.v5.23613 · Cited by 99
Children’s Cognitive Performance and Selective Attention Following Recent Community Violence
McCoy, D. C., Raver, C. C., & Sharkey, P. T. (2015) · Journal of Health and Social Behavior · doi:10.1177/0022146514567576 · Cited by 99
Low ambient temperature increases food intake and dropping production, leading to incorrect estimates of hormone metabolite concentrations in European stonechats
Goymann, W., Trappschuh, M., Jensen, W., & Schwabl, I. (2006) · Hormones and Behavior · doi:10.1016/j.yhbeh.2005.12.006 · Cited by 99
Trans-generational Immune Priming Protects the Eggs Only against Gram-Positive Bacteria in the Mealworm Beetle
Dubuffet, A., Zanchi, C., Boutet, G., Moreau, J., Teixeira, M., & Moret, Y. (2015) · PLoS Pathogens · doi:10.1371/journal.ppat.1005178 · Cited by 97
Are smiles a sign of happiness? Spontaneous expressions of judo winners
Crivelli, C., Carrera, P., & Dols, J. M. F. (2014) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2014.08.009 · Cited by 96
“1) shows that the overlap and average margin of error ratio (.90) are big enough to support our conclusions (Cumming & Finch, 2005).”
Result
How Mean is the Mean?
Speelman, C. P., & McGann, M. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00451 · Cited by 100
“If they are not justifiable, then alternate tools may be necessary.”
Background
A Communication Researchers’ Guide to Null Hypothesis Significance Testing and Alternatives
Levine, T. R., Weber, R., Park, H. S., & Hullett, C. R. (2008) · Human Communication Research · doi:10.1111/j.1468-2958.2008.00318.x · Cited by 97
“Researchers’ Guide to NHST T. R. Levine et al. 192 Human Communication Research 34 (2008) 188–209 ª 2008 International Communication Association Readers are referred to Cumming and Finch (2005) for a recent and accessible treatment of confidence intervals in general.”
Background
“What exactly are you inferring?” A closer look at hypothesis testing
Newman, M. C. (2008) · Environmental Toxicology and Chemistry · doi:10.1897/07-373.1 · Cited by 103
Stereoscopic perception of real depths at large distances
Palmisano, S. A., Gillam, B., Govan, D. G., Allison, R. S., & Harris, J. M. (2010) · Journal of Vision · doi:10.1167/10.6.19 · Cited by 97
Quantitative syntheses of the effects of administrative segregation on inmates’ well-being.
Morgan, R. D., Gendreau, P., Smith, P., Gray, A. L., Labrecque, R. M., MacLean, N., Horn, S. A. V., Bolaños, A. D., Batastini, A. B., & Mills, J. F. (2016) · Psychology Public Policy and Law · doi:10.1037/law0000089 · Cited by 103
“2 For readers wishing to compare the use of CIs with traditional significance testing conclusions, see Cumming and Finch (2005) and Campbell, French, and Gendreau (2009) for an example in the forensic area.”
Method
Can Early Childhood Curriculum Enhance Social-Emotional Competence in Low-Income Children? A Meta-Analysis of the Educational Effects
Yang, W., Datu, J. A. D., Lin, X., Lau, M. M., & Li, H. (2018) · Early Education and Development · doi:10.1080/10409289.2018.1539557 · Cited by 105
“The method of examining overlapping CIs (Cumming & Finch, 2005) was used to determine whether the mean ESs from different study groups differed significantly.”
MethodInfluential
Statistical reform in medicine, psychology and ecology
Fidler, F. M., Cumming, G., Burgman, M. A., & Thomason, N. R. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.035 · Cited by 95
Self-citation
Trust, but Verify: Optimistic Visualizations of Approximate Queries for Exploring Big Data
Moritz, D., Fisher, D., Ding, B., & Wang, C. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025456 · Cited by 92
“Even with the help of visualization, users struggle to correctly interpret uncertainty and can draw incorrect conclusions [21, 7].”
Background
Strengthening statistical usage in marine ecology
Beninger, P., Boldina, I., & Katsanevakis, S. (2012) · Journal of Experimental Marine Biology and Ecology · doi:10.1016/j.jembe.2012.05.020 · Cited by 85
“…and present misconception, misuse, and misinterpretation of statistics in virtually all fields of research (e.g. Anderson et al., 2001; Cohen, 1994; Cumming and Finch, 2005; Hubbard and Bayarri, 2003; Huck, 2011; Morrison and Henkel, 1970; Sellke et al., 2001; Ziliak and McCloskey, 2008a), lending…”
BackgroundInfluential
Mindfulness-based cognitive therapy for problem gambling
de Lisle, S. M., Dowling, N. A., & Sabura Allen, J. (2011) · Clinical Case Studies · doi:10.1177/1534650111401016 · Cited by 92
Winter cover crops increase readily decomposable soil carbon, but compost drives total soil carbon during eight years of intensive, organic vegetable production in California
White, K. E., Brennan, E. B., Cavigelli, M. A., & Smith, R. (2020) · PLoS ONE · doi:10.1371/journal.pone.0228677 · Cited by 102
“Using this ‘inference by eye’ approach [46,47], the CI of a paired difference that does not include zero provides more evidence of a true effect (P<0.”
Background
Reporting and Interpreting Research in PSPB: Practices, Principles, and Pragmatics
Kashy, D. A., Donnellan, M. B., Ackerman, R. A., & Russell, D. W. (2009) · Personality and Social Psychology Bulletin · doi:10.1177/0146167208331253 · Cited by 100
How Many is Enough? Determining Optimal Sample Sizes for Normative Studies in Pediatric Neuropsychology
Bridges, A. J., & Holler, K. A. (2007) · Child Neuropsychology · doi:10.1080/09297040701233875 · Cited by 92
“The formula used for the calculations was the same formula as presented in Cumming and Finch's (2005) recent article on plotting confidence intervals:”
Method
The Campbell Paradigm as a Behavior-Predictive Reinterpretation of the Classical Tripartite Model of Attitudes
Kaiser, F. G., & Wilson, M. R. (2019) · European Psychologist · doi:10.1027/1016-9040/a000364 · Cited by 113
“Even the means for the two attitudes for people who expressed an intention to vote for or who reported voting for either Reagan or Mondale were not significantly different and were thus comparable, which can be seen in the overlapping 95% confidence intervals in Figure 3 (see Cumming & Finch, 2005).”
BackgroundResult
Work overload, burnout, and psychological ill-health symptoms: a three-wave mediation model of the employee health impairment process
Beer, L. T. D., Pienaar, J., & Rothmann, S. (2015) · Anxiety Stress & Coping · doi:10.1080/10615806.2015.1061123 · Cited by 104
“Furthermore, parameter trace plots need to be visually inspected (Cumming & Finch, 2005; Van de Schoot & Depaoli, 2014), in order to investigate the distribution of the parameters throughout the estimation process.”
Background
The Attraction Effect in Information Visualization
Dimara, E., Bezerianos, A., & Dragicevic, P. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598594 · Cited by 89
“We do not have an explanation for this asymmetry, but the wide conﬁdence intervals and their large overlap suggests that the difference may be due to a large extent to statistical noise [12].”
Background
Working More, Less or the Same During COVID-19? A Mixed Method, Intersectional Analysis of Remote Workers
Fan, W., & Moen, P. (2021) · Work and Occupations · doi:10.1177/07308884211047208 · Cited by 100
An “immigrant paradox” for adolescent externalizing behavior? Evidence from a national sample
Salas‐Wright, C. P., Vaughn, M. G., Schwartz, S. J., & Córdova, D. (2015) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-015-1115-1 · Cited by 93
“To conservatively assess statistical significance, we examined whether or not the 95 % CIs for US-born and immigrant youth overlapped [45].”
The Validity of Implicit Association Test (IAT) Measures of Sexual Attraction to Children: A Meta-Analysis
Babchishin, K., Nunes, K., & Hermann, C. A. (2013) · Archives of Sexual Behavior · doi:10.1007/s10508-012-0022-8 · Cited by 92
What is the probability of replicating a statistically significant effect?
Miller, J. O. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.4.617 · Cited by 88
“Importantly, this article should not be seen as arguing that NHST is superior to alternative statistical techniques (e.g., confidence intervals; cf. Cumming & Finch, 2005), although I do believe that NHST is one of a wide range of techniques that can usefully be employed, as long as its strengths…”
Background
Taking Stock of Criminal Profiling
Snook, B., Eastwood, J., Gendreau, P., Goggin, C., & Cullen, R. M. (2007) · Criminal Justice and Behavior · doi:10.1177/0093854806296925 · Cited by 89
Economic abuse between intimate partners in Australia: prevalence, health status, disability and financial stress
Kutin, J., Russell, R., & Reid, M. (2017) · Australian and New Zealand Journal of Public Health · doi:10.1111/1753-6405.12651 · Cited by 87
An Overview of Scientific Reproducibility: Consideration of Relevant Issues for Behavior Science/Analysis
Laraway, S., Snycerski, S., Pradhan, S., & Huitema, B. E. (2019) · Perspectives on Behavior Science · doi:10.1007/s40614-019-00193-3 · Cited by 93
“…of statistical procedures like NHST. Scientists outside of behavior science/analysis have also noted the importance of visual analysis in analyzing quantitative data (e.g., Cleveland & McGill, 1985, 1986; Cumming & Finch, 2005; Tufte, 1990, 1997, 2006, 2009; Tukey, 1977; Valentine et al., 2015).”
Method
Characteristics of Speech Errors Produced by Children With and Without Delayed Phonological Awareness Skills
Rvachew, S., Chiang, P.-Y., & Evans, N. (2007) · Language Speech and Hearing Services in Schools · doi:10.1044/0161-1461(2007/006 · Cited by 87
Using plant functional traits to explain community composition across a strong environmental filter in Australian alpine snowpatches
Venn, S., Green, K., Pickering, C. M., & Morgan, J. W. (2011) · Plant Ecology · doi:10.1007/s11258-011-9923-1 · Cited by 87
“The differences in the trait-weighted means and divergence between each snowmelt zone were determined by visually examining the overlap of 95% confidence intervals (Cumming and Finch 2005).”
Method
User Education in Automated Driving: Owner’s Manual and Interactive Tutorial Support Mental Model Formation and Human-Automation Interaction
Forster, Y., Hergeth, S., Naujoks, F., Krems, J. F., & Keinath, A. (2019) · Information · doi:10.3390/info10040143 · Cited by 85
“Cumming and Finch [50] recommend a number of n = 10 when applying CI approaches for determining statistical inference.”
Method
Increased glucocorticoid concentrations in early life cause mitochondrial inefficiency and short telomeres
Casagrande, S., Stier, A., Monaghan, P., Loveland, J. L., Boner, W., Lupi, S., Trevisi, R., & Hau, M. (2020) · Journal of Experimental Biology · doi:10.1242/jeb.222513 · Cited by 89
The motivations, values and future plans of Australian academics
Bexley, E., Arkoudis, S., & James, R. H. (2012) · Higher Education · doi:10.1007/s10734-012-9550-3 · Cited by 85

Authors: Cumming, G., & Finch, S.
Affiliations: La Trobe University (AU)
ORCID: Cumming, Geoff
Finch, Sue
Venue: American Psychologist
Publisher: American Psychological Association (APA)
Published: 2005
Volume: 60
Issue: 2
Pages: 170-180
ISSN: 0003-066X (ISSN-L)
1935-990X
DOI: 10.1037/0003-066x.60.2.170
OpenAlex: W2005534705
Cited by: 1,449
References: 59
Citations by year: 2026: 62 · 2025: 66 · 2024: 74 · 2023: 92 · 2022: 102 · 2021: 110 · 2020: 92 · 2019: 86 · 2018: 80 · 2017: 82 · 2016: 76 · 2015: 89 · 2014: 63 · 2013: 59 · 2012: 48
FWCI: 39.36
Type: journal article
Language: en
Open access: closed
Locations: American Psychologist (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords: confidence intervals, error bars, statistical significance testing, interval estimation, scholarly communication
MeSH terms: Humans, Confidence Intervals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Inference by Eye: Confidence Intervals and How to Read Pictures of Data.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Citations

Cites

Cited by

Related

Metadata

Record

About these checks