Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Abstract
Little is known about researchers' understanding of confidence intervals (CIs) and standard error (SE) bars. Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited to visit a Web site where they adjusted a figure until they judged 2 means, with error bars, to be just statistically significantly different (p < .05). Results from 473 respondents suggest that many leading researchers have severe misconceptions about how error bars relate to statistical significance, do not adequately distinguish CIs and SE bars, and do not appreciate the importance of whether the 2 means are independent or come from a repeated measures design. Better guidelines for researchers and less ambiguous graphical conventions are needed before the advantages of CIs for research communication can be realized.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In comparison to interventions for stress, the findings suggest that breathwork might be associated with similar-and non-significantly different-effects.
“Through estimating statistically significant differences and 95% CIs among studies97, in comparison to interventions for stress, our findings suggest that breathwork might be associated with similar—and non-significantly different—effects.”
From Effect of breathwork on stress and mental health: A meta-analysis of randomised-controlled trials (2023) · cites this paper as [97]
CIs are not immune to incorrect interpretations either.
“However, CIs are not immune to incorrect interpretations either (Belia et al., 2005; Hoekstra et al., 2014; Miller and Ulrich, 2016).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Belia et al., 2005
Cites
The earth is round (p < .05).
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003 · see citing works
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791 · see citing works
Things I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781 · see citing works
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449 · cited 20× · see citing works
“In fact, when sample sizes are similar and not small, and CI widths are similar, then if 95% CIs on independent means just touch, the two-tail p value is about .006 ( Cumming & Finch, 2005; Payton, Greenstone, & Schenker, 2003), not .05 as many believe.”
BackgroundMethodInfluentialSelf-citationStatistics with Confidence: Confidence Intervals and Statistical Guidelines.
JOLLIFFE, F. R., Gardner, M. J., & Altman, D. G. (1990) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982808 · Cited by 1,319 · see citing works
On judging the significance of differences by examining the overlap between confidence intervals.
Schenker, N., & Gentleman, J. F. (2001) · The American Statistician · doi:10.1198/000313001317097960 · Cited by 1,218 · see citing works
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090 · see citing works
Beyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034 · see citing works
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,031 · see citing works
Payton, M. E., Greenstone, M. H., & Nathaniel Schenker, A. (2003) · Journal of Insect Science · doi:10.1093/jis/3.1.34 · Cited by 858 · see citing works
Using confidence intervals for graphically based data interpretation.
Masson, M. E., & Loftus, G. (2003) · Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale · doi:10.1037/h0087426 · Cited by 819 · see citing works
Effect sizes for research: A broad practical approach.
Grissom, R. J., & Kim, J. J. (2005) · Cited by 732 · see citing works
What If There Were No Significance Tests?
(2013) · Psychology Press eBooks · doi:10.4324/9781315827353 · Cited by 674 · see citing works
What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology.
Schmidt, F. L. (1992) · American Psychologist · doi:10.1037/0003-066x.47.10.1173 · Cited by 564 · cited 2× · see citing works
“…hypothesis significance testing (NHST) and the use of p values is, across many disciplines, the most common statistical technique, but it is widely misunderstood (Finch, Thomason, & Cumming, 2002; Nickerson, 2000; Oakes, 1986) and can prompt poor research decision making (Schmidt, 1992, 1996).”
MethodThings I have learned (so far).
Cohen, J. (1992) · American Psychological Association eBooks · doi:10.1037/10109-028 · Cited by 553 · see citing works
Cumming, G., & Finch, S. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401614002 · Cited by 528 · see citing works
Self-citationStatistical Inference: A Commentary for the Social and Behavioural Sciences
Oakes, M. W. (1986) · Medical Entomology and Zoology · Cited by 392 · see citing works
AERA Editorial Policies regarding Statistical Significance Testing: Three Suggested Reforms
Thompson, B. (1996) · Educational Researcher · doi:10.2307/1176337 · Cited by 339 · see citing works
Editors Can Lead Researchers to Confidence Intervals, but Can't Make Them Think
Fidler, F. M., Thomason, N. R., Cumming, G., Finch, S. J., & Leeman, J. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01502008.x · Cited by 244 · see citing works
Self-citationThompson, B. (1996) · Educational Researcher · doi:10.3102/0013189x025002026 · Cited by 211 · see citing works
Statistics with Confidence. Confidence Intervals and Statistical Guidelines
Newcombe, R. G. (1990) · Journal of Medical Genetics · doi:10.1136/jmg.27.1.71 · Cited by 181 · see citing works
Replication and Researchers' Understanding of Confidence Intervals and Standard Error Bars
Cumming, G., Williams, J., & Fidler, F. M. (2004) · Understanding Statistics · doi:10.1207/s15328031us0304_5 · Cited by 172 · see citing works
Self-citationIf we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2.
Wolfe, R. S., & Hanley, J. (2002) · PubMed · Cited by 122 · see citing works
Reporting of Statistical Inference in the Journal of Applied Psychology: Little Evidence of Reform
Finch, S. J., Cumming, G., & Thomason, N. R. (2001) · Educational and Psychological Measurement · doi:10.1177/00131640121971167 · Cited by 87 · see citing works
Self-citationThompson, B. (1999) · The Journal of Psychology · doi:10.1080/00223989909599728 · Cited by 44 · see citing works
Past and Future American Psychological Association Guidelines for Statistical Practice
Finch, S., Thomason, N., & Cumming, G. (2002) · Theory & Psychology · doi:10.1177/0959354302126005 · Cited by 44 · see citing works
Self-citation
Flagged first, then by citations
1 of the 27 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
D'Ignazio, C., & Klein, L. F. (2020) · doi:10.7551/mitpress/11805.001.0001 · Cited by 1,738 · see citing works
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449 · see citing works
Self-citationVariables associated with achievement in higher education: A systematic review of meta-analyses.
Schneider, M., & Preckel, F. (2017) · Psychological Bulletin · doi:10.1037/bul0000098 · Cited by 1,162 · see citing works
Best Practices for Estimating, Interpreting, and Presenting Nonlinear Interaction Effects
Mize, T. D. (2019) · Sociological Science · doi:10.15195/v6.a4 · Cited by 982 · see citing works
Error bars in experimental biology
Cumming, G., Fidler, F., & Vaux, D. L. (2007) · The Journal of Cell Biology · doi:10.1083/jcb.200611141 · Cited by 940 · see citing works
Self-citationMccarthy, M. A. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511802454 · Cited by 758 · see citing works
Schmidt, F. L., Oh, I., & Hayes, T. L. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711007x255327 · Cited by 713 · see citing works
Inference by eye: Reading the overlap of independent confidence intervals
Cumming, G. (2008) · Statistics in Medicine · doi:10.1002/sim.3471 · Cited by 673 · see citing works
Self-citationTrust and team performance: A meta-analysis of main effects, moderators, and covariates.
De Jong, B. A., Dirks, K. T., & Gillespie, N. (2016) · Journal of Applied Psychology · doi:10.1037/apl0000110 · Cited by 616 · see citing works
Robust misinterpretation of confidence intervals
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0572-3 · Cited by 425 · see citing works
A Comparison of Expert and Nonexpert Raters Using the Consensual Assessment Technique
Kaufman, J. C., Baer, J., Cole, J. C., & Sexton∗, J. D. (2008) · Creativity Research Journal · doi:10.1080/10400410802059929 · Cited by 374 · see citing works
Chronic Ankle Instability: Evolution of the Model
Hiller, C. E., Kilbreath, S. L., & Refshauge, K. M. (2011) · Journal of Athletic Training · doi:10.4085/1062-6050-46.2.133 · Cited by 373 · see citing works
The Science of Visual Data Communication: What Works
Franconeri, S., Padilla, L., Shah, P., Zacks, J. M., & Hullman, J. R. (2021) · Gothic.net · doi:10.1177/15291006211051956 · Cited by 398 · see citing works
Decision making with visualizations: a cognitive framework across disciplines
Padilla, L., Creem-Regehr, S. H., Hegarty, M., & Stefanucci, J. K. (2018) · Cognitive Research Principles and Implications · doi:10.1186/s41235-018-0120-9 · Cited by 341 · see citing works
Cavina‐Pratesi, C., Monaco, S., Fattori, P., Galletti, C., Figley, T. D., Quinlan, D. J., Goodale, M. A., & Culham, J. C. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.2023-10.2010 · Cited by 322 · see citing works
Krzywinski, M. I., & Altman, N. (2013) · Nature Methods · doi:10.1038/nmeth.2659 · Cited by 294 · see citing works
Meaningful Mediation Analysis: Plausible Causal Inference and Informative Communication
Pieters, R. G. M. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx081 · Cited by 270 · see citing works
Error Bars Considered Harmful: Exploring Alternate Encodings for Mean and Error
Correll, M., & Gleicher, M. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346298 · Cited by 258 · see citing works
Editors Can Lead Researchers to Confidence Intervals, but Can't Make Them Think
Fidler, F. M., Thomason, N. R., Cumming, G., Finch, S. J., & Leeman, J. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01502008.x · Cited by 244 · see citing works
Self-citationKay, M., Kola, T., Hullman, J. R., & Munson, S. A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858558 · Cited by 239 · see citing works
Sotos, A. E. C., Vanhoof, S., den Noortgate, W. V., & Onghena, P. (2007) · Educational Research Review · doi:10.1016/j.edurev.2007.04.001 · Cited by 239 · see citing works
Effect of breathwork on stress and mental health: A meta-analysis of randomised-controlled trials
Fincham, G. W., Strauss, C., Montero-Marin, J., & Cavanagh, K. (2023) · Scientific Reports · doi:10.1038/s41598-022-27247-y · Cited by 224 · see citing works
Calculating and graphing within-subject confidence intervals for ANOVA
Baguley, T. S. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0123-7 · Cited by 217 · see citing works
Psychology's Replication Crisis and Clinical Psychological Science
Tackett, J. L., Brandes, C. M., King, K. M., & Markon, K. E. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095710 · Cited by 196 · see citing works
Hullman, J., Resnick, P., & Adar, E. (2015) · PLOS ONE · doi:10.1371/journal.pone.0142444 · Cited by 194 · see citing works
Confidence intervals in repeated-measures designs: The number of observations principle.
Jarmasz, J., & Hollands, J. G. (2009) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/a0014164 · Cited by 191 · see citing works
In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation
Hullman, J., Qiao, X., Correll, M., Kale, A., & Kay, M. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864889 · Cited by 187 · see citing works
Schubert, E. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00837 · Cited by 183 · see citing works
Why Authors Don't Visualize Uncertainty
Hullman, J. R. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934287 · Cited by 177 · see citing works
Confidence intervals for two sample means: Calculation, interpretation, and a few simple rules
Pfister, R., & Janczyk, M. (2013) · Advances in Cognitive Psychology · doi:10.5709/acp-0133-x · Cited by 173 · see citing works
Replication and Researchers' Understanding of Confidence Intervals and Standard Error Bars
Cumming, G., Williams, J., & Fidler, F. M. (2004) · Understanding Statistics · doi:10.1207/s15328031us0304_5 · Cited by 172 · see citing works
Self-citationConfidence intervals for two sample means: Calculation, interpretation, and a few simple rules.
Pfister, R., & Janczyk, M. (2013) · Europe PMC (PubMed Central) · doi:10.2478/v10053-008-0133-x · Cited by 166 · see citing works
Bringing the real world into the fMRI scanner: Repetition effects for pictures versus real objects
Snow, J. C., Pettypiece, C. E., Figley, T. D., McLean, A. D., Stroman, P. W., Goodale, M. A., & Culham, J. C. (2011) · Scientific Reports · doi:10.1038/srep00130 · Cited by 160 · see citing works
fMRI reveals a preference for near viewing in the human parieto-occipital cortex
Quinlan, D. J., & Culham, J. C. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2007.02.029 · Cited by 155 · see citing works
Ethical Dimensions of Visualization Research
Correll, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300418 · Cited by 150 · see citing works
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147 · see citing works
“While numerical expressions and visualizations allow for high precision, they are notoriously difficult for people to understand and are often misinterpreted, even by experts [9, 40, 43, 86].”
BackgroundInside the Black Box of Organizational Life: The Gendered Language of Performance Assessment
Correll, S. J., Weisshaar, K., Wynn, A. T., & Wehner, J. D. (2020) · American Sociological Review · doi:10.1177/0003122420962080 · Cited by 151 · see citing works
Factors Associated with Teacher Wellbeing: A Meta-Analysis
Zhou, S., Slemp, G. R., & Vella‐Brodrick, D. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09886-x · Cited by 206 · see citing works
Non-expert interpretations of hurricane forecast uncertainty visualizations
Ruginski, I. T., Boone, A. P., Padilla, L., Liu, L., Heydari, N. D., Kramer, H., Hegarty, M., Thompson, W. B., House, D. H., & Creem-Regehr, S. H. (2016) · Spatial Cognition and Computation · doi:10.1080/13875868.2015.1137577 · Cited by 134 · see citing works
Framing and Blame Attribution in Populist Rhetoric
Busby, E. C., Gubler, J. R., & Hawkins, K. A. (2019) · The Journal of Politics · doi:10.1086/701832 · Cited by 134 · see citing works
Curran‐Everett, D., & Benos, D. J. (2007) · AJP Advances in Physiology Education · doi:10.1152/advan.00022.2007 · Cited by 125 · see citing works
Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data
Kale, A., Nguyen, F., Kay, M., & Hullman, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864909 · Cited by 122 · see citing works
Confidence intervals and replication: Where will the next mean fall?
Cumming, G., & Maillardet, R. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.3.217 · Cited by 122 · see citing works
Self-citationConceptualizing personal web usage in work contexts: A preliminary framework
Kim, S. J., & Byrne, S. E. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2011.07.006 · Cited by 125 · see citing works
The Box Plots Alternative for Visualizing Quantitative Data
Nuzzo, R. L. (2016) · PM&R · doi:10.1016/j.pmrj.2016.02.001 · Cited by 119 · see citing works
Visual Reasoning Strategies for Effect Size Judgments and Decisions
Kale, A., Kay, M., & Hullman, J. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3030335 · Cited by 109 · see citing works
Ecological effects on metabolic scaling: amphipod responses to fish predators in freshwater springs
Glazier, D. S., Butler, E. M., Lombardi, S., Deptola, T. J., Reese, A. J., & Satterthwaite, E. V. (2011) · Ecological Monographs · doi:10.1890/11-0264.1 · Cited by 107 · see citing works
Authorship Bias in Violence Risk Assessment? A Systematic Review and Meta-Analysis
Singh, J. P., Grann, M., & Fazel, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0072484 · Cited by 104 · see citing works
Effects of ensemble and summary displays on interpretations of geospatial uncertainty data
Padilla, L., Ruginski, I. T., & Creem-Regehr, S. H. (2017) · Cognitive Research Principles and Implications · doi:10.1186/s41235-017-0076-1 · Cited by 105 · see citing works
Mebane, C. A., Schmidt, T. S., Miller, J. L., & Balistrieri, L. S. (2020) · Environmental Toxicology and Chemistry · doi:10.1002/etc.4663 · Cited by 105 · see citing works
Monaco, S., Cavina-Pratesi, C., Sedda, A., Fattori, P., Galletti, C., & Culham, J. C. (2011) · Journal of Neurophysiology · doi:10.1152/jn.01069.2010 · Cited by 104 · see citing works
Taylor, H., Strauss, C., & Cavanagh, K. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102078 · Cited by 102 · see citing works
Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice
Diong, J. H. L., Butler, A. A., Gandevia, S. C., & Héroux, M. E. (2018) · PLoS ONE · doi:10.1371/journal.pone.0202121 · Cited by 98 · see citing works
Strengthening statistical usage in marine ecology
Beninger, P., Boldina, I., & Katsanevakis, S. (2012) · Journal of Experimental Marine Biology and Ecology · doi:10.1016/j.jembe.2012.05.020 · Cited by 85 · see citing works
Learning About Statistical Inference
Makar, K., & Rubin, A. (2018) · Springer international handbooks of education · doi:10.1007/978-3-319-66195-7_8 · Cited by 88 · see citing works
Confidence intervals for dependent data: Equating non-overlap with statistical significance
Afshartous, D. R., & Preston, R. A. (2010) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2010.04.011 · Cited by 86 · see citing works
Padilla, L., Kay, M., & Hullman, J. R. (2021) · Wiley StatsRef: Statistics Reference Online · doi:10.1002/9781118445112.stat08296 · Cited by 87 · see citing works
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82 · see citing works
Alhadad, S. S. J. (2018) · Journal of Learning Analytics · doi:10.18608/jla.2018.52.5 · Cited by 85 · see citing works
Tryon, W. W., & Lewis, C. (2008) · Psychological Methods · doi:10.1037/a0013158 · Cited by 79 · see citing works
Methodology in Our Education Research Culture
Henson, R. K., Hull, D. M., & Williams, C. S. (2010) · Educational Researcher · doi:10.3102/0013189x10365102 · Cited by 79 · see citing works
Coulson, M. (2010) · Frontiers in Psychology · doi:10.3389/fpsyg.2010.00026 · Cited by 77 · see citing works
Padilla, L. M. K., Powell, M., Kay, M., & Hullman, J. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.579267 · Cited by 74 · see citing works
Fritz, A., Scherndl, T., & Kühberger, A. (2012) · Theory & Psychology · doi:10.1177/0959354312436870 · Cited by 70 · see citing works
Why Figures with Error Bars Should Replace p Values
Fidler, F. M., & Loftus, G. R. (2009) · Zeitschrift für Psychologie · doi:10.1027/0044-3409.217.1.27 · Cited by 72 · see citing works
Self-citationToward improved statistical reporting in the journal of consulting and clinical psychology.
Fidler, F., Cumming, G., Thomason, N., Pannuzzo, D., Smith, J., Fyffe, P., Edmonds, H., Harrington, C., & Schmitt, R. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.1.136 · Cited by 67 · see citing works
Self-citationSakaluk, J. K. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.013 · Cited by 72 · see citing works
Improving communication of breast cancer recurrence risk
Brewer, N. T., Richman, A. R., DeFrank, J. T., Reyna, V. F., & Carey, L. (2011) · Breast Cancer Research and Treatment · doi:10.1007/s10549-011-1791-9 · Cited by 69 · see citing works
Human Performance Benefits of The Automation Transparency Design Principle
Skraaning, G., & Jamieson, G. A. (2019) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819887252 · Cited by 67 · see citing works
Explorations in statistics: confidence intervals
Curran‐Everett, D. (2009) · AJP Advances in Physiology Education · doi:10.1152/advan.00006.2009 · Cited by 67 · see citing works
Varieties of Confidence Intervals
Cousineau, D. (2017) · Advances in Cognitive Psychology · doi:10.5709/acp-0214-z · Cited by 67 · see citing works
Maximum Likelihood for Social Science
Ward, M., & Ahlquist, J. (2018) · Cambridge University Press eBooks · doi:10.1017/9781316888544 · Cited by 37 · see citing works
Looks Good To Me: Visualizations As Sanity Checks
Correll, M., Li, M., Kindlmann, G., & Scheidegger, C. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864907 · Cited by 65 · see citing works
Murtonen, M. (2015) · Teaching in Higher Education · doi:10.1080/13562517.2015.1072152 · Cited by 63 · see citing works
Inference by Eye: Pictures of Confidence Intervals and Thinking About Levels of Confidence
Cumming, G. (2007) · Teaching Statistics · doi:10.1111/j.1467-9639.2007.00267.x · Cited by 62 · see citing works
Self-citationIs the call to abandon p-values the red herring of the replicability crisis?
Savalei, V., & Dunn, E. W. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00245 · Cited by 62 · see citing works
Carpenter, R. W., Wood, P. K., & Trull, T. J. (2015) · Journal of Personality Disorders · doi:10.1521/pedi_2015_29_197 · Cited by 60 · see citing works
In the Eye of the Beholder: Differences Across Ethnicity and Gender in Evaluating Creative Work
Kaufman, J. C., Niu, W., Sexton, J. D., & Cole, J. C. (2010) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2009.00584.x · Cited by 59 · see citing works
Asian Countries in the Global Rice Market
Milovanovic, V., & Smutka, Ľ. (2017) · Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis · doi:10.11118/actaun201765020679 · Cited by 59 · see citing works
Epp, C. D., & Bull, S. (2015) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2015.2411604 · Cited by 55 · see citing works
The effects of newcomer proactive behaviours on socialization outcomes: A meta‐analysis
Zhao, T., Liu, J., Zawacki, A. M., Michel, J. S., & Li, H. (2022) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12407 · Cited by 63 · see citing works
der Giessen, D. V., Hollenstein, T., Hale, W. W., Koot, H. M., Meeus, W., & Branje, S. (2014) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-014-9910-9 · Cited by 58 · see citing works
The Perception of Visual UncertaintyRepresentation by Non-Experts
Tak, S., Toet, A., & van Erp, J. B. F. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.247 · Cited by 57 · see citing works
Kalinowski, P., & Fidler, F. M. (2010) · Newborn and Infant Nursing Reviews · doi:10.1053/j.nainr.2009.12.007 · Cited by 56 · see citing works
Self-citationHullman, J., Kay, M., Kim, Y.-S., & Shrestha, S. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2743898 · Cited by 55 · see citing works
Lane, D. M., & Sándor, A. (2009) · Psychological Methods · doi:10.1037/a0016620 · Cited by 54 · see citing works
The Use of Control Charts to Interpret Environmental Monitoring Data
Morrison, L. W. (2008) · Natural Areas Journal · doi:10.3375/0885-8608(2008)28[66:tuocct]2.0.co;2 · Cited by 53 · see citing works
Uncertainty Visualization: Concepts, Methods, and Applications in Biological Data Visualization
Weiskopf, D. (2022) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2022.793819 · Cited by 50 · see citing works
Type II error and statistical power in reports of small animal clinical trials.
Giuffrida, M. (2014) · Journal of the American Veterinary Medical Association · doi:10.2460/javma.244.9.1075 · Cited by 45 · see citing works
Fidler, F. (2018) · doi:10.31237/osf.io/7xdpq · Cited by 20 · see citing works
Self-citationAssessing the effectiveness of different visualizations for judgments of positional uncertainty
McKenzie, G., Hegarty, M., Barrett, T. J., & Goodchild, M. F. (2015) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2015.1082566 · Cited by 47 · see citing works
STATISTICAL COGNITION: TOWARDS EVIDENCE-BASED PRACTICE IN STATISTICS AND STATISTICS EDUCATION
Beyth-Marom, R., Fidler, F., & Cumming, G. (2008) · Statistics Education Research Journal · doi:10.52041/serj.v7i2.468 · Cited by 50 · see citing works
Self-citationPadilla, L., Fygenson, R., Castro, S. C., & Bertini, E. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209457 · Cited by 45 · see citing works
Maximum likelihood for social science strategies for analysis
Ward, M. D., & Ahlquist, J. S. (2018) · Cited by 48 · see citing works
Moving Beyond the Bar Plot and the Line Graph to Create Informative and Attractive Graphics1
Larson‐Hall, J. (2017) · Modern Language Journal · doi:10.1111/modl.12386 · Cited by 45 · see citing works
Beyond ‘significance’: principles and practice of the Analysis of Credibility
Matthews, R. (2018) · Royal Society Open Science · doi:10.1098/rsos.171047 · Cited by 45 · see citing works
Interpretation of the p value: A national survey study in academic psychologists from Spain
Badenes-Ribera, L., Frías-Navarro, D., Monterde-i-Bort, H., & Pascual-Soler, M. (2015) · Psicothema · doi:10.7334/psicothema2014.283 · Cited by 41 · see citing works
Approaching a fair deal for significance and other concerns
Giner‐Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.01.010 · Cited by 42 · see citing works
Development of the Statistical Reasoning in Biology Concept Inventory (SRBCI)
Deane, T., Nomme, K., Jeffery, E., Pollock, C. A., & Bírol, G. (2016) · CBE—Life Sciences Education · doi:10.1187/cbe.15-06-0131 · Cited by 41 · see citing works
Boone, A. P., Gunalp, P., & Hegarty, M. (2018) · Journal of experimental psychology. Applied · doi:10.1037/xap0000166 · Cited by 39 · see citing works
- Authors
- Belia, S., Fidler, F., Williams, J., & Cumming, G.
- Affiliations
- La Trobe University (AU)
- Venue
- Psychological Methods
- Published
- 2005
- Volume
- 10
- Issue
- 4
- Pages
- 389-396
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2110405340
- Cited by
- 390
- References
- 27
- Citations by year
- 2026: 15 · 2025: 20 · 2024: 27 · 2023: 30 · 2022: 35 · 2021: 23 · 2020: 21 · 2019: 24 · 2018: 43 · 2017: 19 · 2016: 18 · 2015: 15 · 2014: 21 · 2013: 12 · 2012: 11
- FWCI
- 8.98
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- confidence intervals, error bars, statistical significance, repeated measures designs, scholarly communication
- MeSH terms
- Data Interpretation, Statistical, Humans, Research, Confidence Intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review