Confidence Intervals Make a Difference: Effects of Showing Confidence Intervals on Inferential Reasoning
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The use of confidence intervals (CIs) as an addition or as an alternative to null hypothesis significance testing (NHST) has been promoted as a means to make researchers more aware of the uncertainty that is inherent in statistical inference. Little is known, however, about whether presenting results via CIs affects how readers judge the probability that an effect is present in the population of interest and whether a replication would be likely to reveal the same results. In the present study, 66 PhD students were asked to interpret statistical outcomes presented as CIs or as conventional statistics ( t statistics and associated p values). Fewer misinterpretations of statistics—such as accepting the null hypothesis—and more references to effect size were found when results were presented as CIs. Furthermore, participants tended to be more certain about the existence of a population effect in the expected direction and about the replicability of the results when the results were presented following the conventions of NHST than when presented using CIs. Contrary to expectations, no evidence of a more precipitous drop in the belief of the existence of a population effect and replicability estimates when p values exceeded the significance level of .05 was found when data were presented using NHST instead of by CIs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Inference by Eye: Confidence Intervals and How to Read Pictures of Data.
Cumming, G., & Finch, S. (2005) · American Psychologist · doi:10.1037/0003-066x.60.2.170 · Cited by 1,449 · cited 4×
“Confidence intervals (CIs) are frequently mentioned as an alternative for NHST (e.g., Cumming & Finch, 2005; Kline, 2004; Schmidt & Hunter, 1997).”
BackgroundMethodResultInfluentialBeyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034
Cumming, G. (2008) · Perspectives on Psychological Science · doi:10.1111/j.1745-6924.2008.00079.x · Cited by 451
What to believe: Bayesian methods for data analysis
Kruschke, J. K. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.05.001 · Cited by 342 · cited 1×
“We define replicability as the probability that a given result will be significant in a replication of the experiment (see also Kruschke, 2010).”
BackgroundGigerenzer, G. (1998) · Theory & Psychology · doi:10.1177/0959354398082006 · Cited by 196 · cited 2×
“The fact that the use of CIs encourages references to effect size is one of the main reasons given as an argument for their use (e.g., Cumming & Fidler, 2009; Gigerenzer, 1998; Meehl, 1978; Schmidt, 1996).”
BackgroundMethodOn the logic and purpose of significance testing.
Cortina, J. M., & Dunlap, W. P. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.161 · Cited by 166 · cited 3×
“Whereas some (e.g., Cortina & Dunlap, 1997) have argued that replacing NHST by CIs is unlikely to improve scientific practice because both are based on the same information, we suggest that it might.”
BackgroundStatistical Reform in Psychology: Is Anything Changing?
Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., Lo, J., McMenamin, N., & Wilson, S. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01881.x · Cited by 122
Confidence Intervals: Better Answers to Better Questions
Cumming, G., & Fidler, F. (2009) · Zeitschrift für Psychologie / Journal of Psychology · doi:10.1027/0044-3409.217.1.15 · Cited by 83 · cited 2×
“For example, as noted by Cumming and Fidler (2009), most research questions call for a single best estimate and information about the precision of that estimate.”
BackgroundMethodCoulson, M. (2010) · Frontiers in Psychology · doi:10.3389/fpsyg.2010.00026 · Cited by 77
Cited by
Robust misinterpretation of confidence intervals
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0572-3 · Cited by 425
Self-citationThe Science of Visual Data Communication: What Works
Franconeri, S. L., Padilla, L. M., Shah, P., Zacks, J. M., & Hullman, J. (2021) · Psychological Science in the Public Interest · doi:10.1177/15291006211051956 · Cited by 360
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
Misconceptions of the p-value among Chilean and Italian Academic Psychologists
Badenes-Ribera, L., Frias-Navarro, D., Iotti, B., Bonilla-Campos, A., & Longobardi, C. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01247 · Cited by 35
Helske, J., Helske, S., Cooper, M., Ynnerman, A., & Besancon, L. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3073466 · Cited by 38
Metadata
- Authors
- Hoekstra, R., Johnson, A., & Kiers, H. A. L.
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 2012-07-24
- Volume
- 72
- Issue
- 6
- Pages
- 1039-1052
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- OpenAlex
- W2164805121
- Cited by
- 32
- References
- 9
- FWCI
- 2.10
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Confidence Intervals Make a Difference: Effects of Showing Confidence Intervals on Inferential Reasoning. https://app.cochat.ai/papers/10.1177/0013164412450297
- The paper
- Hoekstra, R., Johnson, A., & Kiers, H. A. L. (2012). Confidence Intervals Make a Difference: Effects of Showing Confidence Intervals on Inferential Reasoning. Educational and Psychological Measurement. https://doi.org/10.1177/0013164412450297
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.