The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Optional stopping: No problem for Bayesians
Abstract
Optional stopping refers to the practice of peeking at data and then, based on the results, deciding whether or not to continue an experiment. In the context of ordinary significance-testing analysis, optional stopping is discouraged, because it necessarily leads to increased type I error rates over nominal values. This article addresses whether optional stopping is problematic for Bayesian inference with Bayes factors. Statisticians who developed Bayesian methods thought not, but this wisdom has been challenged by recent simulation results of Yu, Sprenger, Thomas, and Dougherty (2013) and Sanborn and Hills (2013). In this article, I show through simulation that the interpretation of Bayesian quantities does not depend on the stopping rule. Researchers using Bayesian methods may employ optional stopping in their own research and may provide Bayesian analysis of secondary data regardless of the employed stopping rule. I emphasize here the proper interpretation of Bayesian quantities as measures of subjective belief on theoretical positions, the difference between frequentist and Bayesian interpretations, and the difficulty of using frequentist intuition to conceptualize the Bayesian approach.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“They may also demonstrate the robustness of their results by using Bayesian hypothesis tests, because optional stopping is not problematic for Bayesian hypothesis testing”
From Does preregistration improve the credibility of research findings? (2020) · cites this paper as Rouder, 2014
In the citing paper
“In this Bayesian testing framework optional stopping is not considered problematic (Rouder 2014).”
From Evidence for unconscious priming using a Bayesian single-subject approach (2026) · cites this paper as Rouder 2014
In the citing paper
“(3) the ability to monitor the evidence as the data accumulate (Rouder, 2014).”
From The JASP guidelines for conducting and reporting a Bayesian analysis (2021) · cites this paper as Rouder, 2014
Citations
Cites
An introduction to probability theory and its applications
(1958) · Journal of the Franklin Institute · doi:10.1016/0016-0032(58)90688-4 · Cited by 29,269
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1093/oso/9780198503682.001.0001 · Cited by 7,878
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
An Introduction to Probability Theory and Its Applications.
Bharucha-Reid, A. T., & Feller, W. (1967) · The American Mathematical Monthly · doi:10.2307/2314609 · Cited by 6,066
Journal of the American Statistical Association
Fildes, R. (1989) · International Journal of Forecasting · doi:10.1016/0169-2070(89)90106-4 · Cited by 3,874
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
Self-citationA practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
Lindley, D. V., & Savage, L. J. (1973) · The Mathematical Gazette · doi:10.2307/3615588 · Cited by 1,943
Default Bayes factors for ANOVA designs
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2012.08.001 · Cited by 1,800
Self-citationBayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734
Research works printed but not linked: 2
- Jeffreys, H. (1961). Theory of probability (3rd ed.). New York: Oxford University Press.
- Rouder, J. N., Morey, R. D., Verhagen, J., Province, J. M., & Wagenmakers, E. J. (2014). The p < .05 rule and the hidden costs of the free lunch in inference. Manuscript under review.
Cited by
Using Bayes to get the most out of non-significant results
Dienes, Z. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00781 · Cited by 1,998
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,627
Self-citationWhat are the odds? A practical guide to computing and reporting Bayes factors
Jarosz, A. F., & Wiley, J. (2014) · The Journal of Problem Solving · doi:10.7771/1932-6246.1167 · Cited by 1,101
The JASP guidelines for conducting and reporting a Bayesian analysis
van Doorn, J., van den Bergh, D., Böhm, U., Dablander, F., Derks, K., Draws, T., Etz, A., Evans, N. J., Gronau, Q. F., Haaf, J. M., et al. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01798-5 · Cited by 1,114
“); and (3) the ability to monitor the evidence as the data accumulate (Rouder, 2014).”
BackgroundBayes factor design analysis: Planning for compelling evidence
Schönbrodt, F. D., & Wagenmakers, E.-J. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1230-y · Cited by 729
Metadata
- Authors
- Rouder, J. N.
- Affiliations
- University of Missouri (US)
- Corresponding author
- Rouder, Jeffrey N.
- ORCID
- Rouder, Jeffrey N.
- Venue
- Psychonomic Bulletin & Review
- Published
- 2014-03-22
- Volume
- 21
- Issue
- 2
- Pages
- 301-308
- ISSN
- 1069-9384 (ISSN-L)
- 1531-5320
- OpenAlex
- W2149378760
- Cited by
- 492
- References
- 40
- Citations by year
- 2026: 35 · 2025: 45 · 2024: 39 · 2023: 53 · 2022: 55 · 2021: 47 · 2020: 40 · 2019: 50 · 2018: 45 · 2017: 39 · 2016: 23 · 2015: 13 · 2014: 5
- FWCI
- 28.15
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Psychonomic Bulletin & Review (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- optional stopping, Bayes factor, Bayesian inference, stopping rules, type I error, secondary analysis, simulation
- MeSH terms
- Bayes Theorem, Data Interpretation, Statistical, Humans, Research Design, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Optional stopping: No problem for Bayesians. https://app.cochat.ai/papers/10.3758/s13423-014-0595-4
- The paper
- Rouder, J. N. (2014). Optional stopping: No problem for Bayesians. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-014-0595-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.