Sample-size determination for the Bayesian t test and Welch’s test using the approximate adjusted fractional Bayes factor
Open accessAbstract
When two independent means μ 1 and μ 2 are compared, H 0 : μ 1 = μ 2 , H 1 : μ 1 ≠ μ 2 , and H 2 : μ 1 > μ 2 are the hypotheses of interest. This paper introduces the package , which can be used to determine the sample size needed to evaluate these hypotheses using the approximate adjusted fractional Bayes factor (AAFBF) implemented in the package . Both the Bayesian t test and the Bayesian Welch’s test are available in this package. The sample size required will be calculated such that the probability that the Bayes factor is larger than a threshold value is at least η if either the null or alternative hypothesis is true. Using the package and/or the tables provided in this paper, psychological researchers can easily determine the required sample size for their experiments.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For a one-sided t-test with power 0.80 and a Bayes factor of 3, 87 participants are needed; for a Bayes factor of 5, 99 participants are needed.
“For a one-sided t-test with a power of 0.80 and a Bayes factor of 3 (indicating evidence for the data coming from the alternative than the null hypothesis), 87 participants are needed and for a Bayes factor of 5, 99 participants are needed.”
From Eye movement desensitization and reprocessing (EMDR) therapy or supportive counseling prior to exposure therapy in patients with panic disorder: study protocol for a multicenter randomized controlled trial (IMPROVE) (2023) · cites this paper as [57]
Table 4 (Features of SSD; page unavailable) gives 87 and 99 for a one-sided t test at Bayes-factor thresholds of 3 and 5, respectively, with η = 0.80. But these are sample sizes per group, not total participants, and the calculations assume an effect size of d = 0.5. The paper defines η as a probability of exceeding the Bayes-factor threshold, rather than conventional test power. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558 · see citing works
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771 · see citing works
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,837 · see citing works
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 7× · see citing works
“with respect to insufficient power is steadily increasing (Maxwell, 2004; Button et al., 2013; Simonsohn et al., 2014). Numerous articles have criticized the Neyman–Pearson approach to hypothesis testing in the classical framework (e.g., Cohen (1994), Nickerson (2000), Sellke et al.”
BackgroundMethodInfluentialGPOWER: A general power analysis program
Erdfelder, E., Faul, F., & Buchner, A. (1996) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03203630 · Cited by 4,864 · see citing works
The earth is round (p < .05).
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003 · see citing works
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688 · see citing works
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · see citing works
Ruxton, G. D. (2006) · Behavioral Ecology · doi:10.1093/beheco/ark016 · Cited by 1,988 · see citing works
Bayesian estimation supersedes the t-test
Kruschke, J. K. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0029146 · Cited by 1,530 · see citing works
Kruschke, J. K., & Liddell, T. M. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1221-4 · Cited by 1,288 · see citing works
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090 · see citing works
Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test
Delacre, M., Lakens, D., & Leys, C. (2017) · International Review of Social Psychology · doi:10.5334/irsp.82 · Cited by 1,037 · see citing works
The Intrinsic Bayes Factor for Model Selection and Prediction
Berger, J. O., & Pericchi, L. R. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476668 · Cited by 1,008 · see citing works
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890 · see citing works
Calibration of ρ Values for Testing Precise Null Hypotheses
Sellke, T., Bayarri, M. J., & Berger, J. O. (2001) · The American Statistician · doi:10.1198/000313001300339950 · Cited by 856 · see citing works
Bayes factor design analysis: Planning for compelling evidence
Schönbrodt, F. D., & Wagenmakers, E. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1230-y · Cited by 732 · see citing works
<i>p</i> -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717 · see citing works
Fractional Bayes Factors for Model Comparison
O’Hagan, A. (1995) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1995.tb02017.x · Cited by 673 · see citing works
Maxwell, S. E. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.2.147 · Cited by 655 · see citing works
Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences.
Schönbrodt, F. D., Wagenmakers, E., Zehetleitner, M., & Perugini, M. (2015) · Psychological Methods · doi:10.1037/met0000061 · Cited by 562 · see citing works
Optional stopping: No problem for Bayesians
Rouder, J. N. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0595-4 · Cited by 492 · see citing works
The Weighted Likelihood Ratio, Linear Hypotheses on Normal Location Parameters
Dickey, J. (1971) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177693507 · Cited by 485 · see citing works
Mayr, S., Erdfelder, E., Buchner, A., & Faul, F. (2007) · Tutorials in Quantitative Methods for Psychology · doi:10.20982/tqmp.03.2.p051 · Cited by 448 · see citing works
Anderson, S. F., Kelley, K., & Maxwell, S. E. (2017) · Psychological Science · doi:10.1177/0956797617723724 · Cited by 406 · see citing works
A systematic review of Bayesian articles in psychology: The last 25 years.
van de Schoot, R., Winter, S. D., Ryan, O., Zondervan-Zwijnenburg, M. A. J., & Depaoli, S. A. (2017) · Psychological Methods · doi:10.1037/met0000100 · Cited by 376 · see citing works
Albers, C. J., & Lakens, D. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.09.004 · Cited by 366 · see citing works
Bayesian Benefits for the Pragmatic Researcher
Wagenmakers, E.-J., Morey, R. D., & Lee, M. D. (2016) · Current Directions in Psychological Science · doi:10.1177/0963721416643289 · Cited by 334 · see citing works
A tutorial on Bayes Factor Design Analysis using an informed prior
Stefan, A. M., Gronau, Q. F., Schönbrodt, F. D., & Wagenmakers, E.-J. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-01189-8 · Cited by 284 · see citing works
Why P Values Are Not a Useful Measure of Evidence in Statistical Significance Testing
Hubbard, R., & Lindsay, R. (2008) · Theory & Psychology · doi:10.1177/0959354307086923 · Cited by 244 · see citing works
Inequality Constrained Analysis of Variance: A Bayesian Approach.
Klugkist, I. G., Laudy, O., & Hoijtink, H. J. A. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.477 · Cited by 205 · see citing works
Self-citationThe Superego, the Ego, and the Id in Statistical Reasoning
Gigerenzer, G. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195153729.003.0013 · Cited by 204 · see citing works
Safeguard Power as a Protection Against Imprecise Power Estimates
Perugini, M., Gallucci, M., & Costantini, G. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614528519 · Cited by 203 · see citing works
Managing heteroscedasticity in general linear models.
Rosopa, P., Schaffer, M. M., & Schroeder, A. (2013) · Psychological methods · doi:10.1037/a0032553 · Cited by 153 · see citing works
Editorial: Bayesian methods for advancing psychological science
Vandekerckhove, J., Rouder, J. N., & Kruschke, J. K. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1443-8 · Cited by 133 · see citing works
Bayesian sample size calculations for hypothesis testing
Weiss, R. (1997) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00075 · Cited by 107 · see citing works
Training samples in objective Bayesian model selection
Berger, J. O., & Pericchi, L. R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000229 · Cited by 106 · see citing works
An encompassing prior generalization of the Savage–Dickey density ratio
Wetzels, R., Grasman, R. P. P. P., & Wagenmakers, E. (2010) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2010.03.016 · Cited by 96 · see citing works
Approximated adjusted fractional Bayes factors: A general method for testing informative hypotheses
Gu, X., Mulder, J., & Hoijtink, H. J. A. (2017) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12110 · Cited by 93 · see citing works
Self-citationMulder, J., Hoijtink, H., & de Leeuw, C. (2012) · Journal of Statistical Software · doi:10.18637/jss.v046.i02 · Cited by 87 · see citing works
Self-citationPrior adjusted default Bayes factors for testing (in)equality constrained hypotheses
Mulder, J. (2013) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2013.07.017 · Cited by 74 · see citing works
Sakaluk, J. K. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.013 · Cited by 72 · see citing works
Erratum: Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Jonathan Flint, C., Robinson, E. S. J., & Munafo, M. R. (2013) · Nature reviews. Neuroscience · doi:10.1038/nrn3502 · Cited by 65 · see citing works
Statistical evidence and sample size determination for Bayesian hypothesis testing
Santis, F. D. (2004) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(03)00198-8 · Cited by 55 · see citing works
Variance Heterogeneity in Published Psychological Research
Ruscio, J., & Roche, B. (2011) · Methodology · doi:10.1027/1614-2241/a000034 · Cited by 46 · see citing works
Bayesian evaluation of informative hypotheses for multiple populations
Hoijtink, H., Gu, X., & Mulder, J. (2018) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12145 · Cited by 42 · see citing works
Self-citationAlternative Bayes factors: Sample size determination and discriminatory power assessment
Santis, F. D. (2007) · Test · doi:10.1007/s11749-006-0017-7 · Cited by 13 · see citing works
Some experiments with computer synthesized speech
Denes, P. B. (1970) · Behavior Research Methods · doi:10.3758/bf03205713 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Jeffreys H Theory of probability 1961 3rd edn. Oxford Oxford University Press
Cited by
Sorted by citations
Endhoven, B., De Cort, K., Matthijssen, S. J. M. A., de Jongh, A., van Minnen, A., Duits, P., Schruers, K. R. J., van Dis, E. A. M., Krypotos, A. M., Gerritsen, L., et al. (2023) · BMC Psychiatry · doi:10.1186/s12888-022-04320-4 · Cited by 4 · see citing works
“Although no formal method is available to determine a repeated-measures ANOVA sample size using Bayesian statistics, approximations for t-test are available [57].”
Method
- Authors
- Fu, Q., Hoijtink, H., & Moerbeek, M.
- Affiliations
- Utrecht University (NL)
- Venue
- Behavior Research Methods
- Published
- 2020-07-06
- Volume
- 53
- Issue
- 1
- Pages
- 139-152
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- PMC
- PMC7880954
- OpenAlex
- W3038683938
- Cited by
- 47
- References
- 51
- Citations by year
- 2026: 4 · 2025: 16 · 2024: 8 · 2023: 8 · 2022: 4 · 2021: 5 · 2020: 2
- FWCI
- 3.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Behavior Research Methods (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Funders
- Netherlands Institute for Advanced Study in the Humanities and Social Sciences
- Koninklijke Nederlandse Akademie van Wetenschappen
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (024‐001‐003)
- China Scholarship Council
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- sample size determination, Bayesian t-test, Bayes factor, hypothesis testing, psychology research
- MeSH terms
- Bayes Theorem, Humans, Probability, Research Design, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
