What are the odds? A practical guide to computing and reporting Bayes factors
Open accessAbstract
The purpose of this paper is to provide an easy template for the inclusion of the Bayes factor in reporting experimental results, particularly as a recommendation for articles in the Journal of Problem Solving. The Bayes factor provides information with a similar purpose to the p-value – to allow the researcher to make statistical inferences from data provided by experiments. While the p-value is widely used, the Bayes factor provides several advantages, particularly in that it allows the researcher to make a statement about the alternative hypothesis, rather than just the null hypothesis. In addition, it provides a clearer estimate of the amount of evidence present in the data. Building on previous work by authors such as Wagenmakers (2007), Rouder et al. (2009), and Masson (2011), this article provides a short introduction to Bayes factors, before providing a practical guide to their computation using examples from published work on problem solving.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
JZS Bayes factor ANOVAs with default prior scales revealed support for the null hypothesis regarding main and interaction effects for all three cognitive outcomes.
“JZS Bayes factor ANOVAs with default prior scales [49–52] revealed support for the null hypothesis regarding main and interaction effects for all three cognitive outcomes.”
From Dual N-back working memory training in healthy adults: A randomized comparison to processing speed training. (2016) · cites this paper as [52]
Cites
Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal
Ndiaye, M., Lespez, L., Tribolo, C., Rassé, M., Hadjas, I., Davidoux, S., Huysecom, É., & Douze, K. (2024) · Monash University Research Portal (Monash University) · doi:10.3929/ethz-b-000667478 · Cited by 24,015 · see citing works
Bayesian model selection in social research
Raftery, A. E. (1995) · Sociological Methodology · doi:10.2307/271063 · Cited by 6,823 · see citing works
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688 · see citing works
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · see citing works
Default Bayes factors for ANOVA designs
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2012.08.001 · Cited by 1,800 · see citing works
Bayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734 · see citing works
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536 · see citing works
Mixtures of g Priors for Bayesian Variable Selection
Liang, F., Paulo, R., Molina, G., Clyde, M. A., & Berger, J. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000001337 · Cited by 1,308 · see citing works
Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 <i>t</i> Tests
Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406923 · Cited by 945 · see citing works
Bayesian Versus Orthodox Statistics: Which Side Are You On?
Dienes, Z. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406920 · Cited by 938 · see citing works
Revised standards for statistical evidence
Johnson, V. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313476110 · Cited by 808 · see citing works
A tutorial on a practical Bayesian alternative to null-hypothesis significance testing
Masson, M. E. J. (2011) · Behavior Research Methods · doi:10.3758/s13428-010-0049-5 · Cited by 752 · see citing works
Bayes factor approaches for testing interval null hypotheses.
Morey, R. D., & Rouder, J. N. (2011) · Psychological Methods · doi:10.1037/a0024377 · Cited by 584 · see citing works
Optional stopping: No problem for Bayesians
Rouder, J. N. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0595-4 · Cited by 492 · see citing works
Posterior odds ratios for selected regression hypotheses
Zellner, A., & Siow, A. (1980) · Trabajos de Estadistica y de Investigacion Operativa · doi:10.1007/bf02888369 · Cited by 490 · see citing works
Default Bayes Factors for Model Selection in Regression
Rouder, J. N., & Morey, R. D. (2012) · Multivariate Behavioral Research · doi:10.1080/00273171.2012.734737 · Cited by 477 · see citing works
Applying Occam’s razor in modeling cognition: A Bayesian approach
Myung, I. J., & Pitt, M. A. (1997) · Psychonomic Bulletin & Review · doi:10.3758/bf03210778 · Cited by 409 · see citing works
The importance of proving the null
Gallistel, C. R. (2009) · Psychological Review · doi:10.1037/a0015251 · Cited by 392 · see citing works
Uncorking the muse: Alcohol intoxication facilitates creative problem solving
Jarosz, A. F., Colflesh, G. J. H., & Wiley, J. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2012.01.002 · Cited by 223 · see citing works
Self-citationRevisiting Mental Simulation in Language Comprehension: Six Replication Attempts
Zwaan, R. A., & Pecher, D. (2012) · PLoS ONE · doi:10.1371/journal.pone.0051382 · Cited by 214 · see citing works
Statistical Approaches to Interim Monitoring of Medical Trials: A Review and Commentary
Jennison, C., & Turnbull, B. W. (1990) · Statistical Science · doi:10.1214/ss/1177012099 · Cited by 180 · see citing works
We need statistical thinking, not statistical rituals
Gigerenzer, G. (1998) · Behavioral and Brain Sciences · doi:10.1017/s0140525x98281167 · Cited by 78 · see citing works
Why does working memory capacity predict RAPM performance? A possible role of distraction
Jarosz, A. F., & Wiley, J. (2012) · Intelligence · doi:10.1016/j.intell.2012.06.001 · Cited by 63 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
The JASP guidelines for conducting and reporting a Bayesian analysis
van Doorn, J., van den Bergh, D., Böhm, U., Dablander, F., Derks, K., Draws, T., Etz, A., Evans, N. J., Gronau, Q. F., Haaf, J. M., et al. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01798-5 · Cited by 1,114 · see citing works
Indices of Effect Existence and Significance in the Bayesian Framework
Makowski, D., Ben-Shachar, M. S., Chen, S. H. A., & Lüdecke, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02767 · Cited by 983 · see citing works
Steenbergen, L., Sellaro, R., van Hemert, S., Bosch, J. A., & Colzato, L. S. (2015) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2015.04.003 · Cited by 671 · see citing works
“|D) and p(H1|D), assuming a ‘‘unit information prior’’ (for further details, see Kass and Wasserman (1995); see also Jarosz and Wiley (2014)).”
BackgroundOf cricket chirps and car horns: The effect of nature sounds on cognitive performance
Van Hedger, S. C., Nusbaum, H. C., Clohisy, L., Jaeggi, S. M., Buschkuehl, M., & Berman, M. G. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1539-1 · Cited by 134 · see citing works
Fox, K. C. R., Shi, L., Baek, S., Raccah, O., Foster, B. L., Saha, S., Margulies, D. S., Kucyi, A., & Parvizi, J. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0910-1 · Cited by 133 · see citing works
Reasoning supports utilitarian resolutions to moral dilemmas across diverse measures.
Patil, I., Zucchelli, M. M., Kool, W., Campbell, S., Fornasier, F., Calò, M., Silani, G., Cikara, M., & Cushman, F. (2021) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000281 · Cited by 133 · see citing works
Respiration modulates olfactory memory consolidation in humans.
Arshamian, A., Iravani, B., Majid, A., & Lundström, J. N. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.3360-17.2018 · Cited by 115 · see citing works
Distinct processing of ambiguous speech in people with non-clinical auditory verbal hallucinations
Alderson-Day, B., Lima, C. F., Evans, S., Krishnan, S., Shanmugalingam, P., Fernyhough, C., & Scott, S. K. (2017) · Brain · doi:10.1093/brain/awx206 · Cited by 108 · see citing works
A computational tool for creative idea generation based on analogical reasoning and ontology
Han, J., Shi, F., Chen, L., & Childs, P. R. N. (2018) · Artificial Intelligence for Engineering Design, Analysis and Manufacturing · doi:10.1017/s0890060418000082 · Cited by 78 · see citing works
Lawlor-Savage, L., & Goghari, V. M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0151817 · Cited by 74 · see citing works
Foster, N. L., Mueller, M. L., Was, C., Rawson, K. A., & Dunlosky, J. (2019) · Memory & Cognition · doi:10.3758/s13421-019-00918-4 · Cited by 67 · see citing works
Emotion and anxiety potentiate the way attention alters visual appearance
Barbot, A., & Carrasco, M. (2018) · Scientific Reports · doi:10.1038/s41598-018-23686-8 · Cited by 62 · see citing works
Language-Capable Robots may Inadvertently Weaken Human Moral Norms
Jackson, R. B., & Williams, T. (2019) · 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2019.8673123 · Cited by 64 · see citing works
Zhang, Q., Wang, C., Zhao, Q., Yang, L., Buschkuehl, M., & Jaeggi, S. M. (2019) · Developmental Science · doi:10.1111/desc.12748 · Cited by 61 · see citing works
“A uthor M anuscript A uthor M anuscript A uthor M anuscript A uthor M anuscript one are in favor of the hypothesis and that values smaller than one are in favor of the null; see also Jarosz & Wiley, 2014).”
BackgroundMarshall, A. C., Gentsch-Ebrahimzadeh, A., & Schütz-Bosbach, S. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119011 · Cited by 36 · see citing works
Dividing attention influences contextual facilitation and revision during language comprehension
Hubbard, R. J., & Federmeier, K. D. (2021) · Brain Research · doi:10.1016/j.brainres.2021.147466 · Cited by 35 · see citing works
Generalization of socially transmitted and instructed avoidance
Cameron, G., Schlund, M. W., & Dymond, S. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00159 · Cited by 29 · see citing works
Dispositional mindfulness attenuates the emotional attentional blink
Makowski, D., Sperduti, M., Lavallée, S., Nicolas, S., & Piolino, P. (2019) · Consciousness and Cognition · doi:10.1016/j.concog.2018.11.004 · Cited by 28 · see citing works
Instructional Video Design: Investigating the Impact of Monologue- and Dialogue-style Presentations
Lee, B., & Muldner, K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376845 · Cited by 24 · see citing works
High spatial frequency filtered primes hastens happy faces categorization in autistic adults
Lacroix, A., Nalborczyk, L., Dutheil, F., Kovarski, K., Chokron, S., Garrido, M., Gomot, M., & Mermillod, M. (2021) · Brain and Cognition · doi:10.1016/j.bandc.2021.105811 · Cited by 12 · see citing works
- Authors
- Jarosz, A. F., & Wiley, J.
- Affiliations
- University of Illinois Chicago (US)
- Venue
- The Journal of Problem Solving
- Publisher
- Purdue University (bepress)
- Published
- 2014-11-07
- Volume
- 7
- Issue
- 1
- ISSN
- 1932-6246 (ISSN-L)
- OpenAlex
- W2029705137
- Cited by
- 1,101
- References
- 23
- Citations by year
- 2026: 47 · 2025: 59 · 2024: 85 · 2023: 80 · 2022: 123 · 2021: 164 · 2020: 174 · 2019: 137 · 2018: 109 · 2017: 68 · 2016: 36 · 2015: 14 · 2014: 2
- FWCI
- 42.23
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- The Journal of Problem Solving (publishedVersion)
- Purdue e-Pubs (Purdue University System) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- Bayes factor, p-values, alternative hypothesis, problem-solving research, statistical inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review