The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Using Bayes factor hypothesis testing in neuroscience to establish evidence of absence
Open accessAbstract
Most neuroscientists would agree that for brain research to progress, we have to know which experimental manipulations have no effect as much as we must identify those that do have an effect. The dominant statistical approaches used in neuroscience rely on P values and can establish the latter but not the former. This makes non-significant findings difficult to interpret: do they support the null hypothesis or are they simply not informative? Here we show how Bayesian hypothesis testing can be used in neuroscience studies to establish both whether there is evidence of absence and whether there is absence of evidence. Through simple tutorial-style examples of Bayesian t-tests and ANOVA using the open-source project JASP, this article aims to empower neuroscientists to use this approach to provide compelling and rigorous evidence for the absence of an effect.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Although this is common practice, non-significant p-values are not sufficient evidence to claim the absence of an effect, as they may also reflect insufficient power to detect a small non-zero effect (Keysers et al. 2020).”
From Evidence for unconscious priming using a Bayesian single-subject approach (2026) · cites this paper as Keysers et al. 2020
In the citing paper
“including (1) the ability to obtain evidence in favor of the null hypothesis and discriminate between “absence of evidence” and “evidence of absence” (Dienes, 2014; Keysers, Gazzola, & Wagenmakers, 2020);”
From The JASP guidelines for conducting and reporting a Bayesian analysis (2021) · cites this paper as Keysers, Gazzola, & Wagenmakers, 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,877
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
Redefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772
Self-citationUsing Bayes to get the most out of non-significant results
Dienes, Z. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00781 · Cited by 1,998
Statistics notes: Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1995) · BMJ · doi:10.1136/bmj.311.7003.485 · Cited by 1,946
Default Bayes factors for ANOVA designs
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2012.08.001 · Cited by 1,800
Bayesian Cognitive Modeling: A Practical Course
Lee, M. D., & Wagenmakers, E.-J. (2014) · doi:10.1017/cbo9781139087759 · Cited by 1,795
Self-citationBayesian inference for psychology. Part II: Example applications with JASP
Wagenmakers, E., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Selker, R., Gronau, Q. F., Dropmann, D., boutin, B., et al. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1323-7 · Cited by 1,788
Self-citationBayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734
The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,407
(2013) · Cited by 1,351
Self-citationJASP : Graphical Statistical Software for Common Statistical Designs
Love, J., Selker, R., Marsman, M., Jamil, T., Dropmann, D., Verhagen, J., Ly, A., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2019) · Journal of Statistical Software · doi:10.18637/jss.v088.i02 · Cited by 1,020
Self-citationErroneous analyses of interactions in neuroscience: A problem of significance
Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011) · Nature Neuroscience · doi:10.1038/nn.2886 · Cited by 864
Self-citationBayes factor design analysis: Planning for compelling evidence
Schönbrodt, F. D., & Wagenmakers, E. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1230-y · Cited by 810
Self-citationSzucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761
The fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 740
Bayes factor approaches for testing interval null hypotheses.
Morey, R. D., & Rouder, J. N. (2011) · Psychological Methods · doi:10.1037/a0024377 · Cited by 615
The fallacy of placing confidence in confidence intervals
Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E.-J. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0947-8 · Cited by 577
Self-citationOptional stopping: No problem for Bayesians
Rouder, J. N. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0595-4 · Cited by 492
Applying Occam’s razor in modeling cognition: A Bayesian approach
Myung, I. J., & Pitt, M. A. (1997) · Psychonomic Bulletin & Review · doi:10.3758/bf03210778 · Cited by 417
The importance of proving the null
Gallistel, C. R. (2009) · Psychological Review · doi:10.1037/a0015251 · Cited by 392
Why Isn't Everyone a Bayesian?
Efron, B. (1986) · The American Statistician · doi:10.1080/00031305.1986.10475342 · Cited by 379
The relation between statistical power and inference in fMRI
Cremers, H. R., Wager, T. D., & Yarkoni, T. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184923 · Cited by 356
Bayesian Benefits for the Pragmatic Researcher
Wagenmakers, E.-J., Morey, R. D., & Lee, M. D. (2016) · Current Directions in Psychological Science · doi:10.1177/0963721416643289 · Cited by 334
Self-citationCriteria for Bayesian model choice with application to variable selection
Bayarri, M. J., Berger, J. O., Forte, A., & García‐Donato, G. (2012) · The Annals of Statistics · doi:10.1214/12-aos1013 · Cited by 300
A tutorial on Bayes Factor Design Analysis using an informed prior
Stefan, A. M., Gronau, Q. F., Schönbrodt, F. D., & Wagenmakers, E.-J. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-01189-8 · Cited by 284
Self-citationEmotional Mirror Neurons in the Rat’s Anterior Cingulate Cortex
Carrillo, M., Han, Y., Migliorati, F., Liu, M., Gazzola, V., & Keysers, C. (2019) · Current Biology · doi:10.1016/j.cub.2019.03.024 · Cited by 238
Self-citationRouder, J. N., Engelhardt, C. R., McCabe, S., & Morey, R. D. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1026-5 · Cited by 190
Bayesian rank-based hypothesis testing for the rank sum test, the signed rank test, and Spearman's ρ
van Doorn, J. B., Ly, A., Marsman, M., & Wagenmakers, E. (2020) · Journal of Applied Statistics · doi:10.1080/02664763.2019.1709053 · Cited by 167
Self-citationGronau, Q. F., Ly, A., & Wagenmakers, E.-J. (2020) · The American Statistician · doi:10.1080/00031305.2018.1562983 · Cited by 150
Self-citationPrior Distributions for Objective Bayesian Analysis
Consonni, G., Fouskakis, D., Liseo, B., & Ntzoufras, I. (2018) · Bayesian Analysis · doi:10.1214/18-ba1103 · Cited by 139
The Need for Bayesian Hypothesis Testing in Psychological Science
Wagenmakers, E., Verhagen, J., Ly, A., Matzke, D., Steingroever, H., Rouder, J. N., & Morey, R. D. (2017) · Wiley eBooks · doi:10.1002/9781119095910.ch8 · Cited by 124
Self-citationDetermining informative priors for cognitive models
Lee, M. D., & Vanpaemel, W. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1238-3 · Cited by 120
ラット前帯状皮質における情動ミラーニューロン【JST・京大機械翻訳】
Maria, C., Yinging, H., Filippo, M. D., Liu, M., Gazzola, V., & Keysers, C. (2019) · Current Biology · Cited by 52
Self-citationBayesian estimation of explained variance in ANOVA designs
Marsman, M., Waldorp, L., Dablander, F., & Wagenmakers, E. (2019) · Statistica neerlandica (Print) · doi:10.1111/stan.12173 · Cited by 35
Self-citationXXXVII. On certain fundamental principles of scientific inquiry. (Second Paper)
Wrinch, D., & Jeffreys, H. (1923) · The London Edinburgh and Dublin Philosophical Magazine and Journal of Science · doi:10.1080/14786442308634125 · Cited by 31
Bayesian Latent-Normal Inference for the Rank Sum Test, the Signed Rank Test, and Spearman's $\rho$Preprint
van Doorn, J. B., Ly, A., Marsman, M., & Wagenmakers, E. (2017) · arXiv (Cornell University) · Cited by 6
Self-citation
Research works printed but not linked: 3
- Jeffreys, H. Theory of Probability (Oxford University Press, 1961).
- Etz, A. & Wagenmakers, E.-J. J. B. S. Haldane’s contribution to the Bayes factor hypothesis test. Stat. Sci. 32, 313–329 (2017). · DOI
- Jeffreys, H. Theory of Probability (Oxford University Press, 1939).
Cited by
Sorted by citations
The JASP guidelines for conducting and reporting a Bayesian analysis
van Doorn, J., van den Bergh, D., Böhm, U., Dablander, F., Derks, K., Draws, T., Etz, A., Evans, N. J., Gronau, Q. F., Haaf, J. M., et al. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01798-5 · Cited by 1,114
Self-citationArtificial intelligence and the changing sources of competitive advantage
Krakowski, S. M., Luger, J., & Raisch, S. (2022) · Strategic Management Journal · doi:10.1002/smj.3387 · Cited by 773
Zhang, J., He, Y., Liang, S., Liao, X., Li, T., Qiao, Z., Chang, C., Jia, H., & Chen, X. (2021) · Nature Communications · doi:10.1038/s41467-021-23025-y · Cited by 374
Bartos̆, F., Maier, M., Wagenmakers, E., Doucouliagos, H., & Stanley, T. D. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1594 · Cited by 156
Self-citationKeil, A., Bernat, E. M., Cohen, M. X., Ding, M., Fabiani, M., Gratton, G., Kappenman, E. S., Maris, E., Mathewson, K. E., Ward, R. T., et al. (2022) · Psychophysiology · doi:10.1111/psyp.14052 · Cited by 143
Metadata
- Authors
- Keysers, C., Gazzola, V., & Wagenmakers, E.-J.
- Affiliations
- Netherlands Institute for Neuroscience (NL)
- University of Amsterdam (NL)
- Corresponding author
- Keysers, Christian
- Venue
- Nature Neuroscience
- Published
- 2020-06-29
- Volume
- 23
- Issue
- 7
- Pages
- 788-799
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- PMC
- PMC7610527
- OpenAlex
- W3037007306
- Cited by
- 654
- References
- 41
- Citations by year
- 2026: 69 · 2025: 92 · 2024: 127 · 2023: 118 · 2022: 118 · 2021: 98 · 2020: 29
- FWCI
- 14.74
- Type
- journal article
- Language
- en
- Update notices
- correction (2020-10-06)
- Open access
- green
- pure.knaw.nl
- Licence
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- KNAW research portal (Royal Academy of Art and Sciences (KNAW)) (publishedVersion, other-oa)
- PubMed Central (acceptedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
- KNAW Research Portal (The Royal Netherlands Academy of Arts and Sciences) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (453-15-009, 452-14-015)
- H2020 European Research Council (758703)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- Bayes factor, evidence of absence, absence of evidence, Bayesian hypothesis testing, null hypothesis, p-values, Bayesian t-test, JASP
- MeSH terms
- Animals, Bayes Theorem, Brain, Humans, Neurosciences, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
