The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
How Bayes factors change scientific practice
Open accessAbstract
Bayes factors provide a symmetrical measure of evidence for one model versus another (e.g. H1 versus H0) in order to relate theory to data. These properties help solve some (but not all) of the problems underlying the credibility crisis in psychology. The symmetry of the measure of evidence means that there can be evidence for H0 just as much as for H1; or the Bayes factor may indicate insufficient evidence either way. PP-values cannot make this three-way distinction. Thus, Bayes factors indicate when the data count against a theory (and when they count for nothing); and thus they indicate when replications actually support H0 or H1 (in ways that power cannot). There is every reason to publish evidence supporting the null as going against it, because the evidence can be measured to be just as strong either way (thus the published record can be more balanced). Bayes factors can be BB-hacked but they mitigate the problem because a) they allow evidence in either direction so people will be less tempted to hack in just one direction; b) as a measure of evidence they are insensitive to the stopping rule; c) families of tests cannot be arbitrarily defined; and d) falsely implying a contrast is planned rather than post hoc becomes irrelevant (though the value of pre-registration is not mitigated).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1×
“Butwhere the key result is non-significant, papers are less likely to be published (Rosenthal, 1979; Simonsohn, Nelson, & Simmons, 2014).”
BackgroundEstimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“Now that the problems of what we have been doing up to now are evident (e.g. Ioannidis, 2005; John et al., 2012; Open Science Collaboration, 2015; Pashler & Harris, 2012), I hope Bayes is seriously considered as part of the solution— alongwith, for example, full transparency and online availability…”
BackgroundProbability theory the logic of science
Jaynes, E. T., & Bretthorst, G. L. (2003) · Cited by 4,575
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
(2020) · Radical Treatment · doi:10.2307/j.ctvw1d7dg.9 · Cited by 3,373
ALEXANDER, P. (1963) · Philosophical Books · doi:10.1111/j.1468-0149.1963.tb00798.x · Cited by 2,854
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · cited 1×
“…is seriously considered as part of the solution— alongwith, for example, full transparency and online availability of materials, data and analysis (Nosek et al., 2015); greater emphasis on direct replications as well as multi-experiment theory building (Asendorpf et al., 2013); and increasing…”
BackgroundA practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Using Bayes to get the most out of non-significant results
Dienes, Z. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00781 · Cited by 1,998
Self-citationP-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500
Hewson, P. (2015) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2014.12096_1.x · Cited by 1,367
(2013) · Cited by 1,351
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321 · cited 1×
“The asymmetry of p-values leads to many problems that are part of the ‘credibility crisis’ in science (Pashler & Wagenmakers, 2012).”
BackgroundScientific reasoning: the Bayesian approach
(1990) · Choice Reviews Online · doi:10.5860/choice.27-2696 · Cited by 1,245
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · cited 1×
“The Bayesian answer helps show why pre-registered reports, such as used in Cortex and now at least 16 other journals (Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014; Wagenmakers et al., 2012; see the website Registered Reports, 2015, for regular updates) are valuable.”
BackgroundLINDLEY, D. V. (1957) · Biometrika · doi:10.1093/biomet/44.1-2.187 · Cited by 977
Bayesian Versus Orthodox Statistics: Which Side Are You On?
Dienes, Z. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406920 · Cited by 938 · cited 5×
“The paradoxes mean that inferential outcome depends on more than the actual data obtained, and may depend on things which are in practice unknowable (the intentions and thoughts of experimenters; see Dienes, 2011 for explanation).”
BackgroundResultInfluentialSelf-citationHow persuasive is a good fit? A comment on theory testing.
Roberts, S., & Pashler, H. (2000) · Psychological Review · doi:10.1037//0033-295x.107.2.358 · Cited by 875
Recommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734
Is the Replicability Crisis Overblown? Three Arguments Examined
Pashler, H., & Harris, C. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463401 · Cited by 697
van Meter, K. M. (1994) · Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique · doi:10.1177/075910639404200102 · Cited by 675
Berger, J. O., & Wolpert, R. L. (1988) · doi:10.1214/lnms/1215466210 · Cited by 616
Bayes factor approaches for testing interval null hypotheses.
Morey, R. D., & Rouder, J. N. (2011) · Psychological Methods · doi:10.1037/a0024377 · Cited by 582
The fallacy of placing confidence in confidence intervals
Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E.-J. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-015-0947-8 · Cited by 577
Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences.
Schönbrodt, F. D., Wagenmakers, E.-J., Zehetleitner, M., & Perugini, M. (2017) · Psychological Methods · doi:10.1037/met0000061 · Cited by 567
Doing Bayesian Data Analysis: A Tutorial with R and BUGS
Barry, J. (2011) · Europe’s Journal of Psychology · doi:10.5964/ejop.v7i4.163 · Cited by 563 · cited 1×
“Gelman et al. (2013) and Kruschke (2010) describe how to set up hierarchical models whereby the posterior distributions of the means of different conditions is automatically influenced by the data from all conditions.”
BackgroundOptional stopping: No problem for Bayesians
Rouder, J. N. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0595-4 · Cited by 492
On the psychology of experimental surprises.
Slovic, P., & Fischhoff, B. (1977) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.3.4.544 · Cited by 485
On the Foundations of Statistical Inference
Birnbaum, A. (1962) · Journal of the American Statistical Association · doi:10.1080/01621459.1962.10480660 · Cited by 476
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
The secret lives of experiments: Methods reporting in the fMRI literature
Carp, J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.07.004 · Cited by 457
Statistical analysis and the illusion of objectivity
(1988) · Infectious Diseases Newsletter · doi:10.1016/0278-2316(88)90057-6 · Cited by 408
Good Thinking: The Foundations of Probability and Its Applications.
Berger, J. O., & Good, I. J. (1985) · Journal of the American Statistical Association · doi:10.2307/2288079 · Cited by 390
On the Plurality of (Methodological) Worlds: Estimating the Analytic Flexibility of fMRI Experiments
Carp, J. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00149 · Cited by 381
Keppel, G., & Zedeck, S. (1989) · W.H. Freeman eBooks · Cited by 355
On the Foundations of Statistical Inference
Birnbaum, A. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-0919-5_32 · Cited by 326
A peculiar prevalence of p values just below .05
Masicampo, E. J., & Lalande, D. R. (2012) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2012.711335 · Cited by 266 · cited 1×
“…to publish relevant evidence concerning a theory, another outcome is p-hacking: Pushing the data in the one direction it can for it to be recognized as strong evidence, by use of analytic flexibility (John, Loewenstein, & Prelec, 2012; Masicampo & Lalande, 2012; Simmons, Nelson, & Simonsohn, 2011).”
BackgroundScientific Reasoning: The Bayesian Approach
Shafer, G. (2007) · Journal of the American Statistical Association · doi:10.1198/jasa.2007.s176 · Cited by 261
D. Chambers, C., 1 Cardiff University Brain Research Imaging Centre, S. O. P., Feredoes, E., D. Muthukumaraswamy, S., & J. Etchells, P. (2014) · AIMS Neuroscience · doi:10.3934/neuroscience.2014.1.4 · Cited by 213
A Bayes factor meta-analysis of Bem’s ESP claim
Rouder, J. N., & Morey, R. D. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0088-7 · Cited by 194
On the Foundations of Statistical Inference: Discussion
Savage, L. J., Barnard, G., Cornfield, J., Bross, I., Box, G. E. P., Good, I. J., Lindley, D. V., Clunies-Ross, C. W., Pratt, J. W., Levene, H., et al. (1962) · Journal of the American Statistical Association · doi:10.2307/2281641 · Cited by 183
Neyman, J., & Pearson, E. S. (1967) · University of California Press eBooks · doi:10.1525/9780520339897 · Cited by 179
Bayes' rule: a tutorial introduction to Bayesian analysis
(2014) · Choice Reviews Online · doi:10.5860/choice.51-3301 · Cited by 174
Registered Reports: Realigning incentives in scientific publishing
Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P., & Willmes, K. (2015) · Cortex · doi:10.1016/j.cortex.2015.03.022 · Cited by 173
Self-citationHead Up, Foot Down: Object Words Orient Attention to the Objects' Typical Location
Estes, Z., Verges, M., & Barsalou, L. W. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02051.x · Cited by 168
Hamer, R. M., Keppel, G., & Zedeck, S. (1990) · Journal of the American Statistical Association · doi:10.2307/2289579 · Cited by 167
Prior sensitivity in theory testing: An apologia for the Bayes factor
Vanpaemel, W. (2010) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2010.07.003 · Cited by 164
Avoiding Model Selection in Bayesian Social Research
Gelman, A., & Rubin, D. B. (1995) · Sociological Methodology · doi:10.2307/271064 · Cited by 148
Informative Hypotheses: Theory and Practice for Behavioral and Social Scientists
Hoijtink, H. (2011) · Medical Entomology and Zoology · Cited by 133
An Ethical Approach to Peeking at Data
Sagarin, B. J., Ambler, J. K., & Lee, E. M. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614528214 · Cited by 126
How Bayesian statistics are needed to determine whether mental states are unconscious
Dienes, Z. P. (2015) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199688890.003.0012 · Cited by 111
Self-citation1/f noise and effort on implicit measures of bias.
Correll, J. (2008) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.94.1.48 · Cited by 99
The Relevance of Stopping Rules in Statistical Inference
Berger, J. O., & Berry, D. A. (1988) · Springer eBooks · doi:10.1007/978-1-4613-8768-8_5 · Cited by 84
Feynman, R. P. (1998) · Cited by 82
When decision heuristics and science collide
Yu, E. C., Sprenger, A. M., Thomas, R. P., & Dougherty, M. R. P. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0495-z · Cited by 82
The Analysis of Experimental Data: The Appreciation of Tea and Wine
Lindley, D. V. (1993) · Teaching Statistics · doi:10.1111/j.1467-9639.1993.tb00252.x · Cited by 78
The frequentist implications of optional stopping on Bayesian hypothesis tests
Sanborn, A. N., & Hills, T. T. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0518-9 · Cited by 66
The Consequences of the Hindsight Bias in Medical Decision Making
Arkes, H. R. (2013) · Current Directions in Psychological Science · doi:10.1177/0963721413489988 · Cited by 57
The humble Bayesian: Model checking from a fully Bayesian perspective
Morey, R. D., Romeijn, J., & Rouder, J. N. (2012) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.2012.02067.x · Cited by 41
Ten reasons why journals must review manuscripts before results are known
Chambers, C. (2014) · Addiction · doi:10.1111/add.12728 · Cited by 33
How medicine is broken, and how we can fix it
Goldacre, B. M., & Heneghan, C. J. (2015) · BMJ · doi:10.1136/bmj.h3397 · Cited by 20
Cited by
Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 3,001
“Second, where traditional significance test only allows researchers to reject the null hypothesis, science needs statistical approaches that allow us to conclude meaningful effects are absent (Dienes, 2016).”
Bayesian inference for psychology. Part II: Example applications with JASP
Wagenmakers, E.-J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Selker, R., Gronau, Q. F., Dropmann, D., Boutin, B., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1323-7 · Cited by 1,715
Kruschke, J. K., & Liddell, T. M. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1221-4 · Cited by 1,288
“(Dienes, 2016, p. 2) When a null hypothesis is rejected, people can mistakenly believe that the effect is solidly large, even though the results may actually allow for a large range of effect magnitudes, including very small effect sizes (e.g., Kline, 2004).”
InfluentialBrysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996
Rejecting or Accepting Parameter Values in Bayesian Estimation
Kruschke, J. K. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918771304 · Cited by 734
Metadata
- Authors
- Dienes, Z.
- Venue
- Journal of Mathematical Psychology
- Publisher
- Elsevier BV
- Published
- 2016-06
- Volume
- 72
- Pages
- 78-89
- ISSN
- 0022-2496 (ISSN-L)
- OpenAlex
- W2234845577
- Cited by
- 401
- References
- 68
- FWCI
- 50.64
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.