The DOI matches its Crossref record; 59 of the 60 references it cites were found in a registry or an index.
Bayesian Versus Orthodox Statistics: Which Side Are You On?
Open accessAbstract
Researchers are often confused about what can be inferred from significance tests. One problem occurs when people apply Bayesian intuitions to significance testing—two approaches that must be firmly separated. This article presents some common situations in which the approaches come to different conclusions; you can see where your intuitions initially lie. The situations include multiple testing, deciding when to stop running participants, and when a theory was thought of relative to finding out results. The interpretation of nonsignificant results has also been persistently problematic in a way that Bayesian inference can clarify. The Bayesian and orthodox approaches are placed in the context of different notions of rationality, and I accuse myself and others as having been irrational in the way we have been using statistics on a key notion of rationality. The reader is shown how to apply Bayesian inference in practice, using free online software, to allow more coherent inferences from data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In terms of statistics, a Bayesian approach might be more appropriate than Fisherian statistics [56] as the Fisher approach is not appropriate with limited data [48].”
From Agile science: creating useful products for behavior change in the real world (2016) · cites this paper as [56]
Citations
Cites
Probability theory the logic of science
Jaynes, E. T., & Bretthorst, G. L. (2003) · Cited by 4,600
Probability Theory: The Logic of Science
Jaynes, E. T. (2003) · doi:10.1017/cbo9780511790423 · Cited by 4,045
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688 · cited 3×
“For a p value, if the null is true, any value in the interval 0 to 1 is equally likely no matter how much data you collect (Rouder et al., 2009).”
BackgroundMethodVannieuwenhoven, N. (2020) · McGill-Queen's University Press eBooks · doi:10.2307/j.ctvw1d7dg.9 · Cited by 3,370
The Methodology of Scientific Research Programmes: Philosophical Papers
Lakatos, I. (1978) · doi:10.1017/cbo9780511621123 · Cited by 3,045
Alexander, P. F. (1963) · Philosophical Books · doi:10.1111/j.1468-0149.1963.tb00798.x · Cited by 2,859
Bargh, J. A., Chen, M., & Burrows, L. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.71.2.230 · Cited by 2,736
Bargh, J. A., Chen, M. I.-C., & Burrows, L. (1996) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.71.2.230 · Cited by 2,646
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · cited 1×
“For this reason, sooner or later, you are guaranteed to get a significant result if you run subjects long enough and stop when you get the p value you want (e.g., Wagenmakers, 2007).”
BackgroundHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Research works printed but not linked: 2
- Blume J. D., Handbook of the philosophy of statistics
- Jeffreys H., 1961, The theory of probability
Cited by
Sorted by citations
Items 801–900 of 938
Spicer, S. G. (2020) · PEARL (University of Plymouth) · doi:10.24382/405 · Cited by 1
Are you for real? Decoding realistic AI-generated faces from neural activityPreprint
Moshel, M. L., Grootswagers, T., Carlson, T. A., & Robinson, A. K. (2019) · OSF Preprints (OSF Preprints) · doi:10.17605/osf.io/n2z73 · Cited by 1
Variability of Bayes Factor estimates in Bayesian Analyses of VariancePreprint
Pfister, R. (2020) · doi:10.17605/osf.io/4djnb · Cited by 1
(2018) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-78743-785-220181022 · Cited by 1
Grootswagers, T., McKay, H., & Varlet, M. (2022) · doi:10.31234/osf.io/xq7d5 · Cited by 1
“…the alternative hypothesis of non-zero correlations (across the 16 participants) and the null hypothesis of no correlation for each point in time ( Dienes, 2011 ; Kass and Raftery, 1995 ; Rouder et al., 2009a ; Teichmann et al., 2022 ; Wagenmakers, 2007 ), using the Bayes Factor R package ( Morey…”
(2014) · Elsevier eBooks · doi:10.1016/b978-0-12-405888-0.10000-5 · Cited by 1
Fornacon-Wood, I., Mistry, H. B., Price, G. J., Finn, C. F., & O’Connor, J. P. B. (2022) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2022.08.048 · Cited by 1
Hirschhorn, R., & Schönberg, T. (2023) · Elsevier eBooks · doi:10.1016/b978-0-12-820480-1.00014-0 · Cited by 1
Decoding Remapped Spatial Information in the Peri-Saccadic PeriodPreprint
Moran, C., Johnson, P. A., Landau, A. N., & Hogendoorn, H. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.07.565952 · Cited by 1
Alcohol and cannabis use predicted by affect-urgency interactions in everyday lifePreprint
Dora, J., McCabe, C. J., Schultz, M. E., Lee, C. M., Shoda, Y., Patrick, M. E., Smith, G. T., & King, K. M. (2024) · doi:10.31234/osf.io/x4t83 · Cited by 1
Introduction matters: Manipulating trust in automation and reliance in automated driving.Preprint
Körber, M., Baseler, E., & Bengler, K. (2018) · Applied Ergonomics · doi:10.31234/osf.io/q6uak_v1 · Cited by 391
“Prior distribution for the Bayesian t test was a Cauchy distribution with r = , while a uniform Beta distribution with width of κ = 1 was the prior distribution for BFs on correlations (Ly, Verhagen, & Wagenmakers, 2016).”
Mid-Sagittal Spatialization of Working Memory: A Spoarc Effect Beyond Cultural DirectionalityPreprint
Attout, L., Suzanne, L., & Charras, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5399336 · Cited by 1
Maiväli, Ü. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-418689-7.00004-1
Violent video games and social behaviour
Tear, M. J. (2015) · The University of Queensland · doi:10.14264/uql.2015.784
When fake becomes truth: placebo-suggestion modulates conflict adaptation in the Stroop Task
Axel Cleeremans, B. (2012) · Frontiers in Human Neuroscience · doi:10.3389/conf.fnhum.2012.210.00048
Conscious and unconscious: passing judgment
Mealor, A. D. (2013) · Sussex Research Online (University of Sussex)
Donse, L. (2015) · Utrecht University Repository (Utrecht University)
A Logical Fallacy: Commentary on Moss et al.
Field, M. (2013) · European Addiction Research · doi:10.1159/000348427
Probing visual consciousness with transcranial magnetic stimulation
Allen, C. P. G. (2012) · ORCA Online Research @Cardiff (Cardiff University)
ERROR, PRAISE, ACTION AND TRAIT EFFECTS OF FEEDBACK ON COGNITIVE PERFORMANCE AND MOTIVATION
Appelgren, A. (2015) · Figshare
Hanley, C. (2015) · ORCA Online Research @Cardiff (Cardiff University)
Attention modulates somatosensory influences in passive speech listening
Schmitz, J., Díaz, B., & Sebastián‐Gallés, N. (2016) · Journal of Cognitive Psychology · doi:10.1080/20445911.2016.1206107
“…general, BFs higher than 1 are considered to show a high likelihood for differences between conditions, above 3 are considered a substantial likelihood for differences between the conditions, and below 0.3 are considered substantially not different likelihoods between the conditions (Dienes, 2011).”
BackgroundInfluentialDefinition, theory and observation
West, R., & Brown, J. (2013) · doi:10.1002/9781118484890.ch2
Investigating trial types putatively evidencing semantic conflict in the Stroop task
Hasshim, N. (2016) · Bournemouth University Research Online (Bournemouth University)
Wingham, G. (2016) · UCL Discovery (University College London)
Civile, C., Obhi, S. S., & McLaren, I. P. L. (2017) · Open Research Exeter (University of Exeter)
[Hemangioma of the vulva in the infant and small child and its treatment].
Moebius, W., & Krause, W. (1974) · PubMed
On the Logic (plus some history and philosophy) of Statistical Tests and Scientific InvestigationPreprint
Saint‐Mont, U. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01829
“sychology included. 4 The scientific style of inference Scientists, statisticians and philosophers have written much about this state of affairs (e.g., Neyman (1955, 1961), Good (1988), Barnett (1999), Dienes (2011), Cumming (2014), Spanos (2014), Haig (2016)). Instead of adding another opinion, it may be wiser to go ba…”
BackgroundInvoluntary attentional orienting to counterproductive exogenous cuesPreprint
Evens, S., Pourtois, G., & Schettino, A. (2017) · OSF Preprints (OSF Preprints) · doi:10.17605/osf.io/azh5r
Priming the production of implications.
Rees, A., & Bott, L. A. (2017) · Lancaster EPrints (Lancaster University)
Predictive processing and source monitoringin the psychosis continuum
Humpston, C. S. (2017) · ORCA Online Research @Cardiff (Cardiff University)
Posterior predictive model checking of local misfit for Bayesian Confirmatory factor analysis
Au, C. H. (2018) · JMU Scholoraly Commons (James Madison University)
Publication bias and statistical power in gerontological psychologyPreprint
Brydges, C. R. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/ruwxt
Some Remarks on the Application of Bayesian Analysis in Law
Janik, B., & Banaś, P. (2015) · Archiwum Filozofii Prawa i Filozofii Społecznej · doi:10.36280/afpifs.2015.1.31
Similar Expectation Effects for Immediate and Delayed Stimulus Repetitions
Amado, C., Rostalski, S., Grotheer, M., Wanke, N., & Kovács, G. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01379
“We also performed a t-test and calculated Bayes factor (e.g., Dienes, 2011) to test the independence of RS/ES from the ISI length and denoted evidence according to the thresholds proposed by Jeffreys (1940).”
MethodModelling and Measuring the Predictive Brain
Whyte, C. J. (2020) · The Sydney eScholarship Repository (The University of Sydney)
Noise thresholds in harmonic series maskers
Labuschagne, I. B., & Ciocca, V. (2021) · The Journal of the Acoustical Society of America · doi:10.1121/10.0004130
Doing Science: Methods and PhilosophyPreprint
van Dongen, N. N. N. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/yegvs
An introductory tutorial on Bayesian inference using a simple pedagogical toolPreprint
Campitelli, G. (2017) · doi:10.31234/osf.io/k6h9x
Static versus dynamic medical images: The role of cue utilization in diagnostic performance
Carrigan, A., Stoodley, P. W., Ng, K., Moerel, D., & Wiggins, M. W. (2021) · Applied Cognitive Psychology · doi:10.1002/acp.3861
Lacko, D., & Prošek, . (2021) · TESTFÓRUM · doi:10.5817/tf2021-14-13648
“Hodnota BF10 vyšší než jedna vyjadřuje podporu pro alternativní hypotézu, hodnota nižší než jedna pro nulovou a hodnoty BF kolem 1 pak naznačují, že data nejsou dostatečně průkazná, aby podporovala jednu, či druhou hypotézu (Dienes, 2011).”
Modifying automatic responses to food cues using cognitive control training and brain stimulation
Sedgmond, J. (2021) · ORCA Online Research @Cardiff (Cardiff University)
Bayesian Statistics in Psychological Research
Surijah, E. A., & Anggara, I. M. F. (2021) · JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia) · doi:10.15408/jp3i.v10i2.20185
“One of the superiorities of Bayesian analysis is the use of previous knowledge in data analysis (Dienes, 2011).”
MethodAttout, L., & Majerus, S. (2021) · Cognitive Development · doi:10.1016/j.cogdev.2021.101139 · Cited by 1
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.001
Construct and Data-Evaluation Validity
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.004
Single-Case Experimental Research Designs
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.009
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.007
Cautions, Negative Effects, and Replication
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.016
Selecting Measures for Research
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.011
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.021
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.013
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.022
Ideas that Begin the Research Process
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.005
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.020
Inferential Statistics in Psychology
Trafimow, D. (2020) · Oxford University Press eBooks · doi:10.1093/obo/9780199828340-0264
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.025
Presenting and Analyzing the Data
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.015
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.024
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.018
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.002
Priming enrichment in childrenPreprint
Rees, A., Carter, E., & Bott, L. A. (2021) · doi:10.31234/osf.io/gpqwr
Assessment: Types of Measures and Their Use
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.012
Experimental Research Using Group Designs
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.006
Ethical Issues and Guidelines for Research
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.017
Communication of Research Findings
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.019
Perski, O., Nikiel, A., Brown, J., & Shahab, L. (2022) · F1000Research · doi:10.12688/f1000research.86670.1
(2015) · Wiley eBooks · doi:10.1002/9781119154082.refs
PreprintGlautierBrudan2019OSIDRevisedPreprint
Glautier, S., & Brudan, O. (2019) · doi:10.31219/osf.io/uspdb
Preprint GlautierAndBrudan2018 Revised Stable Individual Differences in Occasion SettingPreprint
Glautier, S., & Brudan, O. (2018) · doi:10.31219/osf.io/6haqn
Faculty Opinions recommendation of Unconscious avoidance of eye contact in autism spectrum disorder.
Puce, A. (2017) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.732005865.793538309
Tests of Statistical Significance Made Sound
Haig, B. D. (2018) · Studies in applied philosophy, epistemology and rational ethics · doi:10.1007/978-3-030-01051-5_9 · Cited by 1
Body odor disgust sensitivity predicts authoritarian attitudes.Preprint
Liuzza, M. T., Lindholm, T., Hawley, C. N., Sendén, M. G., Ekström, I., Olsson, M. J., & Olofsson, J. (2017) · doi:10.31234/osf.io/xs3f4
The time course of synaesthetic colour perceptionPreprint
Lungu, L., Rothen, N., & Terhune, D. B. (2020) · doi:10.31234/osf.io/w2zg8
Schmid, B. (2020) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.736726087.793578052
Fear Not of Cognition in ContextPreprint
Prather, R. (2021) · doi:10.31234/osf.io/gcmpt
Involuntary attentional orienting to counterproductive exogenous cuesPreprint
Evens, S., Pourtois, G., & Schettino, A. (2017) · doi:10.31237/osf.io/2qe5b
Ioumpa, K., Graham, S. A., Clausner, T., Fisher, S. E., van Lier, R., & van Leeuwen, T. M. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/qbm4y
(2018) · Elsevier eBooks · doi:10.1016/b978-1-78548-284-7.50015-0
Witt_Criteria_For_Statistical_Significance_v1Preprint
Witt, J. K. (2018) · doi:10.31234/osf.io/x587f
Laws, M. L., Matejko, A. A., Lozano, M., Napoliello, E. M., & Eden, G. F. (2022) · NeuroImage Clinical · doi:10.1016/j.nicl.2022.103042
“While frequentist analyses can only provide evidence against the null hypothesis, not in favor of it, Bayes Factors quantify the relative amount of evidence for one hypothesis over another (Dienes, 2011; Wetzels et al., 2011).”
BackgroundReplication and Reproducibility in Psychological, Medical and other Sciences
Iso‐Ahola, S. E. (2022) · Medical Research Archives · doi:10.18103/mra.v10i2.2702
“Is it funny?” or “Is it strange?” Investigating humor through cognitive psychologyPreprint
Romaniello, C., Sedda, A., & Toraldo, A. (2022) · doi:10.31234/osf.io/aj938
Examining how attention and prediction modulate visual perception: A predictive coding view
Baker, K. S. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.235895
[PhD Dissertation] Good Research PracticesPreprint
Sarafoglou, A. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7aj5h
Being Agile: Honing New Skills and Fostering Curiosity for Increased Scientific Impact
Perski, O. (2022) · Springer eBooks · doi:10.1007/978-3-031-10754-2_10
The role of the Frontal Aslant tract in bilingual language controlPreprint
Quartarone, C., Navarrete, E., Gastaldon, S., Budisavljević, S., & Peressotti, F. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.01.526563
“This has the advantage to quantify the evidence in favor of the alternative hypothesis and to overcome the problem of multiple testing (Dienes, 2011).”
BackgroundSelection of animal models for drug efficacy
Veening-Griffioen, D. H. (2023) · Utrecht University Repository (Utrecht University) · doi:10.33540/1605
Perski, O., Nikiel, A., Brown, J., & Shahab, L. (2023) · F1000Research · doi:10.12688/f1000research.86670.2
Wei, Z. X., Dai, M., & Du, F. (2023) · Current Psychology · doi:10.1007/s12144-023-04482-8
“All participants had nor - mal or corrected-to-normal vision (including color vision) and had no previous experience in similar experiments. evidence in favor of the alternative hypothesis (Dienes, 2011 ).”
The p Value is a Poor Measure of Strength of Evidence in Multiple Sclerosis ResearchPreprint
Ma, J., Kalinčík, T., Sharmin, S., & Malpas, C. B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4498419
Clark, K., Lindsay, A., Gilligan-Lee, K. A., de‐Wit, L., Scott‐Samuel, N. E., Azevedo, F., & Pennington, C. R. (2023) · doi:10.31234/osf.io/2qx6m
REMOVED DUE TO POLICY VIOLATIONSPreprint
Brown, C. R. H., & Derakshan, N. (2023) · doi:10.31234/osf.io/5gafd
Anticipated imitation of multiple agentsPreprint
Galang, C. M., Cracco, E., & Braß, M. (2023) · doi:10.31234/osf.io/f7kr4
Metadata
- Authors
- Dienes, Z.
- Affiliations
- University of Sussex (GB)
- Corresponding author
- Dienes, Zoltan
- ORCID
- Dienes, Zoltan
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2011-05
- Volume
- 6
- Issue
- 3
- Pages
- 274-290
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2055754299
- Cited by
- 938
- References
- 60
- Citations by year
- 2026: 25 · 2025: 35 · 2024: 39 · 2023: 46 · 2022: 64 · 2021: 99 · 2020: 87 · 2019: 84 · 2018: 98 · 2017: 82 · 2016: 76 · 2015: 68 · 2014: 48 · 2013: 42 · 2012: 33
- FWCI
- 6.30
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- Bayesian inference, significance testing, null hypothesis significance testing, multiple comparisons, stopping rules
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
