The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
A tutorial on a practical Bayesian alternative to null-hypothesis significance testing
Open accessAbstract
Null-hypothesis significance testing remains the standard inferential tool in cognitive science despite its serious disadvantages. Primary among these is the fact that the resulting probability value does not tell the researcher what he or she usually wants to know: How probable is a hypothesis, given the obtained data? Inspired by developments presented by Wagenmakers (Psychonomic Bulletin & Review, 14, 779-804, 2007), I provide a tutorial on a Bayesian model selection approach that requires only a simple transformation of sum-of-squares values generated by the standard analysis of variance. This approach generates a graded level of evidence regarding which model (e.g., effect absent [null hypothesis] vs. effect present [alternative hypothesis]) is more strongly supported by the data. This method also obviates admonitions never to speak of accepting the null hypothesis. An Excel worksheet for computing the Bayesian analysis is provided as supplemental material.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“JZS Bayes factor ANOVAs with default prior scales [49–52] revealed support for the null hypothesis regarding main and interaction effects for all three cognitive outcomes.”
From Dual N-back working memory training in healthy adults: A randomized comparison to processing speed training. (2016) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
Bayesian model selection in social research
Raftery, A. E. (1995) · Sociological Methodology · doi:10.2307/271063 · Cited by 6,823
ANew Look at the Statistical Model Identification
AKAlKE, H. (2011) · Cited by 4,177
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,359
Computing Bayes Factors by Combining Simulation and Asymptotic Approximations
DiCiccio, T., Kass, R., Raftery, A., & Wasserman, L. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474045 · Cited by 2,029
A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion
Kass, R. E., & Wasserman, L. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476592 · Cited by 1,254
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Lindley, D. V. (1957) · Biometrika · doi:10.1093/biomet/44.1-2.187 · Cited by 994
van Meter, K. M. (1994) · Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique · doi:10.1177/075910639404200102 · Cited by 688
Berger, J. O., & Delampady, M. (1987) · Statistical Science · doi:10.1214/ss/1177013238 · Cited by 664
Sociological Methodology, 1969
Keesling, W., & Borgatta, E. (1972) · American Educational Research Journal · doi:10.2307/1161695 · Cited by 643
The importance of proving the null
Gallistel, C. R. (2009) · Psychological Review · doi:10.1037/a0015251 · Cited by 392
A Critique of the Bayesian Information Criterion for Model Selection
WEAKLIEM, D. L. (1999) · Sociological Methods & Research · doi:10.1177/0049124199027003002 · Cited by 349
Psychonarratology foundations for the empirical study of literary response
Bortolussi, M., & Dixon, P. A. (2003) · Cited by 328
Null hypothesis significance testing: On the survival of a flawed method.
Krueger, J. I. (2001) · American Psychologist · doi:10.1037/0003-066x.56.1.16 · Cited by 288
A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion
Kass, R. E., & Wasserman, L. H. (1995) · Journal of the American Statistical Association · doi:10.2307/2291327 · Cited by 256
RAFTERY, A. E. (1999) · Sociological Methods & Research · doi:10.1177/0049124199027003005 · Cited by 249
Why P Values Are Not a Useful Measure of Evidence in Statistical Significance Testing
Hubbard, R., & Lindsay, R. M. (2008) · Theory & Psychology · doi:10.1177/0959354307086923 · Cited by 226
Likelihood ratios: A simple and flexible statistic for empirical psychologists
Glover, S., & Dixon, P. (2004) · Psychonomic Bulletin & Review · doi:10.3758/bf03196706 · Cited by 218
In praise of the null hypothesis statistical test.
Hagen, R. L. (1997) · American Psychologist · doi:10.1037/0003-066x.52.1.15 · Cited by 194
Grasping beer mugs: On the dynamics of alignment effects induced by handled objects.
Bub, D. N., & Masson, M. E. J. (2010) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/a0017606 · Cited by 172
Self-citationGlover, S., & Dixon, P. (2001) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.27.3.560 · Cited by 164
Why is it easier to identify someone close than far away?
Loftus, G. R., & Harley, E. M. (2005) · Psychonomic Bulletin & Review · doi:10.3758/bf03196348 · Cited by 161
Statistical Significance: Rationale, Validity and Utility
Nayak, T. K., & Chow, S. L. (1998) · Technometrics · doi:10.2307/1270661 · Cited by 136
Glover, S. R., & Dixon, P. (2001) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.27.3.560 · Cited by 97
Précis of Statistical significance: Rationale, validity, and utility
Chow, S. L. (1998) · Behavioral and Brain Sciences · doi:10.1017/s0140525x98001162 · Cited by 94
The p-value fallacy and how to avoid it.
Dixon, P. (2003) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0087425 · Cited by 90
On the Relations among Different Measures of Visible and Informational Persistence
Loftus, G. R., & Irwin, D. E. (1998) · Cognitive Psychology · doi:10.1006/cogp.1998.0678 · Cited by 85
Can corrective feedback improve recognition memory?
Kantner, J., & Lindsay, D. S. (2010) · Memory & Cognition · doi:10.3758/mc.38.4.389 · Cited by 55
A Test of the Null Hypothesis Significance Testing Procedure Correlation Argument
Trafimow, D., & Rice, S. C. (2009) · The Journal of General Psychology · doi:10.3200/genp.136.3.261-270 · Cited by 46
Scientific versus statistical inference.
Dixon, P., & O’Reilly, T. (1999) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0087305 · Cited by 41
Polysemy effects in word naming.
Lichacz, F. M. J., Herdman, C. M., LeFevre, J., & Baird, B. (1999) · Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale · doi:10.1037/h0087309 · Cited by 27
A model-averaging approach to replication: The case of prep.
Iverson, G., Wagenmakers, E., & Lee, M. D. (2010) · Psychological Methods · doi:10.1037/a0017182 · Cited by 26
Effects of an orientation illusion on motor performance and motor imagery
Glover, S., Dixon, P., Castiello, U., & Rushworth, M. F. S. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2328-4 · Cited by 18
Object identification in preschool children and adults
Bernstein, D. M., Loftus, G. R., & Meltzoff, A. N. (2005) · Developmental Science · doi:10.1111/j.1467-7687.2005.00402.x · Cited by 17
Winkel, J. (2009) · Frontiers in Human Neuroscience · doi:10.3389/neuro.09.057.2009 · Cited by 16
Long-term repetition priming of briefly identified objects.
Breuer, A. T., Masson, M. E. J., Cohen, A.-L., & Lindsay, D. S. (2009) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0014734 · Cited by 4
Self-citation
Cited by
Sorted by citations
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,767
(2013) · Cited by 1,351
What are the odds? A practical guide to computing and reporting Bayes factors
Jarosz, A. F., & Wiley, J. (2014) · The Journal of Problem Solving · doi:10.7771/1932-6246.1167 · Cited by 1,101
Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015) · American Psychologist · doi:10.1037/a0039400 · Cited by 1,102
Steenbergen, L., Sellaro, R., van Hemert, S., Bosch, J. A., & Colzato, L. S. (2015) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2015.04.003 · Cited by 671
“To calculate Bayesian probabilities we used the method proposed by Wagenmakers (2007) and Masson (2011).”
Method
Metadata
- Authors
- Masson, M. E. J.
- Affiliations
- University of Victoria (CA)
- Corresponding author
- Masson, Michael E. J.
- Venue
- Behavior Research Methods
- Published
- 2011-02-08
- Volume
- 43
- Issue
- 3
- Pages
- 679-690
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2151069696
- Cited by
- 752
- References
- 44
- Citations by year
- 2026: 7 · 2025: 21 · 2024: 29 · 2023: 54 · 2022: 39 · 2021: 62 · 2020: 61 · 2019: 87 · 2018: 84 · 2017: 63 · 2016: 66 · 2015: 55 · 2014: 53 · 2013: 45 · 2012: 22
- FWCI
- 34.66
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Behavior Research Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- Bayesian model selection, null hypothesis significance testing, Bayes factor, analysis of variance, sums of squares, evidence rating, null hypothesis, effect size
- MeSH terms
- Bayes Theorem, Data Interpretation, Statistical, Probability, Research Design, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
