The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Must psychologists change the way they analyze their data?
Abstract
Wagenmakers, Wetzels, Borsboom, and van der Maas (2011) argued that psychologists should replace the familiar "frequentist" statistical analyses of their data with bayesian analyses. To illustrate their argument, they reanalyzed a set of psi experiments published recently in this journal by Bem (2011), maintaining that, contrary to his conclusion, his data do not yield evidence in favor of the psi hypothesis. We argue that they have incorrectly selected an unrealistic prior distribution for their analysis and that a bayesian analysis using a more reasonable distribution yields strong evidence in favor of the psi hypothesis. More generally, we argue that there are advantages to bayesian analyses that merit their increased use in the future. However, as Wagenmakers et al.'s analysis inadvertently revealed, they contain hidden traps that must be better understood before being more widely substituted for the familiar frequentist analyses currently employed by most research psychologists.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Bem et al. (2011) and Richard et al. (2003) stated that effect sizes in psychology typically fall in the range of 0.2 to 0.3.”
From Using the Student's t-test with extremely small sample sizes (2013) · cites this paper as Bem et al. (2011)
In the citing paper
“However, large effect sizes are uncommon in the behavioral/psychological sciences.”
From Using the Student's t-test with extremely small sample sizes (2013) · cites this paper as Bem et al. (2011)
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,877
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,397
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
Exposure and affect: Overview and meta-analysis of research, 1968–1987.
Bornstein, R. F. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.106.2.265 · Cited by 1,970
One Hundred Years of Social Psychology Quantitatively Described
Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003) · Review of General Psychology · doi:10.1037/1089-2680.7.4.331 · Cited by 1,558 · cited 2×
“As Utts et al. (2010) argued, It is rare that we have no information about a situation before we collect data.”
BackgroundResultWagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893
Self-citationWhy Isn't Everyone a Bayesian?
Efron, B. (1986) · The American Statistician · doi:10.1080/00031305.1986.10475342 · Cited by 379
Cited by
Sorted by citations
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,767
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Using the Student's t-test with extremely small sample sizes
de Winter, J. C. F. (2013) · University of Massachusetts Amherst · doi:10.7275/e4r6-dj05 · Cited by 992
The Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963
Replications in Psychology Research: How Often Do They Really Occur?
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460688 · Cited by 750
Metadata
- Authors
- Bem, D. J., Utts, J., & Johnson, W. O.
- Affiliations
- Cornell University (US)
- University of California, Irvine (US)
- Venue
- Journal of Personality and Social Psychology
- Published
- 2011-10
- Volume
- 101
- Issue
- 4
- Pages
- 716-719
- ISSN
- 0022-3514 (ISSN-L)
- 1939-1315
- OpenAlex
- W2042930386
- Cited by
- 105
- References
- 16
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 1 · 2023: 3 · 2022: 1 · 2021: 5 · 2020: 10 · 2019: 6 · 2018: 28 · 2017: 10 · 2016: 6 · 2015: 9 · 2014: 6 · 2013: 4 · 2012: 8
- FWCI
- 5.33
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Personality and Social Psychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Social Psychology › Paranormal Experiences and Beliefs
- Keywords
- Bayesian analysis, frequentist statistics, prior selection, reanalysis, evidence evaluation
- MeSH terms
- Data Interpretation, Statistical, Humans, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
