Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 977Paranormal Experiences and BeliefsStatistical Methods and Bayesian InferenceMeta-analysis and systematic reviews

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011).

Wagenmakers, Eric-Jan, Wetzels, Ruud, Borsboom, Denny, van der Maas, Han L. J.·2011·Journal of Personality and Social Psychology

Abstract

Does psi exist? D. J. Bem (2011) conducted 9 studies with over 1,000 participants in an attempt to demonstrate that future events retroactively affect people's responses. Here we discuss several limitations of Bem's experiments on psi; in particular, we show that the data analysis was partly exploratory and that one-sided p values may overstate the statistical evidence against the null hypothesis. We reanalyze Bem's data with a default Bayesian t test and show that the evidence for psi is weak to nonexistent. We argue that in order to convince a skeptical audience of a controversial claim, one needs to conduct strictly confirmatory studies and analyze the results with statistical tests that are conservative rather than liberal. We conclude that Bem's p values do not indicate evidence in favor of precognition; instead, they indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data.

Overview

Integrity signals

Registry

Crossref · 10.1037/a0022790

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 41 references

Cited by

See all 977

Metadata

Authors
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J.
Affiliations
University of Amsterdam (NL)
Venue
Journal of Personality and Social Psychology
Published
2011-03
Volume
100
Issue
3
Pages
426-432
ISSN
  • 0022-3514 (ISSN-L)
  • 1939-1315
OpenAlex
W2169244873
Cited by
977
References
41
Citations by year
2026: 35 · 2025: 83 · 2024: 65 · 2023: 72 · 2022: 75 · 2021: 91 · 2020: 85 · 2019: 71 · 2018: 89 · 2017: 69 · 2016: 54 · 2015: 46 · 2014: 46 · 2013: 44 · 2012: 36
FWCI
62.51
Type
journal article
Language
en
Open access
closed
Locations
  • Journal of Personality and Social Psychology (publishedVersion)
  • PubMed (publishedVersion)
  • UvA-DARE (University of Amsterdam) (publishedVersion)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion)
Funders
Dutch Organization for Scientific Research
Topics
  • Social Sciences › Psychology › Social Psychology › Paranormal Experiences and Beliefs
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
extrasensory perception, precognition, exploratory data analysis, statistical evidence, null hypothesis, confirmatory research, data analysis methods
MeSH terms
Bayes Theorem, Cognition, Data Interpretation, Statistical, Humans, Parapsychology, Psychology, Guidelines as Topic, Behavioral Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). https://app.cochat.ai/papers/10.1037/a0022790
The paper
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology. https://doi.org/10.1037/a0022790

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.