CoChat

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.

The (mis)reporting of statistical results in psychology journals

Open access
Bakker, Marjan, Wicherts, Jelte M. · 2011 · Behavior Research Methods

Abstract

In order to study the prevalence, nature (direction), and causes of reporting errors in psychology, we checked the consistency of reported test statistics, degrees of freedom, and p values in a random sample of high- and low-impact psychology journals. In a second study, we established the generality of reporting errors in a random sample of recent psychological articles. Our results, on the basis of 281 articles, indicate that around 18% of statistical results in the psychological literature are incorrectly reported. Inconsistencies were more common in low-impact journals than in high-impact journals. Moreover, around 15% of the articles contained at least one statistical conclusion that proved, upon recalculation, to be incorrect; that is, recalculation rendered the previously significant result insignificant, or vice versa. These errors were often in line with researchers' expectations. We classified the most common errors and contacted authors to shed light on the origins of the errors.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3758/s13428-011-0089-5

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Flow chart of articles in Studies 1 and 2
Fig. 1 Flow chart of articles in Studies 1 and 2CC BY-NC · PubMed Central
Fig. 2 Overview of the different error categories broken down by high- and low-impact journals and by exactly and inexactly reported statistical results. Wald’s confidence intervals are represented in the figure by the error bars attached to each column
Fig. 2 Overview of the different error categories broken down by high- and low-impact journals and by exactly and inexactly reported statistical results. Wald’s confidence intervals are represented in the figure by the error bars attached to each columnCC BY-NC · PubMed Central
Fig. 3 Erroneously reported p values compared with the recalculated p value. Correctly reported p values would be situated on the diagonal. The dots above the diagonal represent statistical results that were reported as a p value lower than the actual p value. The dots below the diagonal represent statistical results that were reported as a p value higher than the actual p value. The dots in the left upper block represent gross errors in which a nonsignificant result is reported as significant. The dots in the right lower block represent gross errors in which a significant result is reported as nonsignificant
Fig. 3 Erroneously reported p values compared with the recalculated p value. Correctly reported p values would be situated on the diagonal. The dots above the diagonal represent statistical results that were reported as a p value lower than the actual p value. The dots below the diagonal represent statistical results that were reported as a p value higher than the actual p value. The dots in the left upper block represent gross errors in which a nonsignificant result is reported as significant. The dots in the right lower block represent gross errors in which a significant result is reported as nonsignificantCC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 34 references

Cited by

See all 358

Metadata

Authors
Bakker, M., & Wicherts, J. M.
Affiliations
University of Amsterdam (NL)
Corresponding author
Bakker, Marjan
Venue
Behavior Research Methods
Published
2011-04-15
Volume
43
Issue
3
Pages
666-678
ISSN
  • 1554-351X (ISSN-L)
  • 1554-3528
OpenAlex
W2168452606
Cited by
356
References
34
Citations by year
2026: 7 · 2025: 12 · 2024: 9 · 2023: 34 · 2022: 16 · 2021: 49 · 2020: 21 · 2019: 34 · 2018: 29 · 2017: 33 · 2016: 31 · 2015: 23 · 2014: 18 · 2013: 20 · 2012: 19
FWCI
24.30
Type
journal article
Language
en
Open access
Licence
  • cc-by-nc
  • www.springer.com/tdm (tdm)
Locations
  • Behavior Research Methods (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • UvA-DARE (University of Amsterdam) (publishedVersion, other-oa)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
  • Research portal (Tilburg University) (submittedVersion, cc-by-nc)
  • UvA-DARE (University of Amsterdam) (submittedVersion, other-oa)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › Academic Writing and Publishing
Keywords
statistical reporting errors, p-values, test statistics, degrees of freedom, journal impact factor, significance testing, research integrity, psychology literature
MeSH terms
Humans, Periodicals as Topic, Psychology, Research Design, Statistics as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The (mis)reporting of statistical results in psychology journals. https://app.cochat.ai/papers/10.3758/s13428-011-0089-5
The paper
Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods. https://doi.org/10.3758/s13428-011-0089-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.