The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Statistical Reporting Errors and Collaboration on Statistical Analyses in Psychological Science
Open accessAbstract
Statistical analysis is error prone. A best practice for researchers using statistics would therefore be to share data among co-authors, allowing double-checking of executed tasks just as co-pilots do in aviation. To document the extent to which this 'co-piloting' currently occurs in psychology, we surveyed the authors of 697 articles published in six top psychology journals and asked them whether they had collaborated on four aspects of analyzing data and reporting results, and whether the described data had been shared between the authors. We acquired responses for 49.6% of the articles and found that co-piloting on statistical analysis and reporting results is quite uncommon among psychologists, while data sharing among co-authors seems reasonably but not completely standard. We then used an automated procedure to study the prevalence of statistical reporting errors in the articles in our sample and examined the relationship between reporting errors and co-piloting. Overall, 63% of the articles contained at least one p-value that was inconsistent with the reported test statistic and the accompanying degrees of freedom, and 20% of the articles contained at least one p-value that was inconsistent to such a degree that it may have affected decisions about statistical significance. Overall, the probability that a given p-value was inconsistent was over 10%. Co-piloting was not found to be associated with reporting errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Finally, the lowest percentage of articles with at least one gross inconsistency is 2.6 % (Berle & Starcevic, 2007) and the highest is 20.5 % (Veldkamp et al., 2014), with a median of 14.3 % (1.4 percentage points higher than the 12.9 % found in the current study).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Veldkamp et al., 2014
The cited source says what the claim attributes to it.
In the citing paper
“The percentage of articles with at least one inconsistency ranged from as low as 10.1 % (Berle & Starcevic, 2007) to as high as 63.0 % (Veldkamp et al., 2014), with a median of 46.7 % (2.9 percentage points lower than the estimated 49.6 % in the current study).”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Veldkamp et al., 2014
The cited source says what the claim attributes to it.
In the citing paper
“the investigated co-pilot activities did not explicitly include the actual checking of each other’s p-values”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Veldkamp et al., 2014
The cited source says what the claim attributes to it.
In the citing paper
“Veldkamp et al. did not find direct evidence for a relation between co-piloting and reduced prevalence of reporting errors.”
From The prevalence of statistical reporting errors in psychology (1985–2013) (2016) · cites this paper as Veldkamp et al., 2014
The cited source says what the claim attributes to it.
In the citing paper
“To minimize errors, we employed an analysis co-piloting model [41] in which every reproducibility check involved the input of at least two team members.”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reason, J. (1990) · doi:10.1017/cbo9781139062367 · Cited by 5,108
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977
Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 t Tests
Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406923 · Cited by 945
Wiegmann, D. A., & Shappell, S. A. (2003) · Cited by 777
Simple tools for understanding risks: from innumeracy to insight
Gigerenzer, G., & Edwards, A. (2003) · BMJ · doi:10.1136/bmj.327.7417.741 · Cited by 763
The poor availability of psychological research data for reanalysis.
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.726 · Cited by 543
Self-citationThe Availability of Research Data Declines Rapidly with Article Age
Vines, T. H., Albert, A. Y. K., Andrew, R. L., Débarre, F., Bock, D. G., Franklin, M. T., Gilbert, K. J., Moore, J.-S., Renaut, S., & Rennison, D. J. (2014) · Current Biology · doi:10.1016/j.cub.2013.11.014 · Cited by 482
Research works printed but not linked: 1
- American Psychological Association (2010) Publication Manual of the American Psychological Association. Sixth Edition. Washington, DC: American Psychological Association.
Cited by
Szucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761
The prevalence of statistical reporting errors in psychology (1985–2013)
Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0664-2 · Cited by 510
“Finally, the lowest percentage of articles with at least one gross inconsistency is 2.6 % (Berle & Starcevic, 2007) and the highest is 20.5 % (Veldkamp et al., 2014), with a median of 14.3 % (1.4 percentage points higher than the 12.9 % found in the current study).”
InfluentialSelf-citationHardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., et al. (2018) · Royal Society Open Science · doi:10.1098/rsos.180448 · Cited by 336
Botha, M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.727542 · Cited by 232
Editorial: Evidence on Questionable Research Practices: The Good, the Bad, and the Ugly
Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., & Rupp, D. E. (2016) · Journal of Business and Psychology · doi:10.1007/s10869-016-9456-7 · Cited by 173
Metadata
- Authors
- Veldkamp, C. L. S., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A. L. M., & Wicherts, J. M.
- Affiliations
- Tilburg University (NL)
- Corresponding author
- Veldkamp, Coosje L. S.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-12-10
- Volume
- 9
- Issue
- 12
- Pages
- e114876
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4262438
- OpenAlex
- W2011295155
- Cited by
- 69
- References
- 40
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 9 · 2022: 7 · 2021: 5 · 2020: 6 · 2019: 8 · 2018: 9 · 2017: 8 · 2016: 13 · 2015: 2
- FWCI
- 10.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research portal (Tilburg University) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- statistical reporting errors, data sharing, degrees of freedom, statistical analysis, research integrity, peer review
- MeSH terms
- Data Accuracy, Cooperative Behavior, Data Interpretation, Statistical, Humans, Periodicals as Topic, Psychology, Research Design, Statistics as Topic, Information Dissemination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Statistical Reporting Errors and Collaboration on Statistical Analyses in Psychological Science. https://app.cochat.ai/papers/10.1371/journal.pone.0114876
- The paper
- Veldkamp, C. L. S., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A. L. M., & Wicherts, J. M. (2014). Statistical Reporting Errors and Collaboration on Statistical Analyses in Psychological Science. PLoS ONE. https://doi.org/10.1371/journal.pone.0114876
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.