The DOI matches its Crossref record.
An excess of positive results: Comparing the standard Psychology literature with Registered Reports
Open accessAbstract
Selectively publishing results that support the tested hypotheses ('positive' results) distorts the available evidence for scientific claims. For the past decade, psychological scientists have been increasingly concerned about the degree of such distortion in their literature. A new publication format has been developed to prevent selective reporting: In Registered Reports, peer review and the decision to publish take place before results are known. We compared the results in published Registered Reports (N = 71 as of November 2018) with a random sample of hypothesis-testing studies from the standard literature (N = 152) in Psychology. Analysing the first hypothesis of each paper, we found 96% positive results in standard reports, but only 44% positive results in Registered Reports. We discuss possible explanations for this large difference and suggest that a plausible factor is the reduction of publication bias and/or Type-1 error inflation in the Registered-Reports literature.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Preregistration, and especially registered reports, helps to increase the reporting of null findings”
From Does preregistration improve the credibility of research findings? (2020) · cites this paper as Scheel, Schijen, & Lakens, 2020
In the citing paper
“Making acceptance decisions prior to knowing the outcomes should and does mitigate publication bias favoring positive results 26,27 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [27]
In the citing paper
“Initial evidence suggests that RRs are (1) effective at mitigating publication bias with a sharp increase in publishing negative results compared to the standard model 26,27 ,”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [27]
In the citing paper
“This hypothesis was supported in two prior investigations 26,27”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [27]
In the citing paper
“Incentive focused innovations such as Registered Reports can shift reward away from achieving exciting results and toward asking important questions and designing rigorous studies to investigate them”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Scheel et al., 2020
Citations
Cited by
Sorted by citations
Items 101–131 of 131
Policies to support Open Source and Open science
Choudhury, S. (2022) · doi:10.4000/books.oep.16192
Nair, S. (2022) · doi:10.4000/books.oep.16091
Construire le pilier logiciel de la science ouverte
Cosmo, R. D. (2022) · doi:10.4000/books.oep.16034
Science ouverte, édition et code
Harrison, M. (2022) · doi:10.4000/books.oep.16052
Conference, O. S. E. (2022) · doi:10.4000/books.oep.16237
To What extend are Early Career Researchers the Harbingers of Change?
Boukacem‐Zeghmouri, C. (2022) · doi:10.4000/books.oep.16159
How Psychology deals with Open Science. Principles and practices
Cristea, I. A. (2022) · doi:10.4000/books.oep.16112
New forms of publication. Why? How? How far?
Polka, J. (2022) · doi:10.4000/books.oep.16139
Gabriel, M. (2022) · doi:10.4000/books.oep.15852
Open Science Awards Ceremony and Conclusions
Giry, C., & Berre, D. L. (2022) · doi:10.4000/books.oep.16194
Conference, O. S. E. (2022) · doi:10.4000/books.oep.16236
Faire de la place pour les talents de chacun
Huijpen, K. (2022) · doi:10.4000/books.oep.16006
Transparency of Health Research
Boutron, I. (2022) · doi:10.4000/books.oep.16107
Academic Publishing and Latin America
Becerril-García, A. (2022) · doi:10.4000/books.oep.16142
Transparency and Reproducibility in the Journal of Contextual Behavioral Science (preprint)Preprint
Lear, M. K., Spata, A., Tittler, M. V., Fishbein, J. N., Arch, J. J., & Luoma, J. B. (2023) · doi:10.31219/osf.io/ugcwd
MacGregor, O., & Tolgay, B. (2023) · doi:10.31234/osf.io/v4675
Registered Reports: benefits and challenges of implementing in medicinePreprint
Lloyd, K. E., & Chambers, C. (2023) · doi:10.31219/osf.io/mfj6s
The extreme relevance of avoiding the file drawer effect in educational research (Editorial).
Montaño, J. L. A. (2023) · EDUCADE - Revista de Educación en Contabilidad Finanzas y Administración de Empresas · doi:10.12795/educade.2023.i14.01
Why we should all be reflexive: The case for reflexivity in quantitative sciencePreprint
Lammer, L. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/re6zq · Cited by 3
Classifying Positive Results in Clinical Psychology Using Natural Language Processing. Preprint.Preprint
Schiekiera, L., Niemeyer, H., & Diederichs, J. (2024) · doi:10.31234/osf.io/uxyzh
Mitä toistettavuuskriisistä pitäisi ajatella? Kymmenen suomalaistutkijaa sähköpostihaastattelussa
Heino, M., Ilmarinen, V., Leikas, S., Lipsanen, J. O., Lönnqvist, J., Olkkonen, M., Palosaari, E. J., Renvik, T. A., Saurio, K. J., & Silfver-Kuhalampi, M. (2025) · Psykologia · doi:10.62443/psykologia.v54i6.143375
Open and Reproducible Special Education Research
Kalandadze, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-40858-8_24-1
Metadata
- Authors
- Scheel, A. M., Schijen, M., & Lakens, D.
- Publisher
- Center for Open Science
- Published
- 2020-02-05
- OpenAlex
- W4252093822
- Cited by
- 136
- References
- 0
- Citations by year
- 2025: 4 · 2024: 13 · 2023: 9 · 2022: 55 · 2021: 31 · 2020: 23
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- TU/e Research Portal (submittedVersion, other-oa)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- publication bias, selective reporting, positive results, hypothesis testing, pre-registered study, type I error inflation, psychology literature, peer review
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
