The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
The Peer Reviewers' Openness Initiative: incentivizing open research practices through peer review
Open accessAbstract
Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Sharing Detailed Research Data Is Associated with Increased Citation Rate
Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000308 · Cited by 881
The poor availability of psychological research data for reanalysis.
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.726 · Cited by 543
Sweave: Dynamic Generation of Statistical Reports Using Literate Data Analysis
Leisch, F. (2002) · Compstat · doi:10.1007/978-3-642-57489-4_89 · Cited by 422
Empirical Study of Data Sharing by Authors Publishing in PLoS Journals
Savage, C. J., & Vickers, A. J. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007078 · Cited by 411 · cited 1×
“For decades, authors have found low rates of data sharing, across many scientific disciplines [8–15].”
BackgroundReproducible Epidemiologic Research
Peng, R. D., Dominici, F., & Zeger, S. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj093 · Cited by 292
Wolins, L. (1962) · American Psychologist · doi:10.1037/h0038819 · Cited by 226
Normative Dissonance in Science: Results from a National Survey of U.S. Scientists
Anderson, M. S., Martinson, B. C., & De Vries, R. (2007) · Journal of Empirical Research on Human Research Ethics · doi:10.1525/jer.2007.2.4.3 · Cited by 208
A Note on Science and Democracy?: Robert K. Mertons Ethos of Science
Menke, C. (2015) · Ethos und Pathos der Geisteswissenschaften · doi:10.1515/9783110375008-013 · Cited by 199
Cited by
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Self-citationEquivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 3,001
“Complete transparency (sharing all materials) is a partial solution since it allows peers to evaluate whether the experiment was well designed (Morey et al., 2016), but this issue is not easily resolved when the actions of an experimenter might influence the data.”
Self-citationIncreasing Transparency Through a Multiverse Analysis
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616658637 · Cited by 1,372
Self-citationPrzybylski, A. K., & Weinstein, N. (2017) · Psychological Science · doi:10.1177/0956797616678438 · Cited by 890
Detecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436
Self-citationData sharing practices and data availability upon request differ across scientific disciplines
Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä., Pedaste, M., Raju, M., Astapova, A., Lukner, H., et al. (2021) · Scientific Data · doi:10.1038/s41597-021-00981-0 · Cited by 387
The limitations to our understanding of peer review
Tennant, J. P., & Ross-Hellauer, T. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00092-1 · Cited by 384
Internet Gaming Disorder: Investigating the Clinical Relevance of a New Phenomenon
Przybylski, A. K., Weinstein, N., & Murayama, K. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.16020224 · Cited by 372
Effects of tDCS on motor learning and memory formation: A consensus and critical position paper
Buch, E. R., Santarnecchi, E., Antal, A., Born, J., Celnik, P. A., Classen, J., Gerloff, C., Hallett, M., Hummel, F. C., Nitsche, M. A., et al. (2017) · Clinical Neurophysiology · doi:10.1016/j.clinph.2017.01.004 · Cited by 367
A weak scientific basis for gaming disorder: Let us err on the side of caution
van Rooij, A. J., Ferguson, C. J., Colder Carras, M., Kardefelt-Winther, D., Shi, J., Aarseth, E., Bean, A. M., Bergmark, K. H., Brus, A., Coulson, M., et al. (2018) · Journal of Behavioral Addictions · doi:10.1556/2006.7.2018.19 · Cited by 346
Self-citationInhibition in aging: What is preserved? What declines? A meta-analysis
Rey-Mermet, A., & Gade, M. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1384-7 · Cited by 296
Swedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
A multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J. P., Dugan, J. M., Graziotin, D., Jacques, D. C., Waldner, F., Mietchen, D., Elkhatib, Y., B. Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.1 · Cited by 289
Conspiracy Theories and the People Who Believe Them
(2018) · doi:10.1093/oso/9780190844073.001.0001 · Cited by 284
Data Sharing in Psychology: A Survey on Barriers and Preconditions
Houtkoop, B. L., Chambers, C., Macleod, M., Bishop, D. V. M., Nichols, T. E., & Wagenmakers, E.-J. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917751886 · Cited by 283
Self-citationHardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020) · Royal Society Open Science · doi:10.1098/rsos.190806 · Cited by 217
Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620979806 · Cited by 187
Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm
Vanpaemel, W., Vermorgen, M., Deriemaecker, L., & Storms, G. (2015) · Collabra · doi:10.1525/collabra.13 · Cited by 152
Self-citationReproducible and replicable pain research: a critical review
Lee, H., Lamb, S. E., Bagg, M. K., Toomey, E., Cashin, A. G., & Moseley, G. L. (2018) · Pain · doi:10.1097/j.pain.0000000000001254 · Cited by 150
Aspiring to greater intellectual humility in science
Hoekstra, R., & Vazire, S. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01203-8 · Cited by 117
Self-citationQuality Uncertainty Erodes Trust in Science
Vazire, S. (2017) · Collabra: Psychology · doi:10.1525/collabra.74 · Cited by 106
“In the scientific market, peer reviewers are the mechanics, and the rebellion is starting (see the Peer Reviewer’s Openness Initiative, opennessinitiative. org; Morey et al., 2016).”
BackgroundCalibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98
Hardwicke, T. E., & Ioannidis, J. P. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201856 · Cited by 78
Aligning physiology with psychology: Translational neuroscience in neuropsychiatric drug discovery
McArthur, R. A. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.004 · Cited by 42
Mischievous responding in Internet Gaming Disorder research
Przybylski, A. K. (2016) · PeerJ · doi:10.7717/peerj.2401 · Cited by 36
“Given the high reputational and clinical stakes for this research area, studies on IGDmust use both open data and materials (Elson, Przybylski & Krämer, 2015; Morey et al., 2016).”
BackgroundRiley, S., Brooks, J., Goodman, S., Cahill, S., Branney, P., Treharne, G. J., & Sullivan, C. (2019) · Qualitative Research in Psychology · doi:10.1080/14780887.2019.1605275 · Cited by 31
Internet Filtering Technology and Aversive Online Experiences in Adolescents
Przybylski, A. K., & Nash, V. (2017) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2017.01.063 · Cited by 11
Open Science and Transparency in Applied Linguistics Research
Marsden, E. (2019) · The Encyclopedia of Applied Linguistics · doi:10.1002/9781405198431.wbeal1493 · Cited by 6
Metadata
- Authors
- Morey, R. D., Chambers, C. D., Etchells, P. J., Harris, C. R., Hoekstra, R., Lakens, D., Lewandowsky, S., Morey, C. C., Newman, D. P., Schönbrodt, F. D., Vanpaemel, W., Wagenmakers, E.-J., & Zwaan, R. A.
- Venue
- Royal Society Open Science
- Publisher
- The Royal Society
- Published
- 2016-01
- Volume
- 3
- Issue
- 1
- Pages
- 150547
- ISSN
- 2054-5703 (ISSN-L)
- OpenAlex
- W2235693693
- Cited by
- 284
- References
- 24
- FWCI
- 82.22
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.