Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency
Open accessPublished version of a preprint
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for badges that accompanied published articles. Before badges, less than 3% of Psychological Science articles reported open data. After badges, 23% reported open data, with an accelerating trend; 39% reported open data in the first half of 2015, an increase of more than an order of magnitude from baseline. There was no change over time in the low rates of data sharing among comparison journals. Moreover, reporting openness does not guarantee openness. When badges were earned, reportedly available data were more likely to be actually available, correct, usable, and complete than when badges were not earned. Open materials also increased to a weaker degree, and there was more variability among comparison journals. Badges are simple, effective signals to promote open practices and improve preservation of data and materials by using independent repositories.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Poor distribution of labor and lack of task specialization may be part of the reason data and code sharing remain rare [48,54]”
From Open Science 2.0: Towards a truly collaborative research ecosystem (2023) · cites this paper as [54]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, simply making data available does not guarantee that the data have utility: Kidwell et al. also observed that a substantial proportion (50/111, 45%) of reportedly available data was in fact not available or was incomplete, incorrect, or had insufficient documentation.”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, in an observational study examining the impact of a voluntary ‘open badges’ scheme at the journal Psychological Science, Kidwell et al. [10] found that the proportion of articles reporting available data increased from around 2.5% during the pre-policy period to 39.5% by the end of the post-policy period.”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reproducibility badges adopted by several journals45 recognize stronger forms of reproducibility without penalizing authors who cannot yet reach them.”
From Designing reproducible large-language-model-assisted scientific analyses (2026) · cites this paper as [45]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They then studied what happened when the journal Psychological Science introduced the open data badge and found a substantial increase in the percentage of publications reporting open data [54].”
From Open Science 2.0: Towards a truly collaborative research ecosystem (2023) · cites this paper as [54]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Spence, M. (1973) · The Quarterly Journal of Economics · doi:10.2307/1882010 · Cited by 19,518
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Signaling Theory: A Review and Assessment
Connelly, B. L., Certo, S. T., Ireland, R. D., & Reutzel, C. R. (2011) · Journal of Management · doi:10.1177/0149206310388419 · Cited by 6,735
The Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N., Merton, R. K., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,005
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Self-citationTHE SOCIOLOGY OF SCIENCE:: Theoretical and Empirical Investigations
Merton, R. Κ., & Storer, N. W. (2017) · Cited by 1,462
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Recommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationThe poor availability of psychological research data for reanalysis.
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.726 · Cited by 543 · cited 1×
“The key problem with “upon request” policies is that requests come after the researcher has completed the research, published the article, and lost or forgot detailed materials and data [9,11,16].”
BackgroundPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citation
Research works printed but not linked: 1
- RK Merton, 1942, Science and technology in a democratic order, J of Legal and Pol Soc, 1, 115
Cited by
Items 701–736 of 736
Digital tools and services to support research replicability and verifiability
(2018) · JISC Information Environment Repository (Jisc)
Claesen, A. (2024) · Lirias
Open Science by Design: Realizing a Vision for 21st Century Research
(2018) · bepress Legal Repository · Cited by 47
More Than Preservation: A Researcher-Centered Approach to Reproducibility in Data Science
Feger, S. S., & Woźniak, P. W. (2019) · CERN Document Server (European Organization for Nuclear Research)
ラジカルランダマイゼーションを用いたフラッシュ顔変形効果の一般化可能性の探索
益田, 佳. (2023) · Institutional Repositories DataBase (IRDB)
[PhD Dissertation] Good Research PracticesPreprint
Sarafoglou, A. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7aj5h
Vier, C. M. (2019) · K-State Research Exchange (Kansas State University)
A Unified Framework to Quantify the Credibility of Scientific FindingsPreprint
McCarthy, R. J., LeBel, E. P., Elson, M., vanpaemel, W., & Earp, B. D. (2018) · OSF Preprints (OSF Preprints)
Advancing openness in economic research through the lens of behavioral and experimental economics
Rahal, R.-M. (2025) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2025.102454
Open-science practices in two communication sciences and disorders journals: a preliminary review
Hess, A. E., Cook, B. G., Pfeiffer, D. L., Ford, A. L. B., & Reed, A. C. (2025) · Evidence-Based Communication Assessment and Intervention · doi:10.1080/17489539.2025.2579798
“Kidwell et al. (2016) (Michie et al., 2011), open-science badges can enhance motivation and therefore promote greater engagement in open-science practices.”
Open science indicators as metadata fields ?
Mugnaini, R. (2023) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Assessing Questionable and Responsible Research Practices in Psychology Master’s Theses
Augusteijn, H. E. M., van Assen, M. A. L. M., Sijtsma, K., & Wicherts, J. M. (2026) · Behavioral Sciences · doi:10.3390/bs16010110 · Cited by 1
“Many initiatives have been launched to counter QRPs and to improve the quality and transparency of research, such as preregistration and data sharing (Kidwell et al., 2016; Miguel et al., 2014).”
How open was accounting research in 2025? An analysis of open science practices in leading journals
Derks, K. (2026) · Accounting Open · doi:10.1016/j.accop.2026.100012
The Challenges and Solutions of In‐Person Big‐Team Science
Buchanan, E. M., Cavagnaro, D. R., Premachandra, B., Wu, D. J., Lewis, N., Taylor, V. J., Urry, H. L., & Forscher, P. S. (2026) · Social and Personality Psychology Compass · doi:10.1111/spc3.70121
Investigating Data Sharing Practices in Teaching Effectiveness Research
Sajjad, T., Hartig, J., Lösch, T., & Köhler, C. (2026) · AERA Open · doi:10.1177/23328584251409658
Measurement-Reporting Practices in Social-and-Personality-Psychology Articles
Lawson, K. M., Bottesini, J. G., Khong, L. D., & Vazire, S. (2026) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251407612 · Cited by 1
“This relatively low bar for articles being deemed as having “open materials” means that we likely overestimate the proportion of articles with actually available, correct, usable, and complete materials (Hardwicke et al., 2024; Kidwell et al., 2016).”
Kou, Y., Zhou, Q., Wang, J., & Yu, N. X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791151 · Cited by 1
“Open data referred to making publicly available the digitally shareable data necessary to reproduce the reported results [49].”
Extent of Open Science Practices in the Reporting of Real World Evidence Research
Liu, L., Jorm, L. R., Kim, N., Honeyman, T., & Vajdic, C. M. (2026) · International Journal for Population Data Science · doi:10.23889/ijpds.v10i2.2960
“For instance, open data badges used to recognise data sharing have been demonstrated to increase data sharing within some scientific disciplines [34, 35].”
Kreusch, R., Groß, C. S., & Saal, S. (2026) · Gait & Posture · doi:10.1016/j.gaitpost.2026.110176
Sánchez‐Tarragó, N., & Oliveira, T. D. A. (2025) · Dialnet (Universidad de la Rioja)
Psychology’s RenaissancePreprint
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2022) · OSF Preprints (OSF Preprints)
Open Science is Robust SciencePreprint
Zickar, M. J., Grubbs, J. B., & McAbee, S. T. (2017) · OSF Preprints (OSF Preprints)
From superficial scholarship to algorithmic slop in analytical chromatography
Salehi‐Reyhani, A. (2026) · Journal of Chromatography B · doi:10.1016/j.jchromb.2026.125130
The Call for Open Science in Modeling
Ringuette, R., Schmidt, G. A., Jin, M., Kubaryk, A., Erdmann, C. C., Hengeveld, G., Caplan, R. M., Hartnett, M., & Carver, J. C. (2026) · F1000Research · doi:10.12688/f1000research.183312.1
Leveraging Social Networks for Knowledge Exchange in the Era of Emerging Technologies
Nanda, P. K. (2026) · Springer eBooks · doi:10.1007/978-981-95-8974-6_5
Psychoanalysis in the Midst of Psychology’s Crisis
Wendler, H., & Storck, T. (2026) · Springer eBooks · doi:10.1007/978-3-032-01453-5_16
Endacott, C. G., Reed, B., & Steltenpohl, C. N. (2026) · The Leadership Quarterly · doi:10.1016/j.leaqua.2026.101977
Kozula, M., den Eynden, V. V., DeVito, N., Ross‐Hellauer, T., Kormann, E., Leeflang, M., & Onghena, P. (2026) · Royal Society Open Science · doi:10.1098/rsos.252472
Declaring the machine: A disclosure policy on artificial intelligence for authors and reviewers
Iliescu, D., & Greiff, S. (2026) · Intelligence · doi:10.1016/j.intell.2026.102060
Survey of Open Science Practices in Behavior Analysis
Jennings, A. M., Breeman, S. L., Vladescu, J. C., & Cox, D. J. (2026) · Behavior Analysis in Practice · doi:10.1007/s40617-026-01235-4
A Perspective on Badges, Including Open Science and Reproducibility Badges, in Academic PublishingPreprint
da Silva, J. A. T. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7441243
Lewis, S. C., Trenge, C., & Tullett, A. M. (2026) · Teaching of Psychology · doi:10.1177/00986283261487196
O Fórum SOL de Ciência Aberta e o Desafio da Reprodutibilidade
Viterbo, J., Araujo, A., & Kon, F. (2026) · Computação Brasil. · doi:10.5753/compbr.2026.57.9305
Research Data-Sharing Practices Through the Lens of Data Availability Statements
TSUNODA, H., Sun, Y., Nishizawa, M., Liu, X., Amano, K., & Kawamura, S. (2026) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3805696.3846017
“As DASs have become common, they have emerged as a valuable source for examining research transparency at scale, since data sharing, materials availability, and other reproducibility-related behaviours can be measured systematically from structured reporting elements [15, 22].”
The evolution of data sharing practices in the psychological literature
Neve, J. A., & Rousselet, G. A. (2024) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam)
Metadata
- Authors
- Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A.
- Affiliations
- Center for Open Science (US)
- University of Belgrade (RS)
- University of California, Riverside (US)
- University College London (GB)
- Max Planck Institute for Behavioral Economics (DE)
- Mississippi State University (US)
- University of Vienna (AT)
- Corresponding author
- Kidwell, Mallory C.
- ORCID
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-05-12
- Volume
- 14
- Issue
- 5
- Pages
- e1002456
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC4865119
- OpenAlex
- W2390706404
- Cited by
- 700
- References
- 19
- Citations by year
- 2026: 31 · 2025: 35 · 2024: 39 · 2023: 71 · 2022: 110 · 2021: 87 · 2020: 98 · 2019: 80 · 2018: 70 · 2017: 54 · 2016: 25
- FWCI
- 170.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- WU Research (publishedVersion)
- Phaidra (Universität Wien) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Faculty of Philosophy (University of Belgrade) (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- Funders
- The Laura and John Arnold Foundation
- The John Templeton Foundation
- National Institute on Aging
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open badges, open data, open practices, research transparency, data sharing, open materials, data preservation, scholarly publishing, author incentives
- MeSH terms
- Cost-Benefit Analysis, Psychology, Publishing, Serial Publications, Internet, Information Dissemination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.