Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency
Open accessPublished version of a preprint
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for badges that accompanied published articles. Before badges, less than 3% of Psychological Science articles reported open data. After badges, 23% reported open data, with an accelerating trend; 39% reported open data in the first half of 2015, an increase of more than an order of magnitude from baseline. There was no change over time in the low rates of data sharing among comparison journals. Moreover, reporting openness does not guarantee openness. When badges were earned, reportedly available data were more likely to be actually available, correct, usable, and complete than when badges were not earned. Open materials also increased to a weaker degree, and there was more variability among comparison journals. Badges are simple, effective signals to promote open practices and improve preservation of data and materials by using independent repositories.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Poor distribution of labor and lack of task specialization may be part of the reason data and code sharing remain rare [48,54]”
From Open Science 2.0: Towards a truly collaborative research ecosystem (2023) · cites this paper as [54]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, simply making data available does not guarantee that the data have utility: Kidwell et al. also observed that a substantial proportion (50/111, 45%) of reportedly available data was in fact not available or was incomplete, incorrect, or had insufficient documentation.”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, in an observational study examining the impact of a voluntary ‘open badges’ scheme at the journal Psychological Science, Kidwell et al. [10] found that the proportion of articles reporting available data increased from around 2.5% during the pre-policy period to 39.5% by the end of the post-policy period.”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reproducibility badges adopted by several journals45 recognize stronger forms of reproducibility without penalizing authors who cannot yet reach them.”
From Designing reproducible large-language-model-assisted scientific analyses (2026) · cites this paper as [45]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They then studied what happened when the journal Psychological Science introduced the open data badge and found a substantial increase in the percentage of publications reporting open data [54].”
From Open Science 2.0: Towards a truly collaborative research ecosystem (2023) · cites this paper as [54]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Spence, M. (1973) · The Quarterly Journal of Economics · doi:10.2307/1882010 · Cited by 19,518
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Signaling Theory: A Review and Assessment
Connelly, B. L., Certo, S. T., Ireland, R. D., & Reutzel, C. R. (2011) · Journal of Management · doi:10.1177/0149206310388419 · Cited by 6,735
The Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N., Merton, R. K., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,005
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Self-citationTHE SOCIOLOGY OF SCIENCE:: Theoretical and Empirical Investigations
Merton, R. Κ., & Storer, N. W. (2017) · Cited by 1,462
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Recommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationThe poor availability of psychological research data for reanalysis.
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.726 · Cited by 543 · cited 1×
“The key problem with “upon request” policies is that requests come after the researcher has completed the research, published the article, and lost or forgot detailed materials and data [9,11,16].”
BackgroundPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citation
Research works printed but not linked: 1
- RK Merton, 1942, Science and technology in a democratic order, J of Legal and Pol Soc, 1, 115
Cited by
Items 201–300 of 736
Hensel, W. M. (2020) · European Journal for Philosophy of Science · doi:10.1007/s13194-020-00317-6 · Cited by 31
“…a materials availability statement but of the materials “reportedly available at a website or repository, 60% (N = 27) were actually available, 46.7% (N = 21) were correct materials, 33.3% (N = 15) were usable materials, and 13.3% (N = 6) were complete materials” (Kidwell et al. 2016, p. 9).”
BackgroundInfluentialTracing the Adoption and Effects of Open Science in Communication Research*
Markowitz, D. M., Song, H., & Taylor, S. H. (2021) · Journal of Communication · doi:10.1093/joc/jqab030 · Cited by 34
Repository Approaches to Improving the Quality of Shared Data and Code
Trišović, A., Mika, K. A., Boyd, C., Feger, S. S., & Crosas, M. (2021) · Data · doi:10.3390/data6020015 · Cited by 28
“Such a result is corroborated by the successful use of the Open Science Badges (OSB)4 that proved to incentivize data sharing for submissions to a medical and health journal [41].”
ResultOpen Access in Special Education: A Review of Journal and Publisher Policies
Fleming, J. I., & Cook, B. G. (2021) · Remedial and Special Education · doi:10.1177/0741932521996461 · Cited by 28
Psychology Exceptionalism and the Multiple Discovery of the Replication Crisis
Nelson, N. C., Chung, J., Ichikawa, K., & Malik, M. M. (2021) · Review of General Psychology · doi:10.1177/10892680211046508 · Cited by 27
“Psychologists have conspicuously and proactively implemented new policies to mitigate irreproducibility and increase rigor (Kidwell et al., 2016; Klein et al., 2014; Leif, 2015; Woolston, 2015), making the discipline the subject of not only negative but also positive attention as scholars look for…”
BackgroundFeger, S. S., Dallmeier-Tiessen, S., Schmidt, A., & Woźniak, P. W. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300685 · Cited by 27
“hat emphasizing “the potential of data citations can affect researchers’ data sharing preferences from private to more open.” And also badges have proven to encourage research sharing. Kidwell et al. [38] compared contributions to the Psychological Science journal, that adopted open science badges, to other journals i…”
BackgroundOpen Science Practices in Communication Sciences and Disorders: A Survey
Amin, M. E., Borders, J. C., Long, H. L., Keller, M. A., & Kearney, E. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2022_jslhr-22-00062 · Cited by 27
Prototyping Usable Privacy and Security Systems: Insights from Experts
Mathis, F., Vaniea, K. E., & Khamis, M. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1949134 · Cited by 29
“, sharing 3D models or circuit diagrams of prototypes), including their accessibility as a criteria for acceptance, or make additions to the reviewing process through, for example, the inclusion of badges (Kay, Haroz, Guha, Dragicevic, & Wacharamanotham, 2017; Kidwell et al., 2016). This would help mitigate Key Challe…”
BackgroundInfluentialThe Science of Technology and Human Behavior: Standards, Old and New
Elson, M., & Przybylski, A. K. (2017) · Journal of Media Psychology Theories Methods and Applications · doi:10.1027/1864-1105/a000212 · Cited by 28
“…the Transparency and Open-ness Promotion Guidelines, the Peer Reviewers’ Openness Initiative, and the Commitment to Research Transparency 3 have been successful in raising awareness of the benefits of open science and increasing the rate of public sharing of datasets (Kidwell et al., 2016).”
BackgroundAnother step towards scientific transparency: Requiring research materials for publication
Grahe, J. E. (2017) · The Journal of Social Psychology · doi:10.1080/00224545.2018.1416272 · Cited by 29
“Kidwell et al. (2016) evaluated rate of earning badges and degree of scientific transparency for articles published in Psychological Science.”
BackgroundResearch data management compliance: is there a bigger role for university libraries?
Shelly, M., & Jackson, M. (2018) · Journal of the Australian Library and Information Association · doi:10.1080/24750158.2018.1536690 · Cited by 30
“Kidwell et al. (2016) showed that of the articles from journals . . . that promised open data, only 40.5 percent actually had data available’ (2017, p. 48).”
BackgroundResultInfluentialEfendić, E., & van Zyl, L. E. (2019) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v45i0.1607 · Cited by 28
“It was found that by the first half of 2015, the proportion of articles stating that data were available increased to around 40%, even when rates in four other psychology journals remained at below 10% (Baker, 2016; Kidwell et al., 2016).”
BackgroundThelwall, M. A., Munafo, M. R., Más-Bleda, A., Stuart, E., Makita, M., Weigert, V., Keene, C., Khan, N., Drax, K., & Kousha, K. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229578 · Cited by 30
“Thus, if more journals required data sharing statements and employed guidelines to ensure that the shared data was described in detail, or provided virtual rewards [54] for these activities, then this would support the level of automated data discovery that would be necessary to monitor data sharing and systematically…”
BackgroundKrahe, M. A., Wolski, M., Mickan, S. M., Toohey, J., Scuffham, P., & Reilly, S. (2020) · Health Information Management Journal · doi:10.1177/1833358320917207 · Cited by 28
“4%), but upon review, data and materials had somehow disappeared from public view (Kidwell et al., 2016).”
BackgroundInfluentialA guide for social science journal editors on easing into open science
Silverstein, P., Elman, C., Montoya, A. K., McGillivray, B., Pennington, C. R., Harrison, C. H., Steltenpohl, C. N., Röer, J. P., Corker, K. S., Charron, L., et al. (2024) · Research Integrity and Peer Review · doi:10.1186/s41073-023-00141-5 · Cited by 29
“Journals can incentivise pre-registration through offering a preregistration badge to articles that meet set criteria [68].”
BackgroundGlöckner, A., Fiedler, S., & Renkewitz, F. (2018) · Psychologische Rundschau · doi:10.1026/0033-3042/a000384 · Cited by 28
Self-citationKwaśnicka, D., ten Hoor, G. A., van Dongen, A., Gruszczyńska, E., Hagger, M. S., Hamilton, K., Hankonen, N. E., Heino, M. T. J., Kotzur, M., Noone, C., et al. (2020) · Health Psychology Review · doi:10.1080/17437199.2020.1844037 · Cited by 25
“Nonetheless, recent research shows that the badges facilitate an increasing rate of data sharing (Kidwell et al., 2016) and promoting new open science norms; however, they are insufficient to permanently change the norms in absence of other incentives (Rowhani-Farid et al., 2020).”
BackgroundLeveraging Containers for Reproducible Psychological Research
Wiebels, K., & Moreau, D. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211017853 · Cited by 27
“Facilitated by the rise of open-source programming languages (R, Python), free repositories (e.g., OSF, GitHub), and incentive journal policies (e.g., open science badges; Kidwell et al., 2016), the field of psychology has made important progress toward reproducible results in recent years.”
BackgroundOpening the Door to Registered Reports: Census of Journals Publishing Registered Reports (2013–2020)
Montoya, A. K., Krenzer, W. L. D., & Fossum, J. L. (2021) · Collabra Psychology · doi:10.1525/collabra.24404 · Cited by 25
“Though most journals, not just those publishing registered reports, require data to be made open and available “upon request,” only 19% of such requests are fulfilled (Stodden et al., 2018; Vines et al., 2014); however, responses to these requests are becoming more common (Kidwell et al., 2016).”
BackgroundMayo‐Wilson, E., Grant, S., & Supplee, L. H. (2021) · Prevention Science · doi:10.1007/s11121-021-01284-x · Cited by 27
“This information could be visualized using badges that acknowledge and signal the use of open practices in intervention evaluations (Kidwell et al., 2016).”
BackgroundHow Will Open Science Impact on University-Industry Collaboration?
Chataway, J., Parks, S., & Smith, E. (2017) · Foresight-Russia · doi:10.17323/2500-2597.2017.2.44.53 · Cited by 31
Assessing and Improving Robustness of Psychological Research Findings in Four Steps
Nuijten, M. B. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_17 · Cited by 28
Promoting transparency in evolutionary biology and ecology
Parker, T. H., Nakagawa, S., Gurevitch, J., & participants, I. (. I. I. E. B. A. E. W. (2016) · Ecology Letters · doi:10.1111/ele.12610 · Cited by 28
Chin, J. M., & Ibaviosa, C. M. (2022) · Science & Justice · doi:10.1016/j.scijus.2022.02.006 · Cited by 29
Perceptions regarding open science appraised by editors of scholarly publications published in Spain
Melero, R., Bóte-Vericad, J.-J., & López‐Borrull, A. (2022) · Learned Publishing · doi:10.1002/leap.1511 · Cited by 26
“The use of badges to certify a journal’s adoption of open science practices, such as editorial policies that require or recommend sharing underlying research data, has also contributed to increasing the proportion of shared data in publications (Kidwell et al., 2016).”
BackgroundSharing GWAS summary statistics results in more citations
Reales, G., & Wallace, C. (2023) · Communications Biology · doi:10.1038/s42003-023-04497-8 · Cited by 26
The availability of raw data in substance abuse scientific journals
Vidal‐Infer, A., Aleixandre‐Benavent, R., Lucas-Domínguez, R., & Sixto-Costoya, A. (2018) · Journal of Substance Use · doi:10.1080/14659891.2018.1489905 · Cited by 23
“Another possible barrier to data exchange is the lack of rewards for researchers who deposit their data (Kidwell et al., 2016).”
BackgroundWhat Constitutes Authorship in the Social Sciences?
Pruschak, G. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.655350 · Cited by 24
“As a case in point, data publishing has gained more and more attention in the recent years (Kidwell et al., 2016).”
BackgroundSamota, E. K., & Davey, R. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.678554 · Cited by 23
“Encouraging and incentivising scientists to conduct robust, transparent, reproducible and replicable research, such as with badges to recognise open practices should be prioritised to help solve the irreproducibility issue (Kidwell et al., 2016).”
BackgroundReckoning With Our Crisis: An Agenda for the Field of Social and Personality Psychology
Schiavone, S. R., & Vazire, S. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221101060 · Cited by 25
“…by tracking the number of articles published in Psychological Science that had (or claimed to have) open data and open materials before and after the journal introduced “badges” that could be earned for open practices (Kidwell et al., 2016; cf. Bastian, 2017; Rowhani-Farid & Barnett, 2020).”
BackgroundKlöcking, M., Wyborn, L., Lehnert, K. A., Ware, B., Prent, A., Profeta, L. R., Kohlmann, F., Noble, W. P., Bruno, I. J., Lambart, S., et al. (2022) · Geochimica et Cosmochimica Acta · doi:10.1016/j.gca.2023.04.024 · Cited by 24
“Options to increase community773 uptake of data sharing practices have been discussed at length in other communities and774 center around a balance between the perceived cost versus benefit of data sharing (e.g.775 Kim and Stanton, 2012; Kidwell et al., 2016).”
BackgroundAn open toolkit for tracking open science partnership implementation and impact
Gold, E. R., Ali‐Khan, S. E., Allen, L., Ballell, L., Barral‐Netto, M., Carr, D. L., Chalaud, D., Chaplin, S., Clancy, M., Clarke, P., et al. (2019) · Gates Open Research · doi:10.12688/gatesopenres.12958.2 · Cited by 24
“…to investigate how organizations implement OS (Lampert et al ., 2017; Nosek et al ., 2015; Smith, 2017; Smith et al ., 2016; Tuomi, 2016) and a few studies have evaluated the implementation or impact of specific OS policies or practices (Hardwicke & Ioannidis, 2018; Kidwell et al ., 2016).”
Self-citationSeven steps toward transparency and replicability in psychological science.
Lindsay, D. S. (2020) · Canadian Psychology/Psychologie canadienne · doi:10.1037/cap0000222 · Cited by 26
“Kidwell et al. (2016) attempted to obtain and reproduce the analyses of data from a (smallish) set of recently published articles that claimed that data were available on request.”
MethodStability and Change in Gender Identity and Sexual Orientation Across Childhood and Adolescence
deMayo, B., Gallagher, N. M., Leshin, R. A., & Olson, K. R. (2025) · Monographs of the Society for Research in Child Development · doi:10.1111/mono.12479 · Cited by 24
Reproducibility and Replicability in SAR Remote Sensing
Balz, T., & Rocca, F. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.3005912 · Cited by 23
“Such badges can help increase the rate of data sharing [27].”
BackgroundAre You Open? A Content Analysis of Transparency and Openness Guidelines in HCI Journals
Ballou, N., Warriar, V. R., & Deterding, S. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445584 · Cited by 23
“The limited existing evidence suggests that awarding badges is associated with increased rates of data sharing [29].”
BackgroundAxford, N., Morpeth, L., Bjornstad, G. J., Hobbs, T., & Berry, V. (2022) · Children and Youth Services Review · doi:10.1016/j.childyouth.2022.106469 · Cited by 23
Gardener, A. D., Hick, E. J., Jacklin, C., Tan, G., Cashin, A. G., Lee, H., Nunan, D., Toomey, E. C., & Richards, G. C. (2022) · JRSM Open · doi:10.1177/20542704221132139 · Cited by 23
“Adopting open science badges may provide a valuable incentive for scientists to practice openly.(37) In a study by Kidwell et al.”
BackgroundEmbracing Open Science and Transparency in Health Psychology
Hagger, M. S. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1605614 · Cited by 22
“…greater means to signal the commitment to open science, such as the introduction of a system of ‘badges’ on articles that demonstrate good practice in open science, such as providing open data, open materials, and pre-registration (Association of Psychological Science, 2019; Kidwell et al., 2016).”
BackgroundCharting the Constellation of Science Reform
Field, S. M. (2022) · University of Groningen research database (University of Groningen / Centre for Information Technology) · doi:10.33612/diss.229114775 · Cited by 28
Questionable Research Practices
O’Boyle, E. H., & Götz, M. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780190938550.003.0010 · Cited by 28
Zong, Q., Huang, Z., & Huang, J. (2023) · Scientometrics · doi:10.1007/s11192-023-04720-7 · Cited by 22
“Starting in January 2014, Psychological Science allows authors to earn up to three open science badges if they qualify for badges that accompany published articles (Eich, 2014; Kidwell et al., 2016).”
BackgroundCatalá-López, F., Ridao-López, M., Tejedor-Romero, L., Caulley, L., Hutton, B., Husereau, D., Alonso‐Arroyo, A., Bernal‐Delgado, E., Drummond, M., & Moher, D. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.10.024 · Cited by 21
Researcher Degrees of Freedom in the Psychology of Religion
Charles, S. J., Bartlett, J. E., Messick, K. J., Coleman, T. J., III, & Uzdavines, A. (2019) · International Journal for the Psychology of Religion · doi:10.1080/10508619.2019.1660573 · Cited by 21
“Offering badges has been shown to improve the adoption of Open Science practices (Kidwell et al., 2016; Rowhani-Farid, Allen, & Barnett, 2017).”
BackgroundAssigning credit to scientific datasets using article citation networks
Zeng, T., Wu, L., Bratt, S., & Acuna, D. E. (2020) · Journal of Informetrics · doi:10.1016/j.joi.2020.101013 · Cited by 23
“Scientists may be incentivized to adopt better and broader data sharing behaviors if they, their peers, and institutions are able to measure the impact of datasets (e.g., see Silvello (2018) and Kidwell et al. (2016)).”
BackgroundAdvancing the Cognitive Science of Religion through Replication and Open Science
Hoogeveen, S., & van Elk, M. (2020) · Journal for the Cognitive Science of Religion · doi:10.1558/jcsr.39039 · Cited by 21
Sex differences in brain and behavioral development
Beltz, A. M., Kelly, D. P., & Berenbaum, S. A. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-814411-4.00027-5 · Cited by 29
Torka, A., Mazei, J., Bosco, F. A., Cortina, J. M., Götz, M., Kepes, S., O’Boyle, E. H., & Hüffmeier, J. (2023) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2023.2206571 · Cited by 20
“Research investigating the example of Psychological Science as a big-tent psychological flagship journal that has led the way towards a stronger implementation of OSPs (Kidwell et al., 2016) already showed that enabling Open Science Badges can increase data and material sharing.”
BackgroundSynthesizing connectivity information from migratory marine species for area-based management
Kot, C. Y., DeLand, S. E., Harrison, A., Alberini, A., Blondin, H. E., Chory, M., Cleary, J., Curtice, C., Donnelly, B., Fujioka, E., et al. (2023) · Biological Conservation · doi:10.1016/j.biocon.2023.110142 · Cited by 25
What meta‐research has taught us about research and changes to research practices
Ioannidis, J. P. A. (2024) · Journal of Economic Surveys · doi:10.1111/joes.12666 · Cited by 25
Syed, M. (2025) · Journal of Trial and Error · doi:10.36850/e5ce-4cc5 · Cited by 20
Cao, C. C., & Reimann, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.550204 · Cited by 21
“…important discussion regarding how to promote openness and transparency of research in the fields of psychology and neuroscience (e.g., Aarts et al., 2015; Gorgolewski and Poldrack, 2016; Kidwell et al., 2016; Nichols et al., 2016; Gilmore et al., 2017; Klein et al., 2018; Shrout and Rodgers, 2018).”
Verifiable Badging System for scientific data reproducibility
Radha, S. K., Taylor, I., Nabrzyski, J., & Barclay, I. (2021) · Blockchain Research and Applications · doi:10.1016/j.bcra.2021.100015 · Cited by 20
Open Practices in Visualization Research : Opinion Paper
Haroz, S. (2018) · IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV) · doi:10.1109/beliv.2018.8634427 · Cited by 20
“Early evidence shows that adopting the badges substantially increases open practices among papers [7], as a journal that implemented the open data badge saw substantial increase in data sharing compared with other journals in the same field.”
BackgroundPerspectives on Machine Learning from Psychology's Reproducibility CrisisPreprint
Bell, S. J., & Kampman, O. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08878 · Cited by 20
“One journal reports that badges led to a rise from 3% to 39% in papers making their data publicly-available (Kidwell et al., 2016).”
BackgroundThe AI-DEC: A Card-based Design Method for User-centered AI Explanations
Lee, C. P., Lee, M. K., & Mutlu, B. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661576 · Cited by 22
Renaut, S., Budden, A. E., Gravel, D., Poisot, T., & Peres‐Neto, P. R. (2018) · BioScience · doi:10.1093/biosci/biy038 · Cited by 19
Open Data Policies among Library and Information Science Journals
Jackson, B. (2021) · Publications · doi:10.3390/publications9020025 · Cited by 21
“These effects may be strengthened with the promotion of other incentives for publishing data, such as increased citations [17,23], badges [24], and awards [25].”
BackgroundSharing is caring: Ethical implications of transparent research in psychology.
Bosma, C. M., & Granger, A. M. (2022) · American Psychologist · doi:10.1037/amp0001002 · Cited by 24
Open Access, Scholarly Communication, and Open Science in Psychology: An Overview for Researchers
Mullen, L. B. (2024) · SAGE Open · doi:10.1177/21582440231205390 · Cited by 23
Sharing, collaborating, and building trust: How Open Science advances language testing
Winke, P. M. (2024) · Language Testing · doi:10.1177/02655322231211159 · Cited by 19
“…Preregistration, Open Materials, Open Data, or Open Code, participating journals will recognize those efforts by literally placing an electronic badge representing the effort on the article’s front page, like a teacher putting a gold star on the top of your paper (see Kidwell et al., 2016).”
Should Psychology Journals Adopt Specialized Statistical Review?
Hardwicke, T. E., Frank, M. C., Vazire, S., & Goodman, S. N. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919858428 · Cited by 23
“At some psychology journals, practices such as data sharing are already being actively encouraged (e.g., Psychological Science; Kidwell et al., 2016) or mandated (e.”
BackgroundSelf-citationDigital badges motivate scientists to share data
Baker, M. (2016) · Nature · doi:10.1038/nature.2016.19907 · Cited by 18
Bonn, N. A., & Bouter, L. M. (2021) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/82rmj · Cited by 20
“Th e Open Science badges — registration, open data, open materials — are a good example of a simple change that allows readers or eventually assessors to quickly capture open science practices behind published works [ 33 ].”
BackgroundThe replication crisis has led to positive structural, procedural, and community changesPreprint
Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M. M., Breznau, N., Robertson, O. M., Kalandadze, T., et al. (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/r6cvx · Cited by 23
Aspiroz, L. S., Mennecozzi, M., Batlle‐Morera, L., Corneo, B., Healy, L. E., Kotter, M., Kurtz, A., Ludwig, T. E., Mummery, C. L., Pera, M. F., et al. (2025) · Stem Cell Reports · doi:10.1016/j.stemcr.2025.102531 · Cited by 19
Tunç, D. U., Tunç, M. N., & Eper, Z. B. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221114835 · Cited by 21
Hallinan, D., Boehm, F., Külpmann, A. I., & Elson, M. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231151944 · Cited by 17
DataPipe: Born-open data collection for online experiments
de Leeuw, J. R. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02161-x · Cited by 17
“…have adopted data sharing policies that require open data (Nosek et al., 2015)—though the enforcement of and follow-through on such policies is sometimes lax (Alsheikh-Ali et al., 2011; Hussey, 2023)—and/or recognition for articles that elect to include open data (e.g., Kidwell et al., 2016).”
Bonn, N. A., & Bouter, L. M. (2023) · Collaborative bioethics · doi:10.1007/978-3-031-29455-6_27 · Cited by 23
Cumulative Science via Bayesian Posterior Passing
Brand, C. O., Ounsley, J., van der Post, D. J., & Morgan, T. J. H. (2019) · Meta-Psychology · doi:10.15626/mp.2017.840 · Cited by 17
Rigor and reproducibility for data analysis and design in the behavioral sciences
Hildebrandt, T., & Prenoveau, J. M. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103552 · Cited by 19
Replicability in Empirical Legal Research
Chin, J. M., & Zeiler, K. (2021) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-121620-085055 · Cited by 21
Open Science in regulatory environmental risk assessment
Brock, T. C. M., Elliott, K. C., Gladbach, A., Moermond, C., Romeis, J., Seiler, T., Solomon, K. R., & Dohmen, G. P. (2021) · Integrated Environmental Assessment and Management · doi:10.1002/ieam.4433 · Cited by 18
“Thus, relevance encompasses the extent to which data, tools, and tests are appropriate to the problem formulation for a particular hazard identification or risk characterization (Klimisch et al., 1997; Rudén et al., 2017).”
BackgroundMethodDoing reliable research in comparative psychology: Challenges and proposals for improvement.
Tecwyn, E. C. (2021) · Journal of comparative psychology · doi:10.1037/com0000291 · Cited by 20
Tiller, N. B., & Ekkekakis, P. (2023) · Kinesiology Review · doi:10.1123/kr.2022-0039 · Cited by 19
The necessity of data transparency to publish
Grahe, J. E. (2020) · The Journal of Social Psychology · doi:10.1080/00224545.2020.1847950 · Cited by 17
Mosha, N. F., & Ngulube, P. (2023) · Library Management · doi:10.1108/lm-05-2023-0042 · Cited by 16
Evaluating the Pedagogical Effectiveness of Study Preregistration in the Undergraduate Dissertation
Pownall, M., Pennington, C. R., Norris, E., Juanchich, M., Smailes, D., Russell, P. S., Gooch, D., Evans, T. R., Persson, S., Mak, M. H., et al. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231202724 · Cited by 19
“…much work has considered how to promote uptake of preregistration practices of early career (Zecˇevic´ et al., 2020) and more established researchers (Kidwell et al., 2016; Munafò et al., 2017), little research has explicitly focused on the utility of preregistration for undergraduate students’…”
BackgroundREES: A Registry of Efficacy and Effectiveness Studies in Education
Anderson, D., Spybrook, J. K., & Maynard, R. A. (2018) · Educational Researcher · doi:10.3102/0013189x18810513 · Cited by 16
Weißgerber, S. C., Brunmair, M., & Rummer, R. (2021) · Educational Psychology Review · doi:10.1007/s10648-020-09579-1 · Cited by 17
One finding is no finding: Toward a replication culture in family business research
Brinkerink, J., Massis, A. D., & Kellermanns, F. W. (2022) · Journal of Family Business Strategy · doi:10.1016/j.jfbs.2022.100521 · Cited by 18
Reproducibility in Small- N Treatment Research: A Tutorial Using Examples From Aphasiology
Cavanaugh, R., Quique, Y. M., Swiderski, A. M., Kallhoff, L., Terhorst, L. A., Wambaugh, J. L., Hula, W. D., & Evans, W. S. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2022_jslhr-22-00333 · Cited by 16
Lubega, N., Anderson, A., & Nelson, N. C. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293584 · Cited by 16
Reflection over compliance: Critiquing mandatory data sharing policies for qualitative research
Prosser, A. M. B., Bagnall, R., & Higson‐Sweeney, N. (2024) · Journal of Health Psychology · doi:10.1177/13591053231225903 · Cited by 19
Publishing Open, Reproducible Research With Undergraduates
Strand, J. F., & Brown, V. A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00564 · Cited by 17
Jussim, L. J., Krosnick, J. A., Stevens, S. T., & Anglin, S. M. (2019) · Psychologica Belgica · doi:10.5334/pb.496 · Cited by 16
Buckley, P. R., Ebersole, C. R., Steeger, C. M., Michaelson, L., Hill, K. G., & Gardner, F. E. M. (2021) · Prevention Science · doi:10.1007/s11121-021-01252-5 · Cited by 18
Hensel, P. G. (2021) · Accountability in Research · doi:10.1080/08989621.2021.1981870 · Cited by 17
Maximizing the Reproducibility of Your ResearchPreprint
Bosco, F. A., Carp, J., Field, J. G., IJzerman, H. R., Lewis, M., Munafo, M. R., Nosek, B. A., Prenoveau, J. M., Spies, J. R., & Giner‐Sorolla, R. (2016) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/k9mn3_v1 · Cited by 17
Self-citationPsychology’s Replication Crisis and Clinical Psychological SciencePreprint
Tackett, J. L., Brandes, C. M., King, K. M., & Markon, K. E. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/kc8xt · Cited by 16
Norris, E., Clark, K., Munafo, M. R., Jay, C., Baldwin, J. R., Lautarescu, A., Pedder, H., Page, M. P. A., Rinke, E. M., Burn, C. C., et al. (2022) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/w48yh · Cited by 18
Lakens, D. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dtvs7 · Cited by 17
Analytical code sharing practices in biomedical research
Sharma, N. K., Ayyala, R., Deshpande, D., Patel, Y. M., Munteanu, V., Ciorbă, D., Bostan, V., Fiscutean, A., Vahed, M., Sarkar, A. P., et al. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2066 · Cited by 17
Skubera, M., Korbmacher, M., Evans, T. R., Azevedo, F., & Pennington, C. R. (2025) · Royal Society Open Science · doi:10.1098/rsos.241726 · Cited by 18
“Incentives to increase the adoption of open research practices have been developed, such as Open Science Badges [154,155] that acknowledge when preregistration, open materials and open data have been implemented.”
Making Research Data Accessible
Kapiszewski, D., & Karcher, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108762519.008 · Cited by 16
Maximizing the Reproducibility of Your Research
Open Science Collaboration (2017) · Wiley eBooks · doi:10.1002/9781119095910.ch1 · Cited by 17
Osborne, C., & Norris, E. (2022) · Cogent Psychology · doi:10.1080/23311908.2022.2066304 · Cited by 15
“For example, some journals incentivise researchers to pre-register with a pre-registration badge, demonstrating their commitment to open research practices (Kidwell et al., 2016).”
BackgroundTracing the Adoption and Effects of Open Science in Communication ResearchPreprint
Markowitz, D. M., Song, H., & Taylor, S. H. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dsf67 · Cited by 16
Assessing Open Science practices in physical activity behaviour change intervention evaluations
Norris, E., Sulevani, I., Finnerty, A. N., & Castro, O. (2022) · BMJ Open Sport & Exercise Medicine · doi:10.1136/bmjsem-2021-001282 · Cited by 14
“adopted by journals as a simple, low-cost scheme to reward these research behaviours (Kidwell et al., 2016; Rowhani-Farid & Barnett, 2018).”
BackgroundMethod
Metadata
- Authors
- Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A.
- Affiliations
- Center for Open Science (US)
- University of Belgrade (RS)
- University of California, Riverside (US)
- University College London (GB)
- Max Planck Institute for Behavioral Economics (DE)
- Mississippi State University (US)
- University of Vienna (AT)
- Corresponding author
- Kidwell, Mallory C.
- ORCID
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-05-12
- Volume
- 14
- Issue
- 5
- Pages
- e1002456
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC4865119
- OpenAlex
- W2390706404
- Cited by
- 700
- References
- 19
- Citations by year
- 2026: 31 · 2025: 35 · 2024: 39 · 2023: 71 · 2022: 110 · 2021: 87 · 2020: 98 · 2019: 80 · 2018: 70 · 2017: 54 · 2016: 25
- FWCI
- 170.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- WU Research (publishedVersion)
- Phaidra (Universität Wien) (submittedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Faculty of Philosophy (University of Belgrade) (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- Funders
- The Laura and John Arnold Foundation
- The John Templeton Foundation
- National Institute on Aging
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open badges, open data, open practices, research transparency, data sharing, open materials, data preservation, scholarly publishing, author incentives
- MeSH terms
- Cost-Benefit Analysis, Psychology, Publishing, Serial Publications, Internet, Information Dissemination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.