Promoting an open research culture
Open accessPublished version of a preprint
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Author guidelines for journals could help to promote transparency, openness, and reproducibility
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, many are closed source applications with black box algorithms, which is problematic as transparency and data ownership are essential in the era of open science 22 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“Open science will clearly help in this regard (Nosek et al., 2015).”
From Intrusive memories of trauma: A target for research bridging cognitive science and its clinical application (2019) · cites this paper as Nosek et al., 2015
In the citing paper
“These issues may be strongly suspected and their magnitude can sometimes be estimated 63 , but cannot truly be corrected 64,65 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Self-citationRecommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationRegistered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Self-citationPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citationPublication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Self-citation
Cited by
Items 201–300 of 1,016
Boonstra, T. W., Nicholas, J., Wong, Q. J. J., Shaw, F., Townsend, S., & Christensen, H. M. (2018) · Journal of Medical Internet Research · doi:10.2196/10131 · Cited by 121
“Finally, the data itself should be made available on open data repositories to enable data aggregation across studies to test potential markers in larger samples, resulting in more reproducible results [60].”
BackgroundThe need for robust characterization of nanomaterials for nanomedicine applications
Mahmoudi, M. (2021) · Nature Communications · doi:10.1038/s41467-021-25584-6 · Cited by 112
Newbury, C. R., Crowley, R., Rastle, K., & Tamminen, J. (2021) · Psychological Bulletin · doi:10.1037/bul0000348 · Cited by 136
“It has been suggested that psychological science more broadly is currently suffering from a replication crisis due to low power, publication bias, selection biases, and analysis errors (Nosek et al., 2015).”
BackgroundQuality Uncertainty Erodes Trust in Science
Vazire, S. (2017) · Collabra: Psychology · doi:10.1525/collabra.74 · Cited by 106
“To change this, we must tie journals’ reputations to the actual quality of their articles instead (Fraley & Vazire, 2014), and to their policies regarding transparency and openness (e.g., using the Transparency and Openness Promotion (TOP) guidelines; Nosek et al., 2015).”
Self-citationLimitations and Misinterpretations of E-Values for Sensitivity Analyses of Observational Studies
Ioannidis, J. P. A., Tan, Y. J., & Blum, M. R. (2018) · Annals of Internal Medicine · doi:10.7326/m18-2159 · Cited by 127
“These additional biases are currently impossible to tackle for most epidemiologic publications without endorsement of open science, data sharing, and reproducible research practices (21).”
BackgroundAn open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., Bruin, J. D., Schram, R. D., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B. M. R., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Utrecht University Repository (Utrecht University) · Cited by 106
Methodology over metrics: current scientific standards are a disservice to patients and society
Van Calster, B., Wynants, L., Riley, R. D., van Smeden, M., & Collins, G. S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.018 · Cited by 105
“io/ our-services/ top-guidelines ) [48] .”
Cognitive Abilities Explain Wording Effects in the Rosenberg Self-Esteem Scale
Gnambs, T., & Schroeders, U. (2020) · Assessment · doi:10.1177/1073191117746503 · Cited by 105
“We also provide all R scripts (R Core Team, 2017) in an online repository of the Open Science Framework (https:// osf.io/bkzjy) to make the present analyses as transparent and reproducible as possible (Nosek et al., 2015).”
MethodBest Practice Recommendations for Replicating Experiments in Public Administration
Walker, R. M., Brewer, G. A., Lee, M. J., Petrovsky, N., & van Witteloostuijn, A. C. (2018) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muy047 · Cited by 117
Franz, D., Acosta, M., Altimir, N., Arriga, N., Arrouays, D., Aubinet, M., Aurela, M., Ayres, E., López‐Ballesteros, A., Barbaste, M., et al. (2018) · International Agrophysics · doi:10.1515/intag-2017-0039 · Cited by 127
“This ensures verifiable, reproducible and transparent science, and accelerates scientific progress while returning scientific impact for the data producers (Nosek et al., 2015; Dai et al., 2018).”
BackgroundA Roadmap for the Development of Applied Computational Psychiatry
Paulus, M. P., Huys, Q. J. M., & Maia, T. V. (2016) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.05.001 · Cited by 41
Replication in field ecology: Identifying challenges and proposing solutions
Filazzola, A., & Cahill, J. F. (2021) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13657 · Cited by 137
Analyses of Social Issues and Public PolicyPreprint
Aberson, C. L. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/j2zbe · Cited by 105
Calibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98
“Nevertheless, the recent credibility crisis has sparked a transdisciplinary discussion (Chalmers &Glasziou 2009,Miguel et al. 2014,Nosek et al. 2015, Poldrack et al. 2017), prompted a cascade of reform initiatives (Ioannidis 2014, Munafò et al. 2017), and catalyzed the emergence of the…”
Research integrity: Don't let transparency damage science
Lewandowsky, S., & Bishop, D. (2016) · Nature · doi:10.1038/529459a · Cited by 107
Gundersen, O. E., Gil, Y., & Aha, D. W. (2018) · AI Magazine · doi:10.1609/aimag.v39i3.2816 · Cited by 99
“…the Data Citation Synthesis Group (2014); and scholars such as Ball and Duke,4 Wilkinson et al. (2016), Stodden et al. (2016), Gil et al. (2016), Nosek et al. (2015), Starr et al. (2015), Downs et al. (2015), Mooney and Newton (2012), Goodman et al. (2014), Garijo et al. (2013), and Altman and…”
Replication, experiments and knowledge in public management research
Walker, R. M., James, O., & Brewer, G. A. (2017) · Public Management Review · doi:10.1080/14719037.2017.1282003 · Cited by 108
Strategic Implications of Openness in AI Development
Bostrom, N. (2017) · Global Policy · doi:10.1111/1758-5899.12403 · Cited by 99
Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice
Diong, J., Butler, A. A., Gandevia, S. C., & Héroux, M. E. (2018) · PLOS ONE · doi:10.1371/journal.pone.0202121 · Cited by 99
Declaring and Diagnosing Research Designs
Blair, G., Cooper, J., Coppock, A., & Humphreys, M. (2019) · American Political Science Review · doi:10.1017/s0003055419000194 · Cited by 139
“Proposed solutions to reduce this kind of bias focus on various types of preregistration of analysis strategies by researchers (Casey, Glennerster, and Miguel 2012; Green and Lin 2016; Nosek et al. 2015; Rennie 2004; Zarin and Tse 2008).”
BackgroundSelf-citationGiofrè, D., Cumming, G., Fresc, L., Boedker, I., & Tressoldi, P. (2017) · PLOS ONE · doi:10.1371/journal.pone.0175583 · Cited by 98
“io/our-services/top-guidelines/) [30], are directed to the editors of scientific journals and encourage adoption of guidelines with the aim of increasing the transparency, openness, and reproducibility of published studies.”
BackgroundCreativity measurement in undergraduate students from 1984–2013: A systematic review.
Snyder, H. T., Hammond, J. A., Grohman, M. G., & Katz-Buonincontro, J. L. (2019) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/aca0000228 · Cited by 113
Zuiderwijk, A., Shinde, R., & Jeng, W. (2020) · PLoS ONE · doi:10.1371/journal.pone.0239283 · Cited by 141
“Prior research has pointed out that the current rewarding system does not sufficiently encourage individual researchers to accomplish open science principles’ best practices such as those involving transparency, reproducibility, openness, and data reuse [22].”
BackgroundVasa, R. A., Mostofsky, S. H., & Ewen, J. B. (2016) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.02.003 · Cited by 97
“Another “endemic problem” in neuroscience research is the low power and lack of reproducibility of studies (93, 94).”
BackgroundShewach, O. R., Sackett, P. R., & Quint, S. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000420 · Cited by 97
“Given recent large-scale failures to replicate numerous effects in social psychology (Open Science Collaboration, 2015), statistical power and transparent research practices (cf., Nosek et al., 2015) are essential when evaluating reproducibility of this effect and combating publication bias.”
BackgroundAugmented reality for pedestrian evacuation research: Promises and limitations
Lovreglio, R., & Kinateder, M. T. (2020) · Safety Science · doi:10.1016/j.ssci.2020.104750 · Cited by 112
Ethnography, Data Transparency, and the Information Age
Murphy, A. K., Jerolmack, C., & Smith, D. (2021) · Annual Review of Sociology · doi:10.1146/annurev-soc-090320-124805 · Cited by 130
Elliott, K. C. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009052597 · Cited by 165
Teacher–student relationships and student outcomes: A systematic second-order meta-analytic review.
Emslander, V., Holzberger, D., Ofstad, S. B., Fischbach, A., & Scherer, R. (2025) · Psychological Bulletin · doi:10.1037/bul0000461 · Cited by 169
Progress toward openness, transparency, and reproducibility in cognitive neuroscience
Gilmore, R. O., Diaz, M. T., Wyble, B. A., & Yarkoni, T. (2017) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13325 · Cited by 95
Self-citationPredicting the replicability of social science lab experiments
Altmejd, A., Dreber, A., Forsell, E., Huber, J., Imai, T., Johannesson, M., Kirchler, M., Nave, G., & Camerer, C. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225826 · Cited by 118
Maureaud, A. A., Frelat, R., Pécuchet, L., Shackell, N. L., Mérigot, B., Pinsky, M. L., Amador, K., Anderson, S. C., Arkhipkin, A. I., Auber, A., et al. (2020) · Global Change Biology · doi:10.1111/gcb.15404 · Cited by 117
“…a global movement towards “open science” (OCDE, 2015), particularly by making data publicly available (Cheruvelil & Soranno, 2018; Gallagher et al., 2020; Nosek et al., 2015; Poisot et al., 2019), it has also sparked considerable debate over how to proceed (Moles et al., 2013; Poisot et al., 2013).”
BackgroundBoth Questionable and Open Research Practices Are Prevalent in Education Research
Makel, M. C., Hodges, J., Cook, B. G., & Plucker, J. A. (2021) · Educational Researcher · doi:10.3102/0013189x211001356 · Cited by 110
Pollet, T. V., & van der Meij, L. (2016) · Adaptive Human Behavior and Physiology · doi:10.1007/s40750-016-0050-z · Cited by 122
“All these previous recommendations will have a greater impact if researchers make the raw data available, so that all outlier handling will be open, transparent and reproducible (Nosek et al. 2015).”
BackgroundMarwick, B., Barton, C. M., Bates, L., Bollwerk, E., Bocinsky, R. K., Carter, A. K., Conrad, C., Costa, S., Crema, E. R., Davies, B., et al. (2017) · doi:10.31235/osf.io/72n8g · Cited by 17
Danchev, V., Min, Y., Borghi, J., Baiocchi, M., & Ioannidis, J. P. A. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.33972 · Cited by 94
Tackling COVID-19 Through Responsible AI Innovation: Five Steps in the Right Direction
Leslie, D. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.4bb9d7a7 · Cited by 122
Writing Effective and Reliable Monte Carlo Simulations with the SimDesign Package
Chalmers, R. P., & Adkins, M. C. (2020) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.16.4.p248 · Cited by 128
Responsible practices for data sharing.
Alter, G. C., & Gonzalez, R. (2018) · American Psychologist · doi:10.1037/amp0000258 · Cited by 93
“It also implies that data, codebooks, relevant analytic and processing scripts, and files used to create tables and figures presented in the published article should be available for reanalysis (Asendorpf et al., 2013; Miguel et al., 2014; Nosek et al., 2015).”
Self-citationResponsible Research and Responsible Leadership Studies
Tsui, A. S. (2019) · Academy of Management Discoveries · doi:10.5465/amd.2019.0244 · Cited by 111
Johnson, D. R., Kaufman, J. C., Baker, B. S., Patterson, J. D., Barbot, B., Green, A. E., van Hell, J., Kennedy, E., Sullivan, G. F., Taylor, C. L., et al. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01986-2 · Cited by 92
“Transparency and replicability are paramount for scientific progress and promoting fairness in assessment (Friedler et al., 2019; Nosek et al., 2015).”
BackgroundMesquida, C., Murphy, J., Lakens, D., & Warne, J. P. (2022) · Royal Society Open Science · doi:10.1098/rsos.220946 · Cited by 116
Amplifying Voice in Organizations
Bain, K., Kreps, T. A., Meikle, N. L., & Tenney, E. R. (2021) · Academy of Management Journal · doi:10.5465/amj.2018.0621 · Cited by 91
Cybulski, Ł., Mayo‐Wilson, E., & Grant, S. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000115 · Cited by 95
Self-citationPrzybylski, A. K., Nguyen, T.-V. T., Law, W., & Weinstein, N. (2021) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00189-w · Cited by 119
Did awarding badges increase data sharing inBMJ Open? A randomized controlled trial
Rowhani-Farid, A., Aldcroft, A., & Barnett, A. G. (2020) · Royal Society Open Science · doi:10.1098/rsos.191818 · Cited by 101
Household Food Insecurity, Diet Quality, and Obesity: An Explanatory Model
Keenan, G. S., Christiansen, P., & Hardman, C. A. (2020) · Obesity · doi:10.1002/oby.23033 · Cited by 102
“Finally, although the hypotheses and analyses were based on previous research (15), they were not formally preregistered, and preregistration is recommended for future studies in this area to enhance transparency and reproducibility (44).”
MethodAI/ML for Network Security: The Emperor has no Clothes
Jacobs, A. S., Beltiukov, R., Willinger, W., Ferreira, R. A., Gupta, A. K., & Granville, L. Z. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560609 · Cited by 156
“years, the scientific community and the network research community, in particular, have argued strongly for more reproducibility [5, 47], and we second this effort.”
BackgroundIs it time to share qualitative research data?
DuBois, J. M., Strait, M., & Walsh, H. A. (2017) · Qualitative Psychology · doi:10.1037/qup0000076 · Cited by 120
“The Open Science Collaboration has developed a series of Transparency and Openness Guidelines (TOP) meant to improve “public trust in science, and science itself” (B. A. Nosek et al., 2015, p. 1425).”
BackgroundMethodSignaling the trustworthiness of science
Jamieson, K. H., McNutt, M., Kiermer, V., & Sever, R. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1913039116 · Cited by 89
Self-citationMusic Training and Nonmusical Abilities
Schellenberg, E. G., & Lima, C. F. (2023) · Annual Review of Psychology · doi:10.1146/annurev-psych-032323-051354 · Cited by 132
The crisis of politicization within and beyond science
Druckman, J. N. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0183-5 · Cited by 88
Open science, communal culture, and women’s participation in the movement to improve science
Murphy, M. C., Mejia, A. F., Mejia, J., Yan, X., Cheryan, S., Dasgupta, N., Destin, M., Fryberg, S. A., García, J. A., Haines, E. L., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1921320117 · Cited by 95
Self-citationEstimating the Reproducibility of Psychological Science
Nosek, B. A., Cohoon, J., Kidwell, M., & Spies, J. R. (2016) · doi:10.31219/osf.io/447b3 · Cited by 88
Self-citationElevating The Status of Code in Ecology
Mislan, K. A. S., Heer, J., & White, E. P. (2015) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2015.11.006 · Cited by 85
Exploratory reports: A new article type for Cortex
McIntosh, R. D. (2017) · Cortex · doi:10.1016/j.cortex.2017.07.014 · Cited by 94
Nielson, D. M., Keren, H., O’Callaghan, G., Jackson, S. M., Douka, I., Vidal‐Ribas, P., Pornpattananangkul, N., Camp, C., Gorham, L. S., Wei, C. Y., et al. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.06.012 · Cited by 96
“Open science practices improve reliability and reduce positive bias in published reports (37,38), but they are not yet broadly adopted and were not observed in the reviewed studies (Table S5).”
Best practices in nutrition science to earn and keep the public's trust
Garza, C., Stover, P. J., Ohlhorst, S. D., Field, M. S., Steinbrook, R., Rowe, S. B., Woteki, C. E., & Campbell, E. G. (2019) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqy337 · Cited by 97
Citizen science and wildlife biology: Synergies and challenges
Frigerio, D., Pipek, P., Kimmig, S., Winter, S., Melzheimer, J., Diblíková, L., Wachter, B., & Richter, A. (2018) · Ethology · doi:10.1111/eth.12746 · Cited by 86
“The policies of scientific journals are central to the research communication process and may considerably help to improve the public trust in science, for instance by supporting publication of null results (Nosek et al., 2015).”
BackgroundCan infants learn phonology in the lab? A meta-analytic answer
Cristià, A. (2017) · Cognition · doi:10.1016/j.cognition.2017.09.016 · Cited by 94
Martone, M. E., García, A., & VandenBos, G. R. (2018) · American Psychologist · doi:10.1037/amp0000242 · Cited by 99
“Scientific Data, 4, 160127. doi: 10.1038/sdata.2016.127 Footnotes 1Material in this article reflects issues discussed in the Dagstuhl Perspectives Workshop on Digital Scholarship and Open Science in Psychology and the Behavioral Sciences, July 19-24, 2015 (DIO: 10.4230/DagRep.5.7.42).”
BackgroundInfluentialSelf-citationPutting the Self in Self-Correction: Findings From the Loss-of-Confidence Project
Rohrer, J. M., Tierney, W., Uhlmann, E. L., DeBruine, L. M., Heyman, T., Jones, B. C., Schmukle, S. C., Silberzahn, R., Willén, R. M., Carlsson, R., et al. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620964106 · Cited by 88
“Guidelines to promote openness (e.g., Nosek et al., 2015) might partly reduce this asymmetry and thus make it easier for third parties to spot flaws.”
BackgroundSelf-citationBajikar, S. S., Wang, C.-C., Borten, M. A., Pereira, E. J., Atkins, K. A., & Janes, K. A. (2017) · Developmental Cell · doi:10.1016/j.devcel.2017.10.027 · Cited by 85
“All terminal histology was preregistered with the Open Science Framework before beginning the tissue processing (https://osf.io/62cse/ and https://osf.io/ay2hu/).”
MethodNanoengineering Particles through Template Assembly
Björnmalm, M., Cui, J., Bertleff‐Zieschang, N., Song, D., Faria, M., Rahim, M. A., & Caruso, F. (2016) · Chemistry of Materials · doi:10.1021/acs.chemmater.6b02848 · Cited by 86
Washburn, A. N., Hanson, B. E., Motyl, M. S., Skitka, L. J., Yantis, C., Wong, K., Sun, J., Prims, J., Mueller, A. B., Melton, Z. J., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918757427 · Cited by 92
“hypotheses and making data publicly available online (the Transparency and Openness Promotion, or TOP, Guidelines; Nosek et al., 2015).”
BackgroundBerente, N., Lyytinen, K., Yoo, Y., & Maurer, C. (2019) · MIS Quarterly · doi:10.25300/misq/2019/14214 · Cited by 122
Šoškić, А., Jovanović, V., Styles, S. J., Kappenman, E. S., & Ković, V. (2021) · Neuropsychology Review · doi:10.1007/s11065-021-09513-4 · Cited by 100
“…there has been a push in the past few years towards improving reproducibility and credibility of research through encouraging open science practices (Ioan-nidis et al., 2014; Nosek et al., 2015), so we were interested whether more recent papers had started to implement these recommendations.”
BackgroundInstitutional characteristics and the development of crowdfunding across countries
Pietro, F. D., & Butticè, V. (2020) · International Review of Financial Analysis · doi:10.1016/j.irfa.2020.101543 · Cited by 87
Living guidelines for generative AI — why scientists must oversee its use
Bockting, C. L., van Dis, E. A. M., van Rooij, R., Zuidema, W., & Bollen, J. (2023) · Nature · doi:10.1038/d41586-023-03266-1 · Cited by 82
Guidelines for conducting research studies with the autism community.
Gowen, E., Taylor, R., Bleazard, T., Greenstein, A., Baimbridge, P., & Poole, D. (2019) · PubMed · Cited by 165
Systematic Subjectivity: How Subtle Biases Infect the Scholarship Review Process
King, E. B., Avery, D. R., Hebl, M. R., & Cortina, J. M. (2017) · Journal of Management · doi:10.1177/0149206317743553 · Cited by 102
“…of ManagemenKing et al. / Bias in the Review Process research-article2017 Editorial Commentary The public and scientific communities are engaging in important conversations about fabrication (Singal, 2015) and reproducibility (Nosek et al., 2015) of findings from the scientific literature.”
BackgroundAvesani, P., McPherson, B., Hayashi, S., Caiafa, C. F., Henschel, R., Garyfallidis, E., Kitchell, L. M., Bullock, D. N., Patterson, A. J., Olivetti, E., et al. (2019) · Scientific Data · doi:10.1038/s41597-019-0073-y · Cited by 86
Distractor probability changes the shape of the attentional template.
Geng, J. J., DiQuattro, N. E., & Helm, J. L. (2017) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/xhp0000430 · Cited by 92
“In order to confirm our conclusions and replicate our previous findings (see Ledgerwood, 2014; Nosek et al., 2015), the discrimination task in Experiment 2 was adjusted so that the target could only be identified by color.”
ResultVideo can make behavioural science more reproducible
Gilmore, R. O., & Adolph, K. E. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0128 · Cited by 81
“We applaud recent efforts to bolster the openness, transparency, and reproducibility of behavioral research, including large scale replications of published findings (2); pre-registration of studies; changes in review and publication practices; new incentives to encourage reproducible practices; and more open sharing…”
Preregistration Is Neither Sufficient nor Necessary for Good Science
Pham, M. T., & Oh, T. T. (2020) · Journal of Consumer Psychology · doi:10.1002/jcpy.1209 · Cited by 104
A Beginner's Guide to Conducting Reproducible Research
Alston, J. M., & Rick, J. A. (2021) · Bulletin of the Ecological Society of America · doi:10.1002/bes2.1801 · Cited by 91
How to make replication the norm
Gertler, P., Galiani, S., & Romero, M. (2018) · Nature · doi:10.1038/d41586-018-02108-9 · Cited by 81
Science as behaviour: Using a behaviour change approach to increase uptake of open science
Norris, E., & O’Connor, D. B. (2019) · Psychology & Health · doi:10.1080/08870446.2019.1679373 · Cited by 81
“Nevertheless, firm and well-documented barriers to adopting and maintaining Open Science behaviours remain for some researchers (Nosek et al., 2015).”
BackgroundDigital open science—Teaching digital tools for reproducible and transparent research
Toelch, U., & Ostwald, D. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2006022 · Cited by 80
“They experience directly how to meet the quality standards of a reproducible and transparent research process [14].”
BackgroundMethodStress and cognition: A user’s guide to designing and interpreting studies
Shields, G. S. (2020) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2019.104475 · Cited by 80
Actionable recommendations for narrowing the science-practice gap in open science
Aguinis, H., Banks, G. C., Rogelberg, S. G., & Cascio, W. F. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.02.007 · Cited by 80
Self-citationClosing the knowledge‐action gap in conservation with open science
Roche, D. G., O’Dea, R. E., Kerr, K. A., Rytwinski, T., Schuster, R. H., Nguyen, V. M., Young, N., Bennett, J., & Cooke, S. J. (2021) · Conservation Biology · doi:10.1111/cobi.13835 · Cited by 89
“These outputs can receive permanent identifiers (e.g., a DOI) that allow them to be cited and the authors credited (Nosek et al., 2015 [https://www.cos.io/initiatives/top-guidelines]).”
BackgroundSzűcs, D., & Myers, T. (2016) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2016.11.002 · Cited by 96
Journal Data Sharing Policies and Statistical Reporting Inconsistencies in Psychology
Nuijten, M. B., Borghuis, J., Veldkamp, C. L. S., Dominguez-Alvarez, L., van Assen, M. A. L. M., & Wicherts, J. M. (2017) · Collabra: Psychology · doi:10.1525/collabra.102 · Cited by 79
“Over the past few years there has been increasing awareness that the availability of research data is essential for scientific progress (Anagnostou et al., 2015; Nosek et al., 2015; Wicherts, 2011; Wilkinson et al., 2016), and several journals have started to request authors to share their data when they submit an art…”
BackgroundLiberating field science samples and data
Mcnutt, M. K., Lehnert, K. A., Hanson, B., Nosek, B. A., Ellison, A. M., & King, J. L. (2016) · Science · doi:10.1126/science.aad7048 · Cited by 77
“These techniques have been applied to describe strongly interacting forms of quantum matter, predicting that they should exhibit hydrodynamic fl ows (5).”
BackgroundMethodSelf-citationThe reuse of public datasets in the life sciences: potential risks and rewards
Sielemann, K., Hafner, A., & Pucker, B. (2020) · PeerJ · doi:10.7717/peerj.9954 · Cited by 94
“Many international data sharing guidelines like FAIR (Findable Accessible Interoperable Reusable) (10), TOP (The Transparency and Openness Promotion) (11), Open Data in a Big Data World (1) and the Beijing Declaration (CODATA) (12) have emerged by necessity of the ‘big data revolution’.”
BackgroundMetascience as a Scientific Social Movement
Peterson, D., & Panofsky, A. L. (2023) · Minerva · doi:10.1007/s11024-023-09490-3 · Cited by 119
Zhang, G., Jin, Q., Zhou, Y., Wang, S., Idnay, B., Luo, Y., Park, E., Nestor, J. G., Spotnitz, M. E., Soroush, A., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01239-w · Cited by 131
“the relevant superior performances compared to open-source alternatives 7,23 – 26 , the disadvantages of closed-source models were rarely discussed or even mentioned 27,28 .”
Mitigating Illusory Results through Preregistration in Education
Gehlbach, H., & Robinson, C. D. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2017.1387950 · Cited by 84
“Responding to Makel, Plucker, and Hegarty (2012) finding that roughly 1% of psychology studies were replications, the field responded with major replication projects (Nosek et al., 2015).”
Hardwicke, T. E., & Ioannidis, J. P. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201856 · Cited by 78
Navigating Open Science as Early Career Feminist Researchers
Pownall, M., Talbot, C. V., Henschel, A., Lautarescu, A., Lloyd, K. E., Hartmann, H., Darda, K. M., Tang, K. T. Y., Carmichael-Murphy, P., & Siegel, J. A. (2021) · Psychology of Women Quarterly · doi:10.1177/03616843211029255 · Cited by 85
“In recent years, the open science movement has prompted a discipline-wide reappraisal of the reproducibility, replicability, and robustness of psychological science (Nosek et al., 2015; Open Science Collaboration, 2015).”
BackgroundMacleod, M., Collings, A. M., Graf, C., Kiermer, V., Mellor, D., Swaminathan, S., Sweet, D., & Vinson, V. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2103238118 · Cited by 77
Publication Bias and Editorial Statement on Negative Findings
Blanco-Perez, C., & Brodeur, A. (2020) · The Economic Journal · doi:10.1093/ej/ueaa011 · Cited by 76
Is Most Published Research Really False?
Leek, J. T., & Jager, L. R. (2016) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-060116-054104 · Cited by 90
A Taxonomy of Transparency in Science
Elliott, K. C. (2020) · Canadian Journal of Philosophy · doi:10.1017/can.2020.21 · Cited by 86
Science and the public: Debate, denial, and skepticism
Lewandowsky, S., MANN, M. E., Brown, N. J. L., & Friedman, H. L. (2016) · Journal of Social and Political Psychology · doi:10.5964/jspp.v4i2.604 · Cited by 101
“In this regard, we strongly support the principles of the Open Science movement (e.g., Nosek & Bar-Anan, 2012; Nosek, Spies, & Motyl, 2012; Nosek et al., 2015).”
The role of replication studies in ecology
Fraser, H. S., Barnett, A., Parker, T. H., & Fidler, F. M. (2020) · Ecology and Evolution · doi:10.1002/ece3.6330 · Cited by 91
“…in other fields, with the inclusion of a criterion describing journals' stance on accepting replication studies as part of the TOP guidelines (Nosek et al., 2015; to which over 5,000 journals are signatories) and the advent of Registered Replication Reports (Simons, Holcombe, & Spellman,…”
BackgroundQuestionable science and reproducibility in electrical brain stimulation research
Héroux, M. E., Loo, C. K. A., Taylor, J. L., & Gandevia, S. C. (2017) · PLoS ONE · doi:10.1371/journal.pone.0175635 · Cited by 80
Why and how we should care about the General Data Protection Regulation
Crutzen, R., Peters, G.-J., & Mondschein, C. F. (2019) · Psychology and Health · doi:10.1080/08870446.2019.1606222 · Cited by 75
“Such datasets can then immediately be made public to comply with Open Science principles.”
BackgroundInfluentialMany Labs 2: Investigating Variation in Replicability Across Sample and SettingPreprint
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M. E., Axt, J., Babalola, M. T., Bahník, Š., et al. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9654g · Cited by 88
“In any case, these findings provide further justification for improving transparency of research (Miguel et al., 2014; Nosek et al., 2015), and preregistering studies to make all findings discoverable even if they are not published and preregistering analysis plans to make clear the distinction…”
Background
Metadata
- Authors
- Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E. L., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T.
- Affiliations
- Affiliations for the authors, all of whom are members of the TOP Guidelines Committee, are given in the supplementary materials.
- Corresponding author
- Nosek, B. A.
- ORCID
- Nosek, B. A.
- Alter, G.
- Banks, G. C.
- Borsboom, D.
- Bowman, S. D.
- Breckler, S. J.
- Buck, S.
- Chambers, C. D.
- Christensen, G.
- Dafoe, A.
- Eich, E.
- Glennerster, R.
- Goroff, D.
- Hesse, B.
- Humphreys, M.
- Ishiyama, J.
- Karlan, D.
- Lupia, A.
- Mabry, P.
- Madon, T.
- Malhotra, N.
- McNutt, M.
- Miguel, E.
- Paluck, E. Levy
- Simonsohn, U.
- Soderberg, C.
- Spellman, B. A.
- VandenBos, G.
- Vazire, S.
- Wagenmakers, E. J.
- Wilson, R.
- Yarkoni, T.
- Venue
- Science
- Published
- 2015-06-26
- Volume
- 348
- Issue
- 6242
- Pages
- 1422-1425
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC4550299
- OpenAlex
- W1767470961
- Cited by
- 2,708
- References
- 15
- Citations by year
- 2026: 214 · 2025: 203 · 2024: 200 · 2023: 232 · 2022: 278 · 2021: 306 · 2020: 360 · 2019: 276 · 2018: 256 · 2017: 211 · 2016: 134 · 2015: 38
- FWCI
- 418.98
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- PubMed Central (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open research culture, author guidelines, journal guidelines, transparency, research transparency, reproducibility
- MeSH terms
- Periodicals as Topic, Research, Reproducibility of Results, Guidelines as Topic, Peer Review, Research, Access to Information, Disclosure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.