Promoting an open research culture
Open accessPublished version of a preprint
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Author guidelines for journals could help to promote transparency, openness, and reproducibility
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, many are closed source applications with black box algorithms, which is problematic as transparency and data ownership are essential in the era of open science 22 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“Open science will clearly help in this regard (Nosek et al., 2015).”
From Intrusive memories of trauma: A target for research bridging cognitive science and its clinical application (2019) · cites this paper as Nosek et al., 2015
In the citing paper
“These issues may be strongly suspected and their magnitude can sometimes be estimated 63 , but cannot truly be corrected 64,65 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Self-citationRecommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationRegistered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Self-citationPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citationPublication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Self-citation
Cited by
Items 501–600 of 1,016
When and Why to Replicate: As Easy as 1, 2, 3?
Field, S. M., Hoekstra, R., Bringmann, L. F., & van Ravenzwaaij, D. (2019) · Collabra Psychology · doi:10.1525/collabra.218 · Cited by 56
Faculty Research Incentives and Business School Health: A New Perspective from and for Marketing
Stremersch, S., Winer, R. S., & Camacho, N. (2021) · Journal of Marketing · doi:10.1177/00222429211001050 · Cited by 47
Developing an open science ‘mindset’
Hagger, M. S. (2021) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2021.2012474 · Cited by 56
“…that aim to improve replicability, reproducibility, credibility, and transparency in research (e.g. Aczel et al., 2020; Appelbaum et al., 2018; Kidwell et al., 2016; Morey et al., 2016; NIH, 2021a; Nosek et al., 2015; NSF, 2021a; Open Science MOOC, 2018; SIPS, 2021; The FOSTER consortium, 2021a).”
BackgroundMethodInfluentialA tale of two 'opens': intersections between Free and Open Source Software and Open ScholarshipPreprint
Tennant, J., Agarwal, R., Baždarić, K., Brassard, D., Crick, T., Dunleavy, D. J., Evans, T. R., Gardner, N., González-Márquez, M., Graziotin, D., et al. (2020) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/2kxq8 · Cited by 55
“We recognise that openness in scholarly research has become a dynamic and evolving spectrum of principles, values, and practices (Willinsky 2006; Nosek et al. 2015; Moore 2017; Chan et al. 2019; Tennant, Beamer, et al. 2019), rooted in the philosophy of unrestrained participation and the Commons (Suber 2007a; Bosman e…”
BackgroundSelf-citationLissa, C. J. V. (2022) · Infant and Child Development · doi:10.1002/icd.2370 · Cited by 48
Does music training improve inhibition control in children? A systematic review and meta-analysis
Jamey, K., Foster, N. E. V., Hyde, K. L., & Bella, S. D. (2024) · Cognition · doi:10.1016/j.cognition.2024.105913 · Cited by 69
Reproducible Tables in Psychology Using the apaTables Package
Stanley, D. J., & Spence, J. R. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918773743 · Cited by 54
Colella, J. P., Stephens, R., Campbell, M., Kohli, B. A., Parsons, D. J., & Mclean, B. S. (2020) · BioScience · doi:10.1093/biosci/biaa146 · Cited by 41
Bean, H. R., Diggens, J., Ftanou, M., Alexander, M., Stafford, L., Bei, B., Francis, P. A., & Wiley, J. F. (2021) · SLEEP · doi:10.1093/sleep/zsab246 · Cited by 52
Do multiple experimenters improve the reproducibility of animal studies?
von Kortzfleisch, V. T., Ambrée, O., Karp, N. A., Meyer, N., Novak, J., Palme, R., Rosso, M., Touma, C., Würbel, H., Kaiser, S., et al. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001564 · Cited by 50
“To improve the situation, several guidelines such as the TOP [11], ARRIVE [12,13], and PREPARE [14] guidelines have been established to improve the planning, conduct, analysis, and reporting of studies.”
MethodSong, H., Markowitz, D. M., & Taylor, S. H. (2022) · Journal of Communication · doi:10.1093/joc/jqac017 · Cited by 41
Open data, open review and open dialogue in making social sciences plausiblePreprint
Vuong, Q. (2021) · doi:10.31219/osf.io/da9zs · Cited by 42
Magnitude‐based Inference is not Bayesian and is not a valid method of inference
Sainani, K. L., Lohse, K. R., Jones, P. R., & Vickers, A. J. (2019) · Scandinavian Journal of Medicine and Science in Sports · doi:10.1111/sms.13491 · Cited by 47
Bergeat, D., Lombard, N., Gasmi, A., Le Floch, B., & Naudet, F. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.15209 · Cited by 41
Begoli, E., Goethert, I., & Knight, K. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671534 · Cited by 50
Shortfalls in tracking data available to inform North American migratory bird conservation
Scarpignato, A. L., Huysman, A. E., Jimenez, M. F., Witko, C. J., Harrison, A., Seavy, N. E., Smith, M., Deppe, J. L., Wilsey, C. B., & Marra, P. P. (2023) · Biological Conservation · doi:10.1016/j.biocon.2023.110224 · Cited by 42
Tamminen, K. A., & Poucher, Z. A. (2018) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2017.12.010 · Cited by 53
Konkol, M., Nüst, D., & Goulier, L. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00095-y · Cited by 47
Questionable research practices may have little effect on replicability
Ulrich, R., & Miller, J. O. (2020) · eLife · doi:10.7554/elife.58237 · Cited by 52
Valuing the Diversity of Research Methods to Advance Nutrition Science
Mattes, R. D., Rowe, S. B., Ohlhorst, S. D., Brown, A. W., Hoffman, D. J., Liska, D. J., Feskens, E. J. M., Dhillon, J., Tucker, K. L., Epstein, L. H., et al. (2022) · Advances in Nutrition · doi:10.1093/advances/nmac043 · Cited by 53
Methods, transparency and reporting of clinical trials in orthodontics and periodontics
Papageorgiou, S. N., Antonoglou, G. N., Martin, C., & Eliades, T. (2019) · Journal of Orthodontics · doi:10.1177/1465312519842315 · Cited by 40
Edmondson, D., Falzon, L., Sundquist, K. J., Julian, J., Meli, L., Sumner, J. A., & Kronish, I. M. (2018) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.10.001 · Cited by 39
“(Nosek et al., 2015) To improve our ability understand behavior change and effectively support it, our field will need to accomplish the following five tasks: 1) Develop a common experimental medicine approach for understanding behavior change; 2) Develop and test valid measures for assessing…”
MethodYu, H., Santos‐Rocha, R., Radzimiński, Ł., Jastrzębski, Z., Bonisławska, I., Szwarc, A., & Szumilewicz, A. (2022) · Nutrients · doi:10.3390/nu14245279 · Cited by 44
“The study adhered to the principles of openness, transparency, reproducibility, and the CONSORT standards [27].”
MethodTaboos and Self-Censorship Among U.S. Psychology Professors
Clark, C. J., Fjeldmark, M., Lu, L., Baumeister, R. F., Ceci, S. J., Frey, K., Miller, G. F., Reilly, W., Tice, D. M., von Hippel, W., et al. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916241252085 · Cited by 56
“For example, many scholars have advocated for greater transparency and reproducibility in science (e.g., Kidwell et al., 2016; Nosek et al., 2015, 2022), but not all scholars have responded favorably to these norm changes (e.g., Bahlai et al., 2019; Whitaker & Guest, 2020).”
Investigating the replicability of the social and behavioural sciences
Tyner, A. H., Abatayo, A. L., Daley, M., Field, S., Fox, N. W., Haber, N., Hahn, K. M., Struhl, M. K., Mawhinney, B., Miske, O., et al. (2026) · Nature · doi:10.1038/s41586-025-10078-y · Cited by 60
Self-citationRichardson, J. E., Middleton, B., Platt, J. E., & Blumenfeld, B. H. (2019) · Learning Health Systems · doi:10.1002/lrh2.10208 · Cited by 42
“25,26 3 Key to Learning Health Systems is the capacity to provide Feedback and Updates on the implemented knowledge artifact or CDS from the vantage point of any user: whether that be a physician, nurse, or other member of the care team, as well as the patient him or herself.”
BackgroundStracke, C. M. (2020) · Lecture notes in educational technology · doi:10.1007/978-981-15-4276-3_2 · Cited by 54
Healthcare chauvinism during the COVID-19 pandemic
Larsen, M. H., & Schaeffer, M. (2020) · Journal of Ethnic and Migration Studies · doi:10.1080/1369183x.2020.1860742 · Cited by 40
“Pre-registration strengthens the validity and reliability of confirmatory (experimental) research by constraining p-hacking and the selective reporting of results (Nosek et al. 2015).”
BackgroundEmbedding open and reproducible science into teaching: A bank of lesson plans and resources.
Pownall, M., FORRT, Azevedo, F., Aldoh, A., Elsherif, M., Vasilev, M., Pennington, C. R., Robertson, O., Tromp, M. V., Liu, M., et al. (2024) · Scholarship of Teaching and Learning in Psychology · doi:10.1037/stl0000307 · Cited by 40
Bennett, E. (2021) · Psychology of Women Quarterly · doi:10.1177/03616843211036460 · Cited by 58
“This approach attempts to pull away from the very real possibility that a focus on open science means that research studies fitting into an open science framework could become a new prerequisite for publishing in top psychological journals (see Nosek et al., 2015, for such guidelines).”
BackgroundEnhancing Climate Change Research With Open Science
Tai, T. C., & Robinson, J. P. W. (2018) · Frontiers in Environmental Science · doi:10.3389/fenvs.2018.00115 · Cited by 46
“transparency policies (Nosek et al., 2015) Standardized metadata reporting will facilitate literature”
BackgroundTowards more evidence-based agricultural and food policies
Benni, N. E., Grovermann, C., & Finger, R. (2023) · Q Open · doi:10.1093/qopen/qoad003 · Cited by 59
Miller, J. O., & Ulrich, R. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616649170 · Cited by 43
A 20-Year Review of Outcome Reporting Bias in Moderated Multiple Regression
O’Boyle, E. H., Banks, G. C., Carter, K., Walter, S. L., & Yuan, Z. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9539-8 · Cited by 43
Self-citationChuan-Peng, H., Kong, X., Er, I.-J., Al, E., & Peng, K. (2018) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2018.00951 · Cited by 56
“…2009) 、 ANOVA(Rouder, et al., 2012) 和相关分析 (Ly, Marsman, & Wagenmakers, 2017; Ly, et al., 2016b) 多样,仅改变统计方法不能让心理学的研究变得可重复。例如数据不开放以及研究过程不透 明 (Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014; Lindsay, 2015; Nosek et al., 2015) 、 对探索性分析与验证性分析不加区分 (Kerr, 1998; Wagenmakers, Wetzels, Borsboom, van”
Schroeders, U., & Gnambs, T. (2018) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000500 · Cited by 46
“…in line with the recommendations of the Association of Psychological Science (Eich, 2014) and the extensive efforts of the Open Science Framework (Nosek et al., 2015) to make scientific research more transparent, open, and reproducible, we strongly advocate to make the raw data and the used…”
MethodResultInfluentialWillemse, C., & Wykowska, A. (2019) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2018.0036 · Cited by 47
“However, on the side-lines of the current reproducibility debate in psychology (e.g. [12–14]) in which one-one-one replication is encouraged, even less is currently known about whether previous findings in social cognition replicate in more naturalistic adaptations.”
BackgroundMaking Every Study Count: Learning From Replication Failure to Improve Intervention Research
Kim, J. S. (2019) · Educational Researcher · doi:10.3102/0013189x19891428 · Cited by 54
“Direct replications that duplicate the methods of an original study are both rare and difficult to implement in the education sciences because the district, school, and classroom context for experimentation changes over time (Makel & Plucker, 2014; Nosek et al., 2015; Van Bavel et al., 2016).”
MethodBistline, J. E. T., Budolfson, M. B., & Francis, B. (2020) · Climate Policy · doi:10.1080/14693062.2020.1781048 · Cited by 51
Behavioural lateralization in a detour test is not repeatable in fishes
Roche, D. G., Amcoff, M., Morgan, R., Sundin, J., Andreassen, A. H., Finnøen, M. H., Lawrence, M. J., Henderson, E., Norin, T., Speers‐Roesch, B., et al. (2020) · Animal Behaviour · doi:10.1016/j.anbehav.2020.06.025 · Cited by 44
“Measurements using these methods and their cross-context repeatability should be validated in accordance with TOP guidelines (Nosek et al., 2015) to establish reproducible protocols that inspire confidence.”
BackgroundMethodBenjamin‐Chung, J., Arnold, B. F., Kennedy, C. J., Mishra, K., Pokpongkiat, N. N., Nguyen, A., Jilek, W., Holbrook, K., Pan, E., Kirley, P. D., et al. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003238 · Cited by 41
“Unique strengths of our study design include the use of multivariate matching and a DID approach to minimize systematic differences between the intervention and comparison sites and the prespecification of our statistical analysis plan [60].”
MethodSewasew, D., & Koester, L. S. (2019) · Learning and Individual Differences · doi:10.1016/j.lindif.2019.05.010 · Cited by 38
Advances in transparency and reproducibility in the social sciences
Freese, J., Rauf, T., & Voelkel, J. G. (2022) · Social Science Research · doi:10.1016/j.ssresearch.2022.102770 · Cited by 60
de Sousa, A. A., Beaudet, A., Calvey, T., Bardo, A., Benoît, J., Charvet, C. J., Dehay, C., Gómez‐Robles, A., Gunz, P., Heuer, K., et al. (2023) · Communications Biology · doi:10.1038/s42003-023-04803-4 · Cited by 45
Kandel, A. W., Sommer, C., Kanaeva, Z., Bolus, M. P., Bruch, A. A., Groth, C., Haidle, M. N., Hertler, C., Heß, J., Malina, M., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289513 · Cited by 43
“It is also supported by general trends in research [6, 7], especially when funding agencies require that a Data Management Plan accompany awarded grants [8].”
Data Management Rubric for Video Data in Organismal Biology
Brainerd, E. L., Blob, R. W., Hedrick, T. L., Creamer, A., & Müller, U. K. (2017) · Integrative and Comparative Biology · doi:10.1093/icb/icx060 · Cited by 41
“In addition to funder expectations, there are also publisher requirements for the retention of data supporting publications (e.g., Joint Data Archiving Policy [datadryad.org/ pages/jdap]) and for the citation, deposit, and longterm archiving of these data in public repositories (Nosek et al. 2015).”
BackgroundMethodInfluentialSchoemann, M., Lüken, M., Grage, T., Kieslich, P. J., & Scherbaum, S. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-1179-4 · Cited by 41
“The problems concern both the validity of mousetracking as a process-tracing method and the principles of reproducible science (i.e., detailed documentation of the setup and procedure; cf. Munafò et al., 2017; Nosek et al., 2015).”
BackgroundGunnell, K. E., Belcourt, V. J., Tomasone, J. R., & Weeks, L. C. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1966823 · Cited by 58
“Open science and reporting practices for systematic reviews Open science and open research practices is a movement with the aim of creating open, transparent, and reproducible science (Nosek et al., 2015).”
BackgroundToward Open and Reproducible Epidemiology
Mathur, M. B., & Fox, M. P. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad007 · Cited by 56
“Researchers can also encourage journals to create open-science guidelines or adopt existing ones, for example by becoming signatories to the Center for Open Science Transparency and Openness Promotion (TOP) guidelines (81).”
BackgroundCampbell, R., Javorka, M., Engleton, J., Fishwick, K., Gregory, K., & Goodman-Williams, R. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231205832 · Cited by 35
Catalá-López, F., Caulley, L., Ridao-López, M., Hutton, B., Husereau, D., Drummond, M., Alonso‐Arroyo, A., Pardo-Fernández, M., Bernal‐Delgado, E., Meneu, R., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034463 · Cited by 41
Competition for priority harms the reliability of science, but reforms can help
Tiokhin, L., Yan, M., & Morgan, T. J. H. (2021) · Nature Human Behaviour · doi:10.1038/s41562-020-01040-1 · Cited by 52
Open research data repositories: Practices, norms, and metadata for sharing images
Hansson, K. E., & Dahlgren, A. N. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24571 · Cited by 43
“Increasing datafication of research practices and their infrastructure also creates new incentives to enable several researchers to share metadata and data (Kidwell et al., 2016; Nosek et al., 2015; Roche et al., 2014).”
BackgroundGuidelines to improve internationalization in the psychological sciences
Puthillam, A., Doble, L. J. M., Santos, J. J. I. D., Elsherif, M. M., Steltenpohl, C. N., Moreau, D., Pownall, M., Silverstein, P., Anand-Vembar, S., & Kapoor, H. (2023) · Social and Personality Psychology Compass · doi:10.1111/spc3.12847 · Cited by 45
All-hazards dataset mined from the US National Incident Management System 1999–2014
Denis, L. A. S., Mietkiewicz, N. P., Short, K. C., Buckland, M., & Balch, J. K. (2020) · Scientific Data · doi:10.1038/s41597-020-0403-0 · Cited by 41
Standards for evidence in policy decision-makingPreprint
Ruggeri, K., van der Linden, S., Wang, C., Papa, F., Riesch, J., & Green, J. S. A. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/fjwvk · Cited by 37
Whose science? A new era in regulatory “science wars”
Wagner, W. E., Fisher, E. C., & Pascual, P. (2018) · Science · doi:10.1126/science.aau3205 · Cited by 36
Considerations for making informed choices about engaging in open qualitative research
Tamminen, K. A., Bundon, A., Smith, B. M., McDonough, M. H., Poucher, Z. A., & Atkinson, M. (2021) · Qualitative Research in Sport Exercise and Health · doi:10.1080/2159676x.2021.1901138 · Cited by 60
Primer for Experimental Methods in Organization Theory
Levine, S. S., Schilke, O. S., Kacperczyk, A. J., & Zucker, L. G. (2023) · Organization Science · doi:10.1287/orsc.2023.18093 · Cited by 42
“The data, code, and instruments should be detailed enough so that any reader can understand and assess the conclusions of the manuscript (Nosek et al. 2015, § 2-4, Davis et al. 2023, pp. 13–14).”
Hurdles in Basic Science Translation
Perry, C. J., & Lawrence, A. J. (2017) · Frontiers in Pharmacology · doi:10.3389/fphar.2017.00478 · Cited by 39
“Although this is a particularly low figure, this trend is not an isolated finding (Prinz et al., 2011), nor is it unique to the field of oncology or research carried out by industry (Nosek et al., 2015).”
BackgroundMarwick, B. (2017) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · doi:10.17605/osf.io/3d6xx · Cited by 41
Scientific rigor and credibility in the nutrition research landscape
Kroeger, C. M., Garza, C., Lynch, C. J., Myers, E. F., Rowe, S. B., Schneeman, B. O., Sharma, A. M., & Allison, D. B. (2018) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqx067 · Cited by 45
“Various fields and parties have expressed such concerns and put forth efforts to strengthen the integrity of the scientific process, such as the American Statistical Association (6), the Center for Open Science (7), individual universities [e.g., (8)], and government funding agencies [e.g., (9, 10)].”
BackgroundKaplan, D. M., Rentscher, K. E., Lim, M., Reyes, R., Keating, D., Romero, J., Shah, A., Smith, A. D., York, K., Milek, A., et al. (2020) · Behavior Research Methods · doi:10.3758/s13428-019-01333-y · Cited by 43
Christian, T.-M. L., Gooch, A., Vision, T., & Hull, E. (2020) · PLoS ONE · doi:10.1371/journal.pone.0230281 · Cited by 42
Foundational Practices of Research Data Management
Briney, K. A., Coates, H., & Goben, A. H. (2020) · Research Ideas and Outcomes · doi:10.3897/rio.6.e56508 · Cited by 39
Kong, X., ENIGMA Laterality Working Group, & Francks, C. (2020) · Human Brain Mapping · doi:10.1002/hbm.25154 · Cited by 35
Does Sociology Need Open Science?
Breznau, N. (2021) · Societies · doi:10.3390/soc11010009 · Cited by 46
“In fact, the TOP guideline, “5 Design Analysis Transparency” promotes making a statement on data sharing; in its ideal this means authors must state what they share and why.”
BackgroundInfluentialSeven steps to enhance Open Science practices in animal science
Muñoz‐Tamayo, R., Nielsen, B. L., Gagaoua, M., Gondret, F., Krause, E. T., Morgavi, D., Olsson, I. A. S., Pastell, M., Taghipoor, M., Tedeschi, L. O., et al. (2022) · PNAS Nexus · doi:10.1093/pnasnexus/pgac106 · Cited by 41
“Use this or similar articles (10) for your next journal club.”
BackgroundProposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science
Murphy, J., Mesquida, C., Caldwell, A. R., Earp, B. D., & Warne, J. P. (2022) · Sports Medicine · doi:10.1007/s40279-022-01749-1 · Cited by 46
“Consequently, one goal of replication is to bolster our collective confidence in the veracity of novel claims by providing diagnostic evidence about those claims [5, 6].”
BackgroundImportance of timely metadata curation to the global surveillance of genetic diversity
Crandall, E. D., Toczydlowski, R. H., Liggins, L., Holmes, A. E., Ghoojaei, M., Gaither, M. R., Wham, B. E., Pritt, A. L., Noble, C., Anderson, T. J., et al. (2023) · Conservation Biology · doi:10.1111/cobi.14061 · Cited by 44
“Taken together, our results support assertions by others in the field that the current research system overly weights publications and citations and underweights scientific openness and reproducibility (Davies, Putnam, et al., 2021; Fidler et al., 2017; McNutt et al., 2016; Nosek et al., 2015).”
ResultEmbedding data provenance into the Learning Health System to facilitate reproducible research
Ćurčin, V. (2016) · Learning Health Systems · doi:10.1002/lrh2.10019 · Cited by 44
Examining the Reproducibility of Meta-Analyses in Psychology: A Preliminary ReportPreprint
Lakens, D., Page‐Gould, E., van Assen, M. A. L. M., Spellman, B., Schönbrodt, F. D., Hasselman, F., Corker, K. S., Grange, J., Sharples, A. E., Cavender, C., et al. (2017) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/xfbjf · Cited by 42
Self-citationA practical guide for transparency in psychological sciencePreprint
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., IJzerman, H. R., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/rtygm · Cited by 46
Fostering reproducibility in industry-academia research
Jasny, B. R., Wigginton, N. S., Mcnutt, M. K., Bubela, T. M., Buck, S., Cook-Deegan, R. M., Gardner, T. S., Hanson, B., Hustad, C. M., Kiermer, V., et al. (2017) · Science · doi:10.1126/science.aan4906 · Cited by 40
Self-citationArchival research: Expanding the methodological toolkit in social psychology
Heng, Y. T., Wagner, D. T., Barnes, C. M., & Guarana, C. L. O. (2018) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2018.04.012 · Cited by 57
Open science practices in clinical psychology journals: An audit study.
Nutu, D., Gentili, C., Naudet, F., & Cristea, I. A. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000414 · Cited by 34
Xafis, V., & Labude, M. K. (2019) · Asian Bioethics Review · doi:10.1007/s41649-019-00097-z · Cited by 35
Caught in the Act: Predicting Cheating in Unproctored Knowledge Assessment
Steger, D., Schroeders, U., & Wilhelm, O. (2020) · Assessment · doi:10.1177/1073191120914970 · Cited by 37
“To make the present analyses transparent and reproducible (Nosek et al., 2015), we provide all material (i.e., data, syntax, and additional tables and figures) online within the Open Science Framework: https://osf .io/74p2w/”
MethodTREND-DB—a transcriptome-wide atlas of the dynamic landscape of alternative polyadenylation
Marini, F., Scherzinger, D., & Danckwardt, S. (2020) · Nucleic Acids Research · doi:10.1093/nar/gkaa722 · Cited by 38
“As computational reproducibility gains more attention (94; 95; 96) in the scientific community, it will be desirable to support the creation of a report-like document, e.”
BackgroundOpen science and reform practices in organizational behavior research over time (2011 to 2019)
Tenney, E. R., Costa, E., Allard, A., & Vazire, S. (2021) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.10.015 · Cited by 37
Self-citationImproving norms in research culture to incentivize transparency and rigor
Mellor, D. T. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1902329 · Cited by 41
“Developed in 2014 among a group of social scientists, editors, and funders, the TOP Guidelines (https://cos.io/top) cover eight specific standards where open science can be practiced in publishing and funding (Nosek et al., 2015).”
BackgroundTransparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 46
“One of many scientific reform efforts (Munafò, 2019), open science aims to promote a shift from traditionally closed to more open research lifecycles (Miguel et al., 2014; Nosek et al., 2015).”
BackgroundInfluentialSelf-citationA manifesto for reproducible science.
Munafo, M. R., Nosek, B. A., Bishop, D., Button, K. S., Chambers, C., du Sert, N. P., Simonsohn, U., Wagenmakers, E., Ware, J. J., & Ioannidis, J. P. A. (2024) · American Psychological Association eBooks · doi:10.1037/0000409-041 · Cited by 42
Self-citationPractical tools and strategies for researchers to increase replicability
Nuijten, M. B. (2018) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.14054 · Cited by 45
Data policies of highly-ranked social science journalsPreprint
Crosas, M., Gautier, J., Karcher, S., Kirilova, D., Otalora, G., & Schwartz, A. (2018) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/9h7ay · Cited by 42
MIReAD, a minimum information standard for reporting arthropod abundance data
Rund, S. S. C., Braak, K., Cator, L. J., Copas, K., Emrich, S., Giraldo-Calderón, G. I., Johansson, M. A., Heydari, N., Hobern, D., Kelly, S. A., et al. (2019) · Scientific Data · doi:10.1038/s41597-019-0042-5 · Cited by 43
Toward assessing clinical trial publications for reporting transparency
Kilicoglu, H., Rosemblat, G., Hoang, L., Wadhwa, S., Peng, Z., Malički, M., Schneider, J., & ter Riet, G. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103717 · Cited by 34
Disentangling knowledge production and data production
Baker, K. S., & Mayernik, M. S. (2020) · Ecosphere · doi:10.1002/ecs2.3191 · Cited by 42
“Both the EPA proposed rule changes and open science advocates point to the concept of transparency as being a prime motivation (Nosek et al. 2015, Friedman 2019).”
BackgroundInstitutionalizing Open Science in Africa: Limitations and Prospects
Okafor, I. A., Mbagwu, S. I., Chia, T. I., Hasim, Z., Udokanma, E. E., & Chandran, K. (2022) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2022.855198 · Cited by 54
Kekecs, Z., Pálfi, B., Szászi, B., Szécsi, P., Zrubka, M., Kovács, M., Bakos, B. E., Cousineau, D., Tressoldi, P. E., Schmidt, K., et al. (2023) · Royal Society Open Science · doi:10.1098/rsos.191375 · Cited by 69
“Low reproducibility of important findings, growing evidence for a systematic publication bias and high prevalence of questionable research practices have increased scepticism about the trustworthiness of research reports [1,2].”
BackgroundNorsworthy, C., Dimmock, J. A., Miller, D. J., Krause, A. E., & Jackson, B. (2023) · International Journal of Applied Positive Psychology · doi:10.1007/s41042-023-00092-8 · Cited by 49
“We describe our sampling plan, all data exclusions (if any), all manipulations, and all measures in the study, and the Transparency and Openness Promotion (TOP) Guidelines (Nosek et al., 2015) were adhered to.”
MethodImproving the Efficiency of Grant and Journal Peer Review: Registered Reports Funding
Munafò, M. R. (2017) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntx081 · Cited by 34
Pedersen, M. J., & Stritch, J. M. (2018) · Public Administration Review · doi:10.1111/puar.12910 · Cited by 38
Assessing heterogeneity and power in replications of psychological experiments.
Schauer, J. M., & Hedges, L. V. (2020) · Psychological Bulletin · doi:10.1037/bul0000232 · Cited by 38
Bakker, M., Veldkamp, C. L. S., van den Akker, O. R., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., & Wicherts, J. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0236079 · Cited by 45
Are Leadership and Management Essential for Good Research? An Interview Study of Genetic Researchers
Antes, A. L., Mart, A., & DuBois, J. M. (2016) · Journal of Empirical Research on Human Research Ethics · doi:10.1177/1556264616668775 · Cited by 45
All science should inform policy and regulation
Ioannidis, J. P. A. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002576 · Cited by 33
McGuinness, L. A., & Sheppard, A. L. (2021) · PLOS ONE · doi:10.1371/journal.pone.0250887 · Cited by 33
“While requiring data availability statements is an admirable first step for journals to take, and as such is viewed favorably by journal evaluation rubrics such as the Transparency and Openness Promotion [TOP] Guidelines [5], a lack of review of the contents of these statements often leads to issues.”
BackgroundJeserich, F., Klein, C., Brinkhaus, B., & Teut, M. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289203 · Cited by 52
Raghu Raman, R. (2023) · Accountability in Research · doi:10.1080/08989621.2023.2273377 · Cited by 46
“Scientists acknowledge the importance of being transparent, open, and replicable (Nosek et al. 2015).”
Metadata
- Authors
- Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E. L., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T.
- Affiliations
- Affiliations for the authors, all of whom are members of the TOP Guidelines Committee, are given in the supplementary materials.
- Corresponding author
- Nosek, B. A.
- ORCID
- Nosek, B. A.
- Alter, G.
- Banks, G. C.
- Borsboom, D.
- Bowman, S. D.
- Breckler, S. J.
- Buck, S.
- Chambers, C. D.
- Christensen, G.
- Dafoe, A.
- Eich, E.
- Glennerster, R.
- Goroff, D.
- Hesse, B.
- Humphreys, M.
- Ishiyama, J.
- Karlan, D.
- Lupia, A.
- Mabry, P.
- Madon, T.
- Malhotra, N.
- McNutt, M.
- Miguel, E.
- Paluck, E. Levy
- Simonsohn, U.
- Soderberg, C.
- Spellman, B. A.
- VandenBos, G.
- Vazire, S.
- Wagenmakers, E. J.
- Wilson, R.
- Yarkoni, T.
- Venue
- Science
- Published
- 2015-06-26
- Volume
- 348
- Issue
- 6242
- Pages
- 1422-1425
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC4550299
- OpenAlex
- W1767470961
- Cited by
- 2,708
- References
- 15
- Citations by year
- 2026: 214 · 2025: 203 · 2024: 200 · 2023: 232 · 2022: 278 · 2021: 306 · 2020: 360 · 2019: 276 · 2018: 256 · 2017: 211 · 2016: 134 · 2015: 38
- FWCI
- 418.98
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- PubMed Central (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open research culture, author guidelines, journal guidelines, transparency, research transparency, reproducibility
- MeSH terms
- Periodicals as Topic, Research, Reproducibility of Results, Guidelines as Topic, Peer Review, Research, Access to Information, Disclosure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.