Promoting an open research culture
Open accessPublished version of a preprint
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Author guidelines for journals could help to promote transparency, openness, and reproducibility
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, many are closed source applications with black box algorithms, which is problematic as transparency and data ownership are essential in the era of open science 22 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“Open science will clearly help in this regard (Nosek et al., 2015).”
From Intrusive memories of trauma: A target for research bridging cognitive science and its clinical application (2019) · cites this paper as Nosek et al., 2015
In the citing paper
“These issues may be strongly suspected and their magnitude can sometimes be estimated 63 , but cannot truly be corrected 64,65 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Self-citationRecommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationRegistered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Self-citationPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citationPublication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Self-citation
Cited by
Items 101–200 of 1,016
Musical Preferences Predict Personality: Evidence From Active Listening and Facebook Likes
Nave, G., Minxha, J., Greenberg, D. M., Kosinski, M., Stillwell, D., & Rentfrow, J. (2018) · Psychological Science · doi:10.1177/0956797618761659 · Cited by 206
Exploration, Inference, and Prediction in Neuroscience and Biomedicine
Bzdok, D., & Ioannidis, J. P. A. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2019.02.001 · Cited by 204
“This step is important to combat reproducibility issues [67].”
BackgroundRecommendations for Increasing the Transparency of Analysis of Preexisting Data Sets
Weston, S. J., Ritchie, S. J., Rohrer, J. M., & Przybylski, A. K. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919848684 · Cited by 203
“To address issues of reproducibility and replicability, many scientists have advocated for the broad adoption of open science values and practices (e.g., Klein et al., 2014; Nosek et al., 2015), most often implemented through disclosure and transparency in various forms.”
BackgroundCONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials
Hopewell, S., Chan, A.-W., Collins, G. S., Hróbjartsson, A., Moher, D., Schulz, K. F., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081124 · Cited by 205
Transparency in Ecology and Evolution: Real Problems, Real Solutions
Parker, T. H., Forstmeier, W., Koricheva, J., Fidler, F., Hadfield, J. D., Chee, Y. E., Kelly, C. D., Gurevitch, J., & Nakagawa, S. (2016) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2016.07.002 · Cited by 201
“This is an historic crossroads for the practice of science in ecology and evolutionary biology, and for empirical disciplines in general [45].”
BackgroundField Experiments Across the Social Sciences
Baldassarri, D., & Abascal, M. (2017) · Annual Review of Sociology · doi:10.1146/annurev-soc-073014-112445 · Cited by 196
“…to replicate numerous others (Nosek et al. 2015a) show.16 More than other methods, however, experiments are subject to strong—and intensifying (Nosek et al. 2015b)—norms of preregistration and data sharing, particularly in those fields, such as economics and political science, where the…”
BackgroundTransparency and Reproducibility of Observational Cohort Studies Using Large Healthcare Databases.
Wang, S., Verpillat, P., Rassen, J., Patrick, A., Garry, E., & Bartels, D. (2016) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.329 · Cited by 197
A large-scale test of the link between intergroup contact and support for social change
Hässler, T., Ullrich, J., Bernardino, M., Shnabel, N., Laar, C. V., Valdenegro, D., Sebben, S., Tropp, L. R., Visintin, E. P., González, R., et al. (2020) · Nature Human Behaviour · doi:10.1038/s41562-019-0815-z · Cited by 195
Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020) · Royal Society Open Science · doi:10.1098/rsos.190806 · Cited by 217
“A cascade of reform initiatives have been launched in an effort to improve transparency [21,22], and close monitoring and evaluation of specific reform initiatives will help to identify efforts that are the most successful and highlight areas for improvement [25,26].”
BackgroundHardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620979806 · Cited by 187
Inference in the age of big data: Future perspectives on neuroscience
Bzdok, D., & Yeo, B. T. T. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.061 · Cited by 189
“In our opinion, the widespread adoption of script programming is likely to propel high-throughput statistical analysis, improve provenance tracking and reproducibility (Nosek et al., 2015), hence accelerating the pace of neuroscientific knowledge production.”
Evaluating the popularity of R in ecology
Lai, J., Lortie, C. J., Muenchen, R. A., Yang, J., & Ma, K. (2019) · Ecosphere · doi:10.1002/ecs2.2567 · Cited by 189
Nonreplicable publications are cited more than replicable ones
Serra-Garcia, M., & Gneezy, U. (2021) · Science Advances · doi:10.1126/sciadv.abd1705 · Cited by 177
“INTRODUCTION The replication crisis in social sciences refers to the failure to replicate a large fraction of published experiments (1) and the selective publication of results and specifications (2–4).”
BackgroundOpen data and digital morphology
Davies, T. G., Rahman, I. A., Lautenschlager, S., Cunningham, J. A., Asher, R. J., Barrett, P. M., Bates, K. T., Bengtson, S., Benson, R. B. J., Boyer, D. M., et al. (2017) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2017.0194 · Cited by 181
Wang, S. V., Schneeweiss, S., Berger, M. L., Brown, J., de Vries, F., Douglas, I., Gagne, J. J., Gini, R., Klungel, O., Mullins, C. D., et al. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4295 · Cited by 182
Reducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E.-J. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 179
Self-citationBehavior change interventions: the potential of ontologies for advancing science and practice
Larsen, K. R., Michie, S., Hekler, E. B., Gibson, B., Spruijt-Metz, D., Ahern, D., Cole-Lewis, H., Ellis, R. J. B., Hesse, B., Moser, R. P., et al. (2017) · Journal of Behavioral Medicine · doi:10.1007/s10865-016-9768-0 · Cited by 178
“When levels of evidence are too low for drawing conclusions, ontologies can ease the path toward gathering new evidence relative to the replicability and generalizability of findings (Nosek et al, 2015).”
BackgroundSelf-citationA Trauma-Informed Approach to Sexual Violence Research Ethics and Open Science
Campbell, R., Goodman-Williams, R., & Javorka, M. (2019) · Journal of Interpersonal Violence · doi:10.1177/0886260519871530 · Cited by 175
“throughout academia to make science more transparent and accessible by redefining expectations for dissemination (Nosek et al., 2015).”
BackgroundInfluentialScientific Utopia III: Crowdsourcing Science
Uhlmann, E. L., Ebersole, C. R., Chartier, C. R., Errington, T. M., Kidwell, M. C., Lai, C. K., McCarthy, R. J., Riegelman, A., Silberzahn, R., & Nosek, B. A. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619850561 · Cited by 174
“Top-down changes in publication requirements at journals (e.g., disclosure rules and open-science badges) are already changing how science is done and what gets published (Everett & Earp, 2015; Nosek et al., 2015).”
BackgroundSelf-citationEditorial: Evidence on Questionable Research Practices: The Good, the Bad, and the Ugly
Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., & Rupp, D. E. (2016) · Journal of Business and Psychology · doi:10.1007/s10869-016-9456-7 · Cited by 173
“Further, journals may consider policies that promote open science and sharing among researchers by following the Transparency and Openness Guidelines (Nosek et al. 2015).”
BackgroundSelf-citationIoannidis, J. P. A., Kim, B. Y. S., & Trounson, A. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0314-y · Cited by 175
Digital neuropsychology: challenges and opportunities at the intersection of science and software
Germine, L., Reinecke, K., & Chaytor, N. S. (2019) · The Clinical Neuropsychologist · doi:10.1080/13854046.2018.1535662 · Cited by 171
“…dedicated to open source software and related solutions — companies such as the nonprofit Center for Open Science (Foster & Deardorff, 2017; Nosek et al., 2015) and Sage Bionetworks (Bot et al., 2016; Wilbanks & Friend, 2016) — have demonstrated it is possible to build viable business…”
BackgroundMaertens, R., Anseel, F., & van der Linden, S. (2020) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2020.101455 · Cited by 167
Linardon, J., Messer, M., Rodgers, R. F., & Fuller‐Tyszkiewicz, M. (2022) · International Journal of Eating Disorders · doi:10.1002/eat.23640 · Cited by 169
Enhancing Rigor in Quantitative Entrepreneurship Research
Maula, M., & Stam, W. (2020) · Entrepreneurship Theory and Practice · doi:10.1177/1042258719891388 · Cited by 167
“prevailing null hypothesis significance testing (NHST) paradigm (e.g., Gigerenzer, 2004; Matthews, 2019; McShane et al., 2019; Nosek et al., 2015; Schwab et al., 2011; Wasserstein et al., 2019),1 rapid developments in empirical methods and software tools also create new opportunities for…”
BackgroundInfluentialNavigating the garden of forking paths for data exclusions in fear conditioning research
Lonsdorf, T. B., Klingelhöfer-Jens, M., Andreatta, M., Beckers, T., Chalkia, A., Gerlicher, A., Jentsch, V. L., Meir Drexler, S., Mertens, G., Richter, J., et al. (2019) · eLife · doi:10.7554/elife.52465 · Cited by 164
The New Statistics for Better Science: Ask How Much, How Uncertain, and What Else Is Known
Calin-Jageman, R. J., & Cumming, G. (2019) · The American Statistician · doi:10.1080/00031305.2018.1518266 · Cited by 166
“We are brief here because others have discussed the benefits of Open Science with more detail and authority (e.g., Nosek et al. 2015, 2018). argued that p-values are no more problematic than any other approach, because all approaches can be used mindlessly.”
BackgroundAssessing data availability and research reproducibility in hydrology and water resources
Stagge, J. H., Rosenberg, D. E., Abdallah, A. M., Akbar, H., Attallah, N. A., & James, R. (2019) · Scientific Data · doi:10.1038/sdata.2019.30 · Cited by 165
Questions About Questionable Research Practices in the Field of Management: A Guest Commentary
Banks, G. C., O’Boyle, E. H., Pollack, J. M., White, C. D., Batchelor, J. H., Whelpley, C. E., Abston, K. A., Bennett, A. A., & Adkins, C. L. (2016) · Journal of Management · doi:10.1177/0149206315619011 · Cited by 163
“Adding to concerns about the veracity of management research is evidence that our field and closely related fields may be susceptible to questionable research practices (QRPs; Bosco, Aguinis, Field, Pierce, & Dalton, in press; Francis, Tanzman, & Matthews, 2014; Franco, Malhotra, & Simonovits, 2014; O’Boyle, Banks, &…”
BackgroundInfluentialSelf-citationFrith, U. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.10.007 · Cited by 176
Gerstner, K., Moreno‐Mateos, D., Gurevitch, J., Beckmann, M., Kambach, S., Jones, H. P., & Seppelt, R. (2017) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12758 · Cited by 159
Rutjens, B. T., Heine, S. J., Sutton, R. M., & van Harreveld, F. (2018) · Advances in Experimental Social Psychology · doi:10.1016/bs.aesp.2017.08.001 · Cited by 146
“Most prominent among these efforts is the open science movement, in which researchers and gatekeepers are increasingly adopting, advocating, and requiring open research practices, including preregistration of hypotheses and full disclosure of materials and results (e.g., Nosek et al., 2015).”
Sikkes, S. A. M., Tang, Y., Jutten, R. J., Wesselman, L. M. P., Turkstra, L. S., Brodaty, H., Clare, L., Cassidy‐Eagle, E. L., Cox, K. L., Chételat, G., et al. (2020) · Alzheimer s & Dementia · doi:10.1002/alz.12188 · Cited by 171
A consensus-based transparency checklist
Aczel, B., Szaszi, B., Sarafoglou, A., Kekecs, Z., Kucharský, Š., Benjamin, D., Chambers, C. D., Fisher, A., Gelman, A., Gernsbacher, M. A., et al. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0772-6 · Cited by 152
Self-citationHalbritter, A. H., Boeck, H. J. D., Eycott, A. E., Reinsch, S., Robinson, D. A., Vicca, S., Berauer, B. J., Christiansen, C. T., Estiarte, M., Grünzweig, J. M., et al. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13331 · Cited by 158
Sharing Is Caring—Data Sharing Initiatives in Healthcare
Hulsen, T. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17093046 · Cited by 162
“In 2015, the Transparency and Openness Promotion (TOP) guidelines [21] were published.”
BackgroundReproducible and replicable pain research: a critical review
Lee, H., Lamb, S. E., Bagg, M. K., Toomey, E., Cashin, A. G., & Moseley, G. L. (2018) · Pain · doi:10.1097/j.pain.0000000000001254 · Cited by 150
“Finally, we contend that pain journals should sign the Transparency and Openness Promotion (TOP) Guidelines [48] - a set of standards intended to assist journals steward transparent scientific practices.”
BackgroundReplication in Criminology and the Social Sciences
Pridemore, W. A., Makel, M. C., & Plucker, J. A. (2017) · Annual Review of Criminology · doi:10.1146/annurev-criminol-032317-091849 · Cited by 169
Answers to 18 Questions About Open Science Practices
Banks, G. C., Field, J. G., Oswald, F. L., O’Boyle, E. H., Landis, R. S., Rupp, D. E., & Rogelberg, S. G. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9547-8 · Cited by 178
“Second, the sharing of design protocols, measures, and analytic scripts can help to improve the rigor of study designs (Nosek et al., 2015) as well as reproducibility and replication rates of success (Open Science Collaboration, 2015; Schmidt & Oh, 2016).”
InfluentialSelf-citationA checklist to assess the quality of survey studies in psychology
Protogerou, C., & Hagger, M. S. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100031 · Cited by 187
REFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 129
“The Transparency and Openness Promotion standards introduced by the Open Science Foundation ( 93 ) have a few sections that focus on computational reproducibility in scientific research.”
THE AGORA HIGH-RESOLUTION GALAXY SIMULATIONS COMPARISON PROJECT. II. ISOLATED DISK TEST
Kim, J.-H., Agertz, O., Teyssier, R., Butler, M. J., Ceverino, D., Choi, J.-H., Feldmann, R., Keller, B., Lupi, A., Quinn, T., et al. (2016) · The Astrophysical Journal · doi:10.3847/1538-4357/833/2/202 · Cited by 170
Zetsche, U., Bürkner, P.-C., & Schulze, L. (2018) · Clinical Psychology Review · doi:10.1016/j.cpr.2018.06.001 · Cited by 142
Self-correction in science at work
Alberts, B., Cicerone, R. J., Fienberg, S. E., Kamb, A., McNutt, M., Nerem, R. M., Schekman, R., Shiffrin, R., Stodden, V., Suresh, S., et al. (2015) · Science · doi:10.1126/science.aab3847 · Cited by 143
Self-citationMayo-Wilson, E., Fusco, N., Li, T., Hong, H., Canner, J. K., & Dickersin, K. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.007 · Cited by 131
Self-citationDynamics of cumulative advantage and threats to equity in open science: a scoping review
Ross‐Hellauer, T., Reichmann, S., Cole, N. L., Fessl, A., Klebel, T., & Pontika, N. (2022) · Royal Society Open Science · doi:10.1098/rsos.211032 · Cited by 206
“Open Science and in particular open methods, which involve practices like sharing analysis code, laboratory notebooks or preregistering analyses, hold the promise to counter current concerns regarding integrity in reporting and the reproducibility of research [44,159,160].”
BackgroundEfficacy of Personalized Psychological Interventions: A Systematic Review and Meta-Analysis
Nye, A., Delgadillo, J., & Barkham, M. (2023) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000820 · Cited by 142
Choosing the negative: A behavioral demonstration of morbid curiosity
Oosterwijk, S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178399 · Cited by 150
Four simple recommendations to encourage best practices in research software
Jiménez, R. C., Kuzak, M., Alhamdoosh, M., Barker, M., Batut, B., Borg, M., Capella-Gutierrez, S., Chue Hong, N., Cook, M., Corpas, M., et al. (2017) · F1000Research · doi:10.12688/f1000research.11407.1 · Cited by 140
Chen, M., Voinov, A. A., Ames, D. P., Kettner, A. J., Goodall, J. L., Jakeman, A. J., Barton, C. M., Harpham, Q., Cuddy, S., DeLuca, C., et al. (2020) · Earth-Science Reviews · doi:10.1016/j.earscirev.2020.103223 · Cited by 139
von Kortzfleisch, V. T., Karp, N. A., Palme, R., Kaiser, S., Sachser, N., & Richter, S. H. (2020) · Scientific Reports · doi:10.1038/s41598-020-73503-4 · Cited by 154
Assessment of transparency indicators across the biomedical literature: How open is open?
Serghiou, S., Contopoulos-Ioannidis, D. G., Boyack, K. W., Riedel, N., Wallach, J. D., & Ioannidis, J. P. A. (2021) · PLOS Biology · doi:10.1371/journal.pbio.3001107 · Cited by 139
The structure of the General Health Questionnaire (GHQ-12): two meta-analytic factor analyses
Gnambs, T., & Staufenbiel, T. (2018) · Health Psychology Review · doi:10.1080/17437199.2018.1426484 · Cited by 155
“To promote transparency and reproducibility of our analyses (see Nosek et al., 2015), all coded data and analyses scripts are provided in an online repository at http://osf.io/z5c4q/.”
MethodReproducible Research and GIScience: An Evaluation Using GIScience Conference Papers
Ostermann, F. O., Nüst, D., Granell, C., Hofer, B., & Konkol, M. (2018) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2021.ii.2 · Cited by 135
Robust Modeling in Cognitive Science
Lee, M. D., Criss, A. H., Devezer, B., Donkin, C., Etz, A., Leite, F. P., Matzke, D., Rouder, J. N., Trueblood, J. S., White, C. N., et al. (2019) · Computational Brain & Behavior · doi:10.1007/s42113-019-00029-y · Cited by 130
Computing environments for reproducibility: Capturing the “Whole Tale”
Brinckman, A., Chard, K., Gaffney, N., Hategan, M., Jones, M. B., Kowalik, K., Kulasekaran, S., Ludäscher, B., Mecum, B. D., Nabrzyski, J., et al. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2017.12.029 · Cited by 132
Promises and pitfalls of data sharing in qualitative research
Tsai, A. C., Kohrt, B. A., Matthews, L. T., Betancourt, T. S., Lee, J. K., Papachristos, A. V., Weiser, S. D., & Dworkin, S. L. (2016) · Social Science & Medicine · doi:10.1016/j.socscimed.2016.08.004 · Cited by 132
“The movement for research transparency has gained irresistible momentum over the past decade (Groves, 2010; Hanson et al., 2011; Laine et al., 2007; Miguel et al., 2014; Nosek et al., 2015; Peng et al., 2006; PLOS Medicine Editors, 2014; Stodden et al., 2013; Tsai, 2011).”
BackgroundThe Production of Knowledge: Enhancing Progress in Social Science
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108762519 · Cited by 241
Holden, M. A., Hattle, M., Runhaar, J., Riley, R. D., Healey, E. L., Quicke, J. G., van der Windt, D., Dziedzic, K., van Middelkoop, M., Burke, D. L., et al. (2023) · The Lancet Rheumatology · doi:10.1016/s2665-9913(23)00122-4 · Cited by 160
Czoty, P. W., Stoops, W. W., & Rush, C. R. (2016) · Pharmacological Reviews · doi:10.1124/pr.115.011668 · Cited by 144
“The NIH recently expressed a commitment to enhancing the rigor and reproducibility of research findings (http://www.nih. gov/science/reproducibility/principles-guidelines.htm) that has been embraced by the major scientific publishing companies (Nosek et al., 2015).”
BackgroundParticle Targeting in Complex Biological Media
Dai, Q., Bertleff‐Zieschang, N., Braunger, J. A., Björnmalm, M., Cortez‐Jugo, C., & Caruso, F. (2017) · Advanced Healthcare Materials · doi:10.1002/adhm.201700575 · Cited by 139
Academic Misconduct, Misrepresentation and Gaming: A ReassessmentPreprint
Biagioli, M., Kenney, M. F., Martin, B. R., & Walsh, J. P. (2018) · SSRN Electronic Journal · doi:10.1016/j.respol.2018.10.025 · Cited by 146
Irreproducibility in Preclinical Biomedical Research: Perceptions, Uncertainties, and Knowledge Gaps
Jarvis, M. F., & Williams, M. (2016) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2015.12.001 · Cited by 129
“TOP Transparency and Openness Promotion Guidelines Center for Open Science guidelines for journal standards [90]”
Backgroundvan Rentergem, J. A. A., Deserno, M. K., & Geurts, H. M. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102033 · Cited by 143
“Preregistration of one ’ s hypotheses and data-analysis plan, through platforms such as AsPredicted.org or the Open Science Framework, are a promising way forward in increasing such transparency (Nosek et al., 2015).”
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education
Adelson, J. L., Barton, E., Bradshaw, C., Bryant, B. R., Bryant, D., Cook, B. G., Coyne, M., DeBettencourt, L., DeHaven, A. C., Dymond, S., et al. (2018) · Exceptional Children · doi:10.1177/0014402918793138 · Cited by 143
“For example, the Transparency and Openness Promotion (TOP) Guidelines provide a policy framework for journals, funders, and institutions to increase transparency and credibility of research (Nosek et al., 2015).”
BackgroundInfluentialSelf-citationAspiring to greater intellectual humility in science
Hoekstra, R., & Vazire, S. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01203-8 · Cited by 117
Self-citationImproving the reporting quality of reliability generalization meta‐analyses: The REGEMA checklist
Sánchez‐Meca, J., Marín‐Martínez, F., López-López, J. A., Núñez, R. M. N., Rubio‐Aparicio, M., García, J. J. L., Pina, J. A. L., Blázquez-Rincón, D., López‐Ibáñez, C., & López‐Nicolás, R. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1487 · Cited by 144
“Furthermore, poor reporting quality hampers replicability and goes against the principle of transparency promoted by the Open Science Framework.29 A systematic review of 150 RG meta-analyses conducted on psychological tests revealed important deficiencies in the reporting quality, especially regarding key methodologic…”
BackgroundInfluentialReplication is important for educational psychology: Recent developments and key issues
Plucker, J. A., & Makel, M. C. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1895796 · Cited by 144
The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing
Lash, T. L. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kwx261 · Cited by 134
Making Open Science Work for Science and Society
Elliott, K. C., & Resnik, D. B. (2019) · Environmental Health Perspectives · doi:10.1289/ehp4808 · Cited by 141
“The open science movement has received a great deal of attention from governments, policy makers, and scientists around the globe (European Commission 2014; NAS 2018; Nosek et al. 2015; Royal Society 2012).”
BackgroundInfluentialIncreasing the Transparency of Research Papers with Explorable Multiverse Analyses
Dragicevic, P., Jansen, Y., Sarma, A., Kay, M., & Chevalier, F. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300295 · Cited by 162
“Much of this movement calls for more transparency in the way statistics are reported, so that findings become more trustworthy, more likely to be interpreted correctly, and easier to verify and replicate [29, 69, 72].”
BackgroundBerry, D. R., Hoerr, J. P., Cesko, S., Alayoubi, A., Carpio, K., Zirzow, H., Walters, W., Scram, G., Rodriguez, K., & Beaver, V. (2020) · Personality and Social Psychology Bulletin · doi:10.1177/0146167219900418 · Cited by 123
“The hypothesis, eligibility criteria, and analytic plan were preregistered on the Open Science Framework (Berry, 2019) after the literature search began but prior to analyses.”
BackgroundInfluentialA meta-analysis of conditioned fear generalization in anxiety-related disorders
Cooper, S. E., van Dis, E. A. M., Hagenaars, M. A., Krypotos, A., Nemeroff, C. B., Lissek, S. M., Engelhard, I. M., & Dunsmoor, J. E. (2022) · Neuropsychopharmacology · doi:10.1038/s41386-022-01332-2 · Cited by 158
“Our QA found some trends that merit discussion, particularly in the context of increasing open science practices [66].”
BackgroundTheory and practice of data citation
Silvello, G. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23917 · Cited by 141
“Moreover, the scientific community is taking action to promote an open research culture (Nosek et al., 2015) and effective means to share, discover, and access scientific data that too often still remain “hidden” at their origin or are shared in suboptimal ways (Ohno-machado et al., 2015).”
BackgroundCognitive Strategies and Natural Environments Interact in Influencing Executive Function
Bourrier, S. C., Berman, M. G., & Enns, J. T. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01248 · Cited by 130
“Also, in keeping with the current renewed emphasis on replicability and generalization in the behavioral sciences (Nosek et al., 2015), this experiment was important in establishing the robustness of the nature-executive functioning link.”
BackgroundHow to run an experimental auction: a review of recent advances
Canavari, M., Drichoutis, A. C., Lusk, J. L., & Nayga, R. M. (2019) · European Review of Agricultural Economics · doi:10.1093/erae/jbz038 · Cited by 140
“Olken (2015) summarizes the benefits and risks of committing to a pre-analysis plan. Researchers who put a lot of detail and effort in coming up with a pre-analysis plan could benefit from the careful thought processes required of going through their data analysis, and selecting which variables to collect and methods…”
BackgroundMethodNo Replication, No Trust? How Low Replicability Influences Trust in Psychology
Wingen, T., Berkessel, J. B., & Englich, B. (2019) · Social Psychological and Personality Science · doi:10.1177/1948550619877412 · Cited by 154
“Central aspects of this movement, such as preregistrations, open data, and open materials, aim to increase the transparency of psychological research (Miguel et al., 2014; Nosek et al., 2015).”
BackgroundKoltes, J. E., Cole, J. B., Clemmens, R., Dilger, R. N., Kramer, L., Lunney, J. K., McCue, M. E., McKay, S., Mateescu, R. G., Murdoch, B. M., et al. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.01197 · Cited by 130
“These restrictions are a challenge to the research community because, under Open Science initiatives, e.g., (Nosek et al., 2015), a growing number of funding agencies and journals require that the data used in a study be made publicly available.”
BackgroundTransparency of CHI Research Artifacts: Results of a Self-Reported Survey
Wacharamanotham, C., Eisenring, L., Haroz, S., & Echtler, F. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376448 · Cited by 118
“The Transparency and Openness Promotion (TOP) guidelines provide policy templates with multiple levels of strictness for three categories of research practices [36].”
BackgroundStrickland, J. C., Lile, J. A., & Stoops, W. W. (2017) · Behavioural Processes · doi:10.1016/j.beproc.2017.03.017 · Cited by 128
“Replication studies such as this one are particularly important given recent challenges the psychological and behavioral sciences have faced regarding reproducibility (Nosek et al., 2015; Open Source Data Collection, 2015).”
BackgroundOlaru, G., Schroeders, U., Hartung, J., & Wilhelm, O. (2019) · European Journal of Personality · doi:10.1002/per.2195 · Cited by 128
“To foster the utility of ACO as an item-sampling method and LSEM as a person-sampling method, we provide the commented R scripts used in this tutorial in an online repository on the Open Science Framework (Nosek et al., 2015): https://osf.io/yx4km/.”
MethodCoyne, M. D., Cook, B. G., & Therrien, W. J. (2016) · Remedial and Special Education · doi:10.1177/0741932516648463 · Cited by 128
Camerer, C., Dreber, A., Holzmeister, F., Ho, T. H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · doi:10.31235/osf.io/4hmb6 · Cited by 117
Self-citationCommunity geography: Toward a disciplinary framework
Shannon, J., Hankins, K., Shelton, T., Bosse, A. J., Scott, D., Block, D., Fischer, H. A., Eaves, L. E., Jung, J.-K., Robinson, J., et al. (2020) · Progress in Human Geography · doi:10.1177/0309132520961468 · Cited by 132
Reproducible research practices are underused in systematic reviews of biomedical interventions
Page, M. J., Altman, D. G., Shamseer, L., McKenzie, J. E., Ahmadzai, N., Wolfe, D., Yazdi, F., Catalá-López, F., Tricco, A. C., & Moher, D. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.017 · Cited by 115
The Structure of the Rosenberg Self-Esteem Scale: A Cross-Cultural Meta-Analysis
Gnambs, T., Scharl, A., & Schroeders, U. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000317 · Cited by 159
Mastrotheodoros, S., Canário, C., Gugliandolo, M. C., Merkaš, M., & Keijsers, L. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01094-z · Cited by 137
Toward standard practices for sharing computer code and programs in neuroscience
Eglen, S. J., Marwick, B., Halchenko, Y. O., Hanke, M., Sufi, S., Gleeson, P., Silver, R. A., Davison, A. P., Lanyon, L., Abrams, M. B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4550 · Cited by 128
Nakagawa, S., & Parker, T. H. (2015) · BMC Biology · doi:10.1186/s12915-015-0196-3 · Cited by 110
Interpersonal Movement Synchrony Responds to High- and Low-Level Conversational Constraints
Paxton, A., & Dale, R. A. C. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01135 · Cited by 127
“Current best practices for open science include the sharing of data and code in public repositories (see Nosek et al., 2015; Blohowiak et al., 2016; Gewin, 2016; Kidwell et al., 2016).”
BackgroundRevisiting and expanding the meta‐analysis of variation: The log coefficient of variation ratio
Senior, A. M., Viechtbauer, W., & Nakagawa, S. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1423 · Cited by 127
“this regard, it is vital that meta-analytic “raw” data are made available and reusable in the spirit of open and transparent science.(42,43)”
MethodTowards wide-scale adoption of open science practices: The role of open science communities
Armeni, K., Brinkman, L., Carlsson, R., Eerland, A., Fijten, R., Fondberg, R., Heininga, V. E., Heunis, S., Koh, W. Q., Masselink, M., et al. (2021) · Science and Public Policy · doi:10.1093/scipol/scab039 · Cited by 140
Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy
Mayo-Wilson, E., Li, T., Fusco, N., Bertizzolo, L., Canner, J. K., Cowley, T., Doshi, P., Ehmsen, J., Gresham, G., Guo, N., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.07.014 · Cited by 108
Self-citationGuidelines for ethical use and acknowledgement of large language models in academic writing
Porsdam Mann, S., Vazirani, A. A., Aboy, M., Earp, B. D., Minssen, T., Cohen, I. G., & Savulescu, J. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00922-7 · Cited by 108
Wang, S. V., Schneeweiß, S., Berger, M. L., Brown, J. S., de Vries, F., Douglas, I., Gagne, J. J., Gini, R., Klungel, O. H., Mullins, C. D., et al. (2017) · Value in Health · doi:10.1016/j.jval.2017.08.3018 · Cited by 126
Reproducible research and GIScience: an evaluation using AGILE conference papers
Nüst, D., Granell, C., Hofer, B., Konkol, M., Ostermann, F., Šileryté, R., & Cerutti, V. (2018) · PeerJ · doi:10.7717/peerj.5072 · Cited by 137
Bishop, D. V. (2020) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021819886519 · Cited by 108
“First, there have been calls for greater openness and transparency (Nosek et al., 2015).”
BackgroundPackaging Data Analytical Work Reproducibly Using R (and Friends)
Marwick, B., Boettiger, C., & Mullen, L. A. (2017) · The American Statistician · doi:10.1080/00031305.2017.1375986 · Cited by 123
“Authors submitting to Political Analysis and the Quarterly Journal of Political Science are required to provide their code for review, with editors reproducing the reported analyses prior to publication (Nosek et al. 2015).”
Five selfish reasons to work reproducibly
Markowetz, F. (2015) · Genome Biology · doi:10.1186/s13059-015-0850-7 · Cited by 106
van der Zee, T., & Reich, J. (2018) · AERA Open · doi:10.1177/2332858418787466 · Cited by 134
“Preregistration is the practice of documenting and sharing the hypotheses, methodology, analytic plans, and other relevant aspects of a study before it is conducted (Gehlbach & Robinson, 2018; Nosek et al., 2015).”
Method
Metadata
- Authors
- Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E. L., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T.
- Affiliations
- Affiliations for the authors, all of whom are members of the TOP Guidelines Committee, are given in the supplementary materials.
- Corresponding author
- Nosek, B. A.
- ORCID
- Nosek, B. A.
- Alter, G.
- Banks, G. C.
- Borsboom, D.
- Bowman, S. D.
- Breckler, S. J.
- Buck, S.
- Chambers, C. D.
- Christensen, G.
- Dafoe, A.
- Eich, E.
- Glennerster, R.
- Goroff, D.
- Hesse, B.
- Humphreys, M.
- Ishiyama, J.
- Karlan, D.
- Lupia, A.
- Mabry, P.
- Madon, T.
- Malhotra, N.
- McNutt, M.
- Miguel, E.
- Paluck, E. Levy
- Simonsohn, U.
- Soderberg, C.
- Spellman, B. A.
- VandenBos, G.
- Vazire, S.
- Wagenmakers, E. J.
- Wilson, R.
- Yarkoni, T.
- Venue
- Science
- Published
- 2015-06-26
- Volume
- 348
- Issue
- 6242
- Pages
- 1422-1425
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC4550299
- OpenAlex
- W1767470961
- Cited by
- 2,708
- References
- 15
- Citations by year
- 2026: 214 · 2025: 203 · 2024: 200 · 2023: 232 · 2022: 278 · 2021: 306 · 2020: 360 · 2019: 276 · 2018: 256 · 2017: 211 · 2016: 134 · 2015: 38
- FWCI
- 418.98
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- PubMed Central (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open research culture, author guidelines, journal guidelines, transparency, research transparency, reproducibility
- MeSH terms
- Periodicals as Topic, Research, Reproducibility of Results, Guidelines as Topic, Peer Review, Research, Access to Information, Disclosure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.