Promoting an open research culture
Open accessPublished version of a preprint
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Author guidelines for journals could help to promote transparency, openness, and reproducibility
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, many are closed source applications with black box algorithms, which is problematic as transparency and data ownership are essential in the era of open science 22 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“Open science will clearly help in this regard (Nosek et al., 2015).”
From Intrusive memories of trauma: A target for research bridging cognitive science and its clinical application (2019) · cites this paper as Nosek et al., 2015
In the citing paper
“These issues may be strongly suspected and their magnitude can sometimes be estimated 63 , but cannot truly be corrected 64,65 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Self-citationRecommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationRegistered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Self-citationPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citationPublication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Self-citation
Cited by
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
“Scientific incentives -- funding, publication, awards -- can be tuned to encourage an optimal balance in the collective effort of discovery (36, 37).”
O'Leary, N. A., Wright, M. W., Brister, J. R., Ciufo, S., Haddad, D., McVeigh, R., Rajput, B., Robbertse, B., Smith-White, B., Ako-Adjei, D., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1189 · Cited by 6,357
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
“The Transparency and Openness Promotion (TOP) guidelines offer standards as a basis for journals and funders to incentivize or require greater transparency in planning and reporting of research 54 .”
Self-citationMeta-analysis and the science of research synthesis
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018) · Nature · doi:10.1038/nature25753 · Cited by 1,709
“Open science practices have been strongly advocated particularly in the past few years to facilitate full access to scientific data, particularly in an unbiased manner 79 .”
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,627
Self-citationCamerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521
Self-citationScanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410
Self-citationEvaluating replicability of laboratory experiments in economics
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., et al. (2016) · Science · doi:10.1126/science.aaf0918 · Cited by 1,314
Increasing Transparency Through a Multiverse Analysis
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616658637 · Cited by 1,372
Bayesian statistics and modelling
van de Schoot, R., Depaoli, S., King, R., Kramer, B., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J., et al. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-020-00001-2 · Cited by 1,176
An open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
“Transparent methods and results are essential to generating reproducible research and interpreting research findings and have recently become a topic of renewed interest among research methodologists (e.g., Nosek et al. 2015).”
MethodMany Labs 2: Investigating variation in replicability across samples and settings
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918810225 · Cited by 944
“In any case, these findings provide further justification for improving the transparency of research (Miguel et al., 2014; Nosek et al., 2015), preregistering studies to make all findings discoverable even if they are not published, and preregistering analysis plans to make clear the distinction between confirmatory t…”
BackgroundThe ecological importance of intraspecific variation
Des Roches, S., Post, D. M., Turley, N. E., Bailey, J. K., Hendry, A. P., Kinnison, M. T., Schweitzer, J. A., & Palkovacs, E. P. (2017) · Nature Ecology & Evolution · doi:10.1038/s41559-017-0402-5 · Cited by 916
The contact hypothesis re-evaluated
PALUCK, E. L., GREEN, S. A., & GREEN, D. P. (2019) · Behavioural Public Policy · doi:10.1017/bpp.2018.25 · Cited by 856
Self-citationWicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01832 · Cited by 754
“The preferred way to assure reproducibility is to share data and analytic details (computer syntaxes/code) in or alongside the paper (Nosek et al., 2015).”
BackgroundReplicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Self-citationSzucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761
People are averse to machines making moral decisions.
Bigman, Y. E., & Gray, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.08.003 · Cited by 738
“Studies 7 and 8 are relatively inconclusive and should be interpreted with caution, but are reported for full transparency and to guard against the “file drawer,” consistent with open science practices (Nosek et al., 2015).”
Minimum information reporting in bio–nano experimental literature
Faria, M., Björnmalm, M., Thurecht, K. J., Kent, S. J., Parton, R. G., Kavallaris, M., Johnston, A. P. R., Gooding, J. J., Corrie, S. R., Boyd, B. J., et al. (2018) · Nature Nanotechnology · doi:10.1038/s41565-018-0246-4 · Cited by 710
Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis
Shrout, P. E., & Rodgers, J. L. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011845 · Cited by 725
“The OSF provides a pragmatic infrastructure for in studies (De Angelis et al. 2005).”
BackgroundInfluentialThe GRoLTS-Checklist: Guidelines for Reporting on Latent Trajectory Studies
van de Schoot, R., Sijbrandij, M., Winter, S. D., Depaoli, S., & Vermunt, J. K. (2017) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2016.1247646 · Cited by 721
Smith, J. D., Li, D. H., & Rafferty, M. R. (2020) · Implementation Science · doi:10.1186/s13012-020-01041-8 · Cited by 706
O'Dea, R. E., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Gurevitch, J., Page, M. J., Stewart, G., Moher, D., et al. (2021) · Biological Reviews · doi:10.1111/brv.12721 · Cited by 697
“Evidence from other fields suggests that data and/or code provided by authors often do not reproduce the results presented in a paper (Nosek et al., 2015; Stodden, Seiler& Ma, 2018).”
BackgroundConstruct Validation in Social and Personality Research: Current Practice and Recommendations
Flake, J. K., Pek, J., & Hehman, E. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617693063 · Cited by 697
“…testing, the reporting of effect sizes and their confidence intervals (Cumming, 2014; Wilkinson & Task Force on Statisical Inference, 1999), data sharing and conducting replications (Nosek et al., 2015; Open Science Collaboration, 2015), and the preregistration of studies (Moore, 2016).”
BackgroundKidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., et al. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002456 · Cited by 700
Self-citationJusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romić, I., Wang, Z., Geček, S., Lipić, T., Podobnik, B., Wang, L., et al. (2022) · Physics Reports · doi:10.1016/j.physrep.2021.10.005 · Cited by 643
The Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
“Several options have been proposed on how journals can respond to enhance reproducible research (Nosek et al., 2015).”
BackgroundOpen science challenges, benefits and tips in early career and beyond
Allen, C., & Mehler, D. M. A. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000246 · Cited by 608
“Some journals are taking a lead in combatting questionable research practices and have signed guidelines promoting open methods [45,46].”
BackgroundNoble, S., Scheinost, D., & Constable, R. T. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116157 · Cited by 580
Estimating Treatment Effects with Causal Forests: An Application
Athey, S., & Wager, S. (2019) · Observational Studies · doi:10.1353/obs.2019.0001 · Cited by 563
Academic engagement: A review of the literature 2011-2019
Perkmann, M., Salandra, R., Tartari, V., McKelvey, M., & Hughes, A. (2021) · Research Policy · doi:10.1016/j.respol.2020.104114 · Cited by 556
Artificial intelligence and big data in entrepreneurship: a new era has begun
Obschonka, M., & Audretsch, D. B. (2020) · Small Business Economics · doi:10.1007/s11187-019-00202-4 · Cited by 527
Registration of systematic reviews in PROSPERO: 30,000 records and counting
Page, M. J., Shamseer, L., & Tricco, A. C. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0699-4 · Cited by 509
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499
Self-citationOn doing better science: From thrill of discovery to policy implications
Antonakis, J. (2017) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.01.006 · Cited by 475
“The study should fully describe its methods and procedures so that other researchers are able to faithfully reproduce it (see Atwater, et al., 2014; Cortina, et al., 2016; Goodman, Fanelli, & Ioannidis, 2016; Nosek et al., 2015); computational steps taken to arrive at analyzed datasets”
BackgroundIncreasing value and reducing waste in biomedical research: who's listening?
Moher, D., Glasziou, P., Chalmers, I., Nasser, M., Bossuyt, P. M. M., Korevaar, D. A., Graham, I. D., Ravaud, P., & Boutron, I. (2016) · The Lancet · doi:10.1016/s0140-6736(15)00307-4 · Cited by 462
Assessing scientists for hiring, promotion, and tenure
Moher, D., Naudet, F., Cristea, I. A., Miedema, F., Ioannidis, J. P. A., & Goodman, S. N. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004089 · Cited by 462
“Some journals are supporting this process by endorsing and implementing the transparency and openness promotion (TOP) guidelines [59].”
How Computational Modeling Can Force Theory Building in Psychological Science
Guest, O., & Martin, A. E. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620970585 · Cited by 460
Transparency, Reproducibility, and the Credibility of Economics Research
Christensen, G., & Miguel, E. (2018) · Journal of Economic Literature · doi:10.1257/jel.20171350 · Cited by 446
“39 Recently, we, the authors of this article, were part of an interdisciplinary group of researchers that developed a detailed set of journal guidelines called the Transparency and Openness Promotion (TOP) Guidelines (Nosek et al. 2015).”
Self-citationNosek, B. A., & Errington, T. M. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000691 · Cited by 444
Self-citationEnhancing reproducibility for computational methods
Stodden, V., McNutt, M., Bailey, D. H., Deelman, E., Gil, Y., Hanson, B., Heroux, M. A., Ioannidis, J. P. A., & Taufer, M. (2016) · Science · doi:10.1126/science.aah6168 · Cited by 411
Self-citationHow Bayes factors change scientific practice
Dienes, Z. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.10.003 · Cited by 401
“…is seriously considered as part of the solution— alongwith, for example, full transparency and online availability of materials, data and analysis (Nosek et al., 2015); greater emphasis on direct replications as well as multi-experiment theory building (Asendorpf et al., 2013); and increasing…”
BackgroundSwedberg, R. (2020) · The Production of Knowledge · doi:10.1017/9781108762519.002 · Cited by 293
Investigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 462
“Finally, journals, funders, and institutions can assess the research cultures and incentives that they create and maintain and introduce new policies that are valuesaligned to promote research rigor and transparency (Hatch and Curry, 2020; Nosek et al., 2015).”
BackgroundSelf-citationBridging Bio–Nano Science and Cancer Nanomedicine
Björnmalm, M., Thurecht, K. J., Michael, M., Scott, A. M., & Caruso, F. (2017) · ACS Nano · doi:10.1021/acsnano.7b04855 · Cited by 382
Data integration enables global biodiversity synthesis
Heberling, J. M., Miller, J. T., Noesgaard, D., Weingart, S. B., & Schigel, D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2018093118 · Cited by 380
Integrating explanation and prediction in computational social science
Hofman, J. M., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., Margetts, H., Mullainathan, S., Salganik, M. J., Vazire, S., et al. (2021) · Nature · doi:10.1038/s41586-021-03659-0 · Cited by 371
Self-citationAn empirical analysis of journal policy effectiveness for computational reproducibility
Stodden, V., Seiler, J., & Ma, Z. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708290115 · Cited by 369
“This is in compliance with Transparency and Openness Promotion Guidelines at Level 2 (6) and Recommendation 6 of the Reproducibility Enhancement Principles in ref.”
MethodEffects of tDCS on motor learning and memory formation: A consensus and critical position paper
Buch, E. R., Santarnecchi, E., Antal, A., Born, J., Celnik, P. A., Classen, J., Gerloff, C., Hallett, M., Hummel, F. C., Nitsche, M. A., et al. (2017) · Clinical Neurophysiology · doi:10.1016/j.clinph.2017.01.004 · Cited by 367
Implications of the Credibility Revolution for Productivity, Creativity, and Progress
Vazire, S. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617751884 · Cited by 359
Self-citationO. B. O. T. C.-S. G., Grant, S., Mayo-Wilson, E., Montgomery, P., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2735-z · Cited by 348
“To facilitate full reproducibility, authors should report software used to run analyses and provide the exact statistical code [24].”
BackgroundMethodSelf-citationHardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Henry Tessler, M., et al. (2018) · Royal Society Open Science · doi:10.1098/rsos.180448 · Cited by 336
Self-citationThe importance of open data and software: Is energy research lagging behind?
Pfenninger, S., DeCarolis, J., Hirth, L., Quoilin, S., & Staffell, I. (2017) · Energy Policy · doi:10.1016/j.enpol.2016.11.046 · Cited by 335
“Changing these incentives will require efforts not only from researchers themselves but also from their employers, from grant agencies, and other stakeholders like publishers (Nosek et al., 2015) – not all of which may have the same interests.”
Backgroundon behalf of the CONSORT-SPI Group, Montgomery, P., Grant, S., Mayo-Wilson, E., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2733-1 · Cited by 327
“Examples of such efforts include the Template for Intervention Description and Replication (TIDieR) checklist (which will replace CONSORT 2010 Item 5) [9], the Behaviour Change Technique taxonomy [21, 44], the Berkeley Initiative for Transparency in the Social Sciences [81], the Data Access and Research Transparency S…”
BackgroundMethodSelf-citationFreese, J., & Peterson, D. (2017) · Annual Review of Sociology · doi:10.1146/annurev-soc-060116-053450 · Cited by 310
Preregistering qualitative research
L. Haven, T., & Van Grootel, D. L. (2019) · Accountability in Research · doi:10.1080/08989621.2019.1580147 · Cited by 311
“More and more journals and editors across fields encourage preregistration of quantitative studies (Nosek et al. 2015).”
BackgroundDoes Oxytocin Increase Trust in Humans? A Critical Review of Research
Nave, G., Camerer, C., & McCullough, M. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615600138 · Cited by 316
Challenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 267
Self-citationPublic Data Archiving in Ecology and Evolution: How Well Are We Doing?
Roche, D. G., Kruuk, L. E. B., Lanfear, R., & Binning, S. A. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002295 · Cited by 303
A user-friendly method to merge Scopus and Web of Science data during bibliometric analysis
Caputo, A., & Kargina, M. (2022) · Journal of Marketing Analytics · doi:10.1057/s41270-021-00142-7 · Cited by 289
Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities
Bligaard, T., Bullock, R. M., Campbell, C. T., Chen, J. G., Gates, B. C., Gorte, R. J., Jones, C. W., Jones, W. D., Kitchin, J. R., & Scott, S. L. (2016) · ACS Catalysis · doi:10.1021/acscatal.6b00183 · Cited by 285
Our path to better science in less time using open data science tools
Lowndes, J. S. S., Best, B. D., Scarborough, C., Afflerbach, J. C., Frazier, M. R., O’Hara, C. C., Jiang, N., & Halpern, B. S. (2017) · Nature Ecology & Evolution · doi:10.1038/s41559-017-0160 · Cited by 286
“Many researchers do not attempt to reproduce their own work 17,72 —ourselves included before 2013—and thus may not realize that there could be reproduc ibility issues in their own approaches.”
BackgroundData Sharing in Psychology: A Survey on Barriers and Preconditions
Houtkoop, B. L., Chambers, C., Macleod, M., Bishop, D. V. M., Nichols, T. E., & Wagenmakers, E.-J. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917751886 · Cited by 283
“COS also developed another initiative to promote an open research culture: the Transparency and Openness Promotion guidelines (TOP; Nosek et al., 2015).”
BackgroundSelf-citationThe Peer Reviewers' Openness Initiative: incentivizing open research practices through peer review
Morey, R. D., Chambers, C. D., Etchells, P. J., Harris, C. R., Hoekstra, R., Lakens, D., Lewandowsky, S., Morey, C. C., Newman, D. P., Schönbrodt, F. D., et al. (2016) · Royal Society Open Science · doi:10.1098/rsos.150547 · Cited by 284
Self-citationWhen Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
“Incentives such as a badge system may help promote availability of more raw data (Nosek et al., 2015).”
BackgroundInfluentialReplication in Psychological Science
Lindsay, D. S. (2015) · Psychological Science · doi:10.1177/0956797615616374 · Cited by 280
Tackett, J. L., Lilienfeld, S. O., Patrick, C. J., Johnson, S. L., Krueger, R. F., Miller, J. D., Oltmanns, T. F., & Shrout, P. E. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617690042 · Cited by 251
“Goal 3: Independent replication As noted earlier, a significant contributor to the replicability crisis has been a string of widely publicized failures to replicate findings from prominent experimental psychology studies (e.g., Donnellan et al., 2015; Klein et al., 2014; Open Science Collaboration, 2015).”
BackgroundInfluentialEntrepreneurial Passion: A Review, Synthesis, and Agenda for Future Research
Newman, A., Obschonka, M., Moeller, J., & Chandan, G. G. (2021) · Applied Psychology · doi:10.1111/apps.12236 · Cited by 276
“This would also follow an open and transparent research culture as it becomes more mainstream in the social science such as psychology (Nosek et al., 2015).”
BackgroundImplementation errors in the GingerALE Software: Description and recommendations
Eickhoff, S. B., Laird, A. R., Fox, P. M., Lancaster, J. L., & Fox, P. T. (2017) · Human Brain Mapping · doi:10.1002/hbm.23342 · Cited by 269
Wallach, J. D., Boyack, K. W., & Ioannidis, J. P. A. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2006930 · Cited by 269
Marsden, E., Morgan‐Short, K., Thompson, S., & Abugaber, D. (2018) · Language Learning · doi:10.1111/lang.12286 · Cited by 268
“Indeed, journals that simply state they publish replications reach only Level 1 of the Transparency and Openness Promotion Guidelines (TOP) about replication (Nosek et al., 2015).”
BackgroundInfluentialResearching Mental Health Disorders in the Era of Social Media: Systematic Review
Wongkoblap, A., Vadillo, M. A., & Curcin, V. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7215 · Cited by 268
“Therefore, future studies should comply with the Transparency and Openness Promotion guidelines [91] at level 2 (which requires authors to deposit data and code in trusted repositories) or 3 (which also requires the reported analyses to be reproduced independently before publication).”
BackgroundResearch Weaving: Visualizing the Future of Research Synthesis
Nakagawa, S., Samarasinghe, G., Haddaway, N. R., Westgate, M. J., O’Dea, R. E., Noble, D. W. A., & Lagisz, M. (2019) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2018.11.007 · Cited by 261
The readability of scientific texts is decreasing over time
Plavén-Sigray, P., Matheson, G. J., Schiffler, B. C., & Thompson, W. H. (2017) · eLife · doi:10.7554/elife.27725 · Cited by 257
Considering aspects of the 3Rs principles within experimental animal biology
Sneddon, L. U., Halsey, L. G., & Bury, N. R. (2017) · Journal of Experimental Biology · doi:10.1242/jeb.147058 · Cited by 254
“Many authors have discussed how to combat irreproducibility (Freedman et al., 2015; Ioannidis et al., 2015; McNutt, 2014; Nosek et al., 2015; Woolston, 2014; http://validation.scienceexchange.com/ #/reproducibility-initiative).”
BackgroundRothman, J. S., & Silver, R. A. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2018.00014 · Cited by 256
“The adoption of more standardized formats for data and metadata will allow integration with data sharing projects and promote transparent, open and reproducible neuroscience research (Nosek et al., 2015; Ascoli et al., 2017; Eglen et al., 2017; Gleeson et al., 2017; Munafò et al., 2017).”
BackgroundBeillouin, D., Cardinael, R., Berre, D., Boyer, A., Corbeels, M., Fallot, A., Feder, F., & Demenois, J. (2022) · Global Change Biology · doi:10.1111/gcb.15998 · Cited by 255
“Therefore, advances must be made so to give full and unbiased access to scientific data in line with open science practices (Nosek et al., 2015).”
BackgroundIyadurai, L., Visser, R. M., Lau-Zhu, A., Porcheret, K., Horsch, A., Holmes, E. A., & James, E. L. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2018.08.005 · Cited by 256
“Open science will clearly help in this regard (Nosek et al., 2015).”
BackgroundThe Open Innovation in Science research field: a collaborative conceptualisation approach
Beck, S., Bergenholtz, C., Bogers, M., Brasseur, T.-M., Conradsen, M. L., Di Marco, D., Distel, A. P., Dobusch, L., Dörler, D., Effert, A., et al. (2022) · Industry and Innovation · doi:10.1080/13662716.2020.1792274 · Cited by 254
“These include the design of specific incentives for ensuring the reproducibility of results, which could benefit faster dissemination and iteration of scientific findings (Nielsen 2011; Nosek et al. 2015).”
BackgroundThe use of crowdsourcing in addiction science research: Amazon Mechanical Turk.
Strickland, J. C., & Stoops, W. W. (2019) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000235 · Cited by 250
A Short (Personal) Future History of Revolution 2.0
Spellman, B. A. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615609918 · Cited by 213
“…dozens of journals in psychology and other sciences (including Science ) have signed onto the Transparency and Openness guidelines, a framework for developing publication practices that would incrementally increase openness and (hopefully) consequently replicability in science (Nosek et al., 2015).”
Self-citationMarwick, B. (2017) · Journal of Archaeological Method and Theory · doi:10.1007/s10816-015-9272-9 · Cited by 179
Nosek, B. A., & Errington, T. M. (2017) · eLife · doi:10.7554/elife.23383 · Cited by 242
“Finally, all methods, proposed analyses and data are made publicly accessible via the Open Science Framework to maximize transparency and accountability (Nosek et al., 2015).”
MethodSelf-citationTen common statistical mistakes to watch out for when writing or reviewing a manuscript
Makin, T. R., & Orban de Xivry, J.-J. (2019) · eLife · doi:10.7554/elife.48175 · Cited by 238
A Practical Guide for Transparency in Psychological Science
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018) · Collabra: Psychology · doi:10.1525/collabra.158 · Cited by 233
“Numerous research funders, institutions, and scientific journals have endorsed transparent and reproducible research practices through the TOP guidelines (Nosek et al., 2015)5 and major psychology journals have begun implementing policy changes that encourage or mandate sharing (see e.”
BackgroundInfluentialHutton, C., Wagener, T., Freer, J., Han, D., Duffy, C., & Arheimer, B. (2016) · Water Resources Research · doi:10.1002/2016wr019285 · Cited by 233
“Nosek et al [2015] [ Nerc.ac.uk , 2016; NSF , 2016].”
BackgroundPineau, J., Vincent-Lamarre, P., Sinha, K., Larivière, V., Beygelzimer, A., d'Alché-Buc, F., Fox, E., & Larochelle, H. (2020) · arXiv · doi:10.48550/arxiv.2003.12206 · Cited by 232
“Reporting guidelines in the form of checklists have been introduced for a wide range of study design in health research (The EQUATOR Network, 2021), and the Transparency and Openness Promotion (TOP) guidelines have been adopted by multiple journals across disciplines (Nosek et al., 2015).”
BackgroundData sharing: An open mind on open data
Gewin, V. (2016) · Nature · doi:10.1038/nj7584-117a · Cited by 231
An updated overview on metal nanoparticles toxicity
Medici, S., Peana, M., Pelucelli, A., & Zoroddu, M. A. (2021) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2021.06.020 · Cited by 218
Predictive analytics in health care: how can we know it works?
Van Calster, B., Wynants, L., Timmerman, D., Steyerberg, E. W., & Collins, G. S. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz130 · Cited by 223
“The scientific community is making efforts to improve data sharing, increase study registration beyond clinical trials, and make reporting transparent and comprehensive with full disclosure of study results.(5,6) We discuss the importance of transparency in the context of medical predictive analytics.”
BackgroundFisher, E., Moore, R. A., Fogarty, A. E., Finn, D. P., Finnerup, N. B., Gilron, I., Haroutounian, S., Krane, E., Rice, A. S. C., Rowbotham, M., et al. (2021) · Pain · doi:10.1097/j.pain.0000000000001929 · Cited by 215
The critiques and criticisms of positive psychology: a systematic review
van Zyl, L. E., Gaffaney, J., van der Vaart, L., Dik, B. J., & Donaldson, S. I. (2024) · The Journal of Positive Psychology · doi:10.1080/17439760.2023.2178956 · Cited by 217
“Given the ‘over reliance on empiricism’ noted by the critics, mentioned in the previous like (Nosek et al., 2015; Van Zyl, 2019; Van Zyl & Junker, 2019).”
BackgroundOpen Science principles for accelerating trait-based science across the Tree of Life
Gallagher, R. V., Falster, D. S., Maitner, B. S., Salguero-Gómez, R., Vandvik, V., Pearse, W. D., Schneider, F. D., Kattge, J., Poelen, J. H., Madin, J. S., et al. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-1109-6 · Cited by 217
An Agenda for Open Science in Communication
Dienlin, T., Johannes, N., Bowman, N. D., Masur, P. K., Engesser, S., Kümpel, A. S., Lukito, J., Bier, L. M., Zhang, R., Johnson, B. K., et al. (2021) · Journal of Communication · doi:10.1093/joc/jqz052 · Cited by 209
“crisis, a growing number of scholars have called for more transparent and open science practices (Nosek et al., 2015).”
BackgroundReproducibility and replicability of rodent phenotyping in preclinical studies
Kafkafi, N., Agassi, J., Chesler, E. J., Crabbe, J. C., Crusio, W. E., Eilam, D., Gerlai, R., Golani, I., Gomez-Marin, A., Heller, R., et al. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.01.003 · Cited by 215
Best practices for addressing missing data through multiple imputation
Woods, A. D., Gerasimova, D., Van Dusen, B., Nissen, J., Bainter, S., Uzdavines, A., Davis‐Kean, P. E., Halvorson, M., King, K. M., Logan, J. A. R., et al. (2024) · Infant and Child Development · doi:10.1002/icd.2407 · Cited by 208
Preregistering Qualitative Research: A Delphi Study
Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., Piñeiro, R., Rosenblatt, F., & Mokkink, L. B. (2020) · International Journal of Qualitative Methods · doi:10.1177/1609406920976417 · Cited by 211
“This is reflected in increasing journals and funders attention toward the topic (Arnold Foundation, 2020; Nosek et al., 2015; Open Science Framework, 2020).”
BackgroundPage, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T., Mulrow, C. D., Shamseer, L., & Moher, D. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.11.010 · Cited by 211
“meta-analyses (1, 2, 14, 15, 17-19, 21, 23, 24, 26-38, 40-72) (included reports are listed in Table S1 in the Appendix, also available at https://osf.”
MethodGlobal ecological impacts of marine exotic species
Anton, A., Geraldi, N. R., Lovelock, C. E., Apostolaki, E. T., Bennett, S., Cebrian, J., Krause-Jensen, D., Marbà, N., Martinetto, P., Pandolfi, J. M., et al. (2019) · Nature Ecology & Evolution · doi:10.1038/s41559-019-0851-0 · Cited by 207
Metadata
- Authors
- Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E. L., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T.
- Affiliations
- Affiliations for the authors, all of whom are members of the TOP Guidelines Committee, are given in the supplementary materials.
- Corresponding author
- Nosek, B. A.
- ORCID
- Nosek, B. A.
- Alter, G.
- Banks, G. C.
- Borsboom, D.
- Bowman, S. D.
- Breckler, S. J.
- Buck, S.
- Chambers, C. D.
- Christensen, G.
- Dafoe, A.
- Eich, E.
- Glennerster, R.
- Goroff, D.
- Hesse, B.
- Humphreys, M.
- Ishiyama, J.
- Karlan, D.
- Lupia, A.
- Mabry, P.
- Madon, T.
- Malhotra, N.
- McNutt, M.
- Miguel, E.
- Paluck, E. Levy
- Simonsohn, U.
- Soderberg, C.
- Spellman, B. A.
- VandenBos, G.
- Vazire, S.
- Wagenmakers, E. J.
- Wilson, R.
- Yarkoni, T.
- Venue
- Science
- Published
- 2015-06-26
- Volume
- 348
- Issue
- 6242
- Pages
- 1422-1425
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC4550299
- OpenAlex
- W1767470961
- Cited by
- 2,708
- References
- 15
- Citations by year
- 2026: 214 · 2025: 203 · 2024: 200 · 2023: 232 · 2022: 278 · 2021: 306 · 2020: 360 · 2019: 276 · 2018: 256 · 2017: 211 · 2016: 134 · 2015: 38
- FWCI
- 418.98
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- PubMed Central (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open research culture, author guidelines, journal guidelines, transparency, research transparency, reproducibility
- MeSH terms
- Periodicals as Topic, Research, Reproducibility of Results, Guidelines as Topic, Peer Review, Research, Access to Information, Disclosure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.