Promoting an open research culture
Open accessPublished version of a preprint
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Author guidelines for journals could help to promote transparency, openness, and reproducibility
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, many are closed source applications with black box algorithms, which is problematic as transparency and data ownership are essential in the era of open science 22 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“Open science will clearly help in this regard (Nosek et al., 2015).”
From Intrusive memories of trauma: A target for research bridging cognitive science and its clinical application (2019) · cites this paper as Nosek et al., 2015
In the citing paper
“These issues may be strongly suspected and their magnitude can sometimes be estimated 63 , but cannot truly be corrected 64,65 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Self-citationRecommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationRegistered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Self-citationPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citationPublication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Self-citation
Cited by
Items 301–400 of 1,016
Konkol, M., Kray, C., & Pfeiffer, M. (2018) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2018.1508687 · Cited by 91
“Furthermore, authors are frequently not aware of the incentives that might be worth the extra e ff ort (Nosek et al .”
BackgroundStudy preregistration: An evaluation of a method for transparent reporting
Toth, A. A., Banks, G. C., Mellor, D., O’Boyle, E. H., Dickson, A., Davis, D. J., DeHaven, A., Bochantin, J., & Borns, J. (2021) · Journal of Business and Psychology · doi:10.1007/s10869-020-09695-3 · Cited by 73
Self-citationSewasew, D., & Schroeders, U. (2019) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2019.03.009 · Cited by 75
Authorization of Animal Experiments Is Based on Confidence Rather than Evidence of Scientific Rigor
Vogt, L., Reichlin, T. S., Nathues, C., & Würbel, H. (2016) · PLOS Biology · doi:10.1371/journal.pbio.2000598 · Cited by 73
“This “trust me model” of science has been criticized before [1,35,36].”
BackgroundA longitudinal assessment of the persistence of twitter datasets
Zubiaga, A. (2018) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24026 · Cited by 94
Defining Creativity: How Far Have We Come Since Plucker, Beghetto, and Dow?
Puryear, J. S., & Lamb, K. N. (2020) · Creativity Research Journal · doi:10.1080/10400419.2020.1821552 · Cited by 106
Practical Reproducibility in Geography and Geosciences
Nüst, D., & Pebesma, E. J. (2020) · Annals of the American Association of Geographers · doi:10.1080/24694452.2020.1806028 · Cited by 85
“The pace of digitization and the trend toward openness (cf. Nosek et al. 2015) put pressure on scientists at all career stages to switch to open tools 14 and require future geo-scientists and geographers to be trained as “ Pi-shaped researchers ” with a deep knowledge both in their domain as well…”
BackgroundJournal policies and editors’ opinions on peer review
Hamilton, D. G., Fraser, H. S., Hoekstra, R., & Fidler, F. M. (2020) · eLife · doi:10.7554/elife.62529 · Cited by 81
Janiszewski, C. A., & van Osselaer, S. M. J. (2021) · Journal of Consumer Psychology · doi:10.1002/jcpy.1280 · Cited by 133
Home Style Opinion: How Local Newspapers Can Slow Polarization
Darr, J., Hitt, M. P., & Dunaway, J. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108950930 · Cited by 92
Striving for transparent and credible research: practical guidelines for behavioral ecologists
Ihle, M., Winney, I. S., Krystalli, A., & Croucher, M. (2017) · Behavioral Ecology · doi:10.1093/beheco/arx003 · Cited by 75
“Journal editors from all fields, including our own, are being encouraged by the COS to award Badges to Acknowledge Open Practices (Table 1) (Nosek et al. 2015).”
BackgroundFuture Directions in the Study and Treatment of Parent–Child Separation
Humphreys, K. L. (2018) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2018.1534209 · Cited by 86
“Preregistration of analyses, part of the open science movement (Nosek et al., 2015), has the advantage of both avoiding “ HARKing ” (hypothesizing after the results are known; Kerr, 1998) and establishing a clear analytical plan from the start, which would ideally capture all relevant variables.”
BackgroundStandard Operating Procedures: A Safety Net for Pre-Analysis Plans
Lin, W., & Green, D. P. (2016) · Political Science and Politics · doi:10.1017/s1049096516000810 · Cited by 71
Self-citationImaging methods are vastly underreported in biomedical research
Marqués, G., Pengo, T., & Sanders, M. A. (2020) · eLife · doi:10.7554/elife.55133 · Cited by 77
Trambaiolli, L. R., Kohl, S. H., Linden, D. E. J., & Mehler, D. M. A. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.02.015 · Cited by 87
Björndahl, L., Barratt, C. L. R., Mortimer, D., Agarwal, A., Aitken, R. J., Alvarez, J. G., Aneck-Hahn, N., Arver, S., Baldi, E., Bassas, L., et al. (2022) · Human Reproduction · doi:10.1093/humrep/deac189 · Cited by 71
“This approach is consistent with the TOP Guidelines (Transparency and Openness Promotion; Centre for Open Science, https://www.cos.io/initiatives/top-guidelines; Nosek et al., 2015).”
ResultOpen Science Is Liberating and Can Foster Creativity
Frankenhuis, W. E., & Nettle, D. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691618767878 · Cited by 81
Developmental temperature affects phenotypic means and variability: A meta‐analysis of fish data
O’Dea, R. E., Lagisz, M., Hendry, A. P., & Nakagawa, S. (2019) · Fish and Fisheries · doi:10.1111/faf.12394 · Cited by 79
Pollack, J. M., MAULA, M. V. J., Allison, T. H., Renko, M., & Günther, C. C. (2019) · Entrepreneurship Theory and Practice · doi:10.1177/1042258719888640 · Cited by 71
“More specifically, we encourage researchers in both quantitative and qualitative endeavors, to engage in transparent and open science principles to help improve the strength and reproducibility of research in the domain of crowd- funded entrepreneurial opportunities (e.g., Aguinis, Hill, & Bailey, 2019; Aguinis, Raman…”
BackgroundOstarek, M., & Bottini, R. (2021) · Journal of Cognition · doi:10.5334/joc.139 · Cited by 78
Need satisfaction in intergroup contact: A multinational study of pathways toward social change.
Häßler, T., Ullrich, J., Sebben, S., Shnabel, N., Bernardino, M., Valdenegro, D., van Laar, C., González, R., Visintin, E. P., Tropp, L. R., et al. (2021) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000365 · Cited by 88
“In addition to using a large and heterogeneous data set, we also followed guidelines for best practices for open research to increase the credibility and transparency of our results (Nosek et al., 2015).”
MethodCurley, C., Krause, R. M., Feiock, R. C., & Hawkins, C. V. (2017) · Urban Affairs Review · doi:10.1177/1078087417726394 · Cited by 87
“This is manifested in the Transparency and Openness Promotion (TOP) guidelines that are being adopted by many journals (Nosek et al. 2015).”
BackgroundDataset decay and the problem of sequential analyses on open datasets
Thompson, W. H., Wright, J., Bissett, P. G., & Poldrack, R. A. (2020) · eLife · doi:10.7554/elife.53498 · Cited by 88
Kaye, J. T., Bradford, D. E., & Curtin, J. J. (2016) · Psychophysiology · doi:10.1111/psyp.12663 · Cited by 72
“Following Open Science recommendations (Nosek et al., 2015), we provide unrestricted access to all raw data and analysis code from this study to the research community via Open Science Framework (https://osf.io/fdjg9/).”
MethodDo machine learning platforms provide out-of-the-box reproducibility?
Gundersen, O. E., Shamsaliei, S., & Isdahl, R. J. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.06.014 · Cited by 84
A critical moment in machine learning in medicine: on reproducible and interpretable learning
Ciobanu-Caraus, O., Aicher, A., Kernbach, J. M., Regli, L., Serra, C., & Staartjes, V. E. (2024) · Acta Neurochirurgica · doi:10.1007/s00701-024-05892-8 · Cited by 97
Statistical analyses for studying replication: Meta-analytic perspectives.
Hedges, L. V., & Schauer, J. M. (2018) · Psychological Methods · doi:10.1037/met0000189 · Cited by 78
Empirical evidence of widespread exaggeration bias and selective reporting in ecology
Kimmel, K. A., Avolio, M. L., & Ferraro, P. J. (2023) · Nature Ecology & Evolution · doi:10.1038/s41559-023-02144-3 · Cited by 93
Kaplan, G., Mitman, K., & Violante, G. L. (2020) · Journal of Public Economics · doi:10.1016/j.jpubeco.2020.104176 · Cited by 90
Refining animal research: The Animal Study Registry
Bert, B., Heinl, C., Chmielewska, J., Schwarz, F., Grune, B., Hensel, A., Greiner, M., & Schönfelder, G. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000463 · Cited by 67
“Transparent documentation and data sharing can help retrace study results and give other researchers the possibility to reproduce experimental outcomes and to build new research questions upon them [15].”
BackgroundSutch, R. C. (2017) · Social Science History · doi:10.1017/ssh.2017.27 · Cited by 85
Data quality over data quantity in computational cognitive neuroscience
Kolossa, A., & Kopp, B. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.01.005 · Cited by 68
“io (Nosek et al., 2015)), and it led to several multi-lab collaboration initiatives (e.”
Schoemann, M., O’Hora, D. P., Dale, R. A. C., & Scherbaum, S. (2020) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01851-3 · Cited by 76
“…design issues suggests that codifying new standards for mouse-tracking research will facilitate strong reproducible findings that can more easily be combined across studies while creating a foundation for new methodological ingenuity (Morey et al., 2016; Munafò et al., 2017; Nosek et al., 2015).”
BackgroundInfluentialHomogeneity of personal values and personality traits in Facebook social networks
Lönnqvist, J., & Itkonen, J. V. A. (2015) · Journal of Research in Personality · doi:10.1016/j.jrp.2015.11.001 · Cited by 83
Scientists’ Reputations Are Based on Getting It Right, Not Being Right
Ebersole, C. R., Axt, J. R., & Nosek, B. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002460 · Cited by 67
“With increasing attention paid to reproducibility in science [1,2], a natural worry for researchers is, “What happens if my finding does not replicate?”Wemight ask ourselves that question out of intellectual curiosity but also out of concern for our reputations.”
BackgroundSelf-citationNeuroadaptive Bayesian Optimization and Hypothesis Testing
Lorenz, R., Hampshire, A., & Leech, R. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.01.006 · Cited by 71
Andrade, J. M., Ferreiro, B., López‐Mahía, P., & Muniategui‐Lorenzo, S. (2020) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2020.111035 · Cited by 73
Elliott, M. L., Knodt, A. R., Ireland, D. R., Morris, M. L., Poulton, R., Ramrakha, S., Sison, M. L., Moffitt, T. E., Caspi, A., & Hariri, A. R. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/681700 · Cited by 72
Kidwell, M. C., Lazarević, L. B., Baranski, E. N., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Słowik, A., Sonnleitner, C., Holden, C. L., et al. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/khbvy · Cited by 88
Self-citationEnacting open science by D4Science
Assante, M., Candela, L., Castelli, D., Cirillo, R., Coro, G., Frosini, L., Lelii, L., Mangiacrapa, F., Pagano, P., Panichi, G., et al. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.05.063 · Cited by 66
“Open science is promising to revolutionise the entire science enterprise by envisioning and proposing new practices aiming at “improving” science [1, 2, 3].”
BackgroundMonitoring worker fatigue using wearable devices: A case study to detect changes in gait parameters
Baghdadi, A., Cavuoto, L., Jones-Farmer, A., Rigdon, S., Esfahani, E., & Megahed, F. (2019) · Journal of QualityTechnology · doi:10.1080/00224065.2019.1640097 · Cited by 57
Wijsen, L. D., Borsboom, D., & Alexandrova, A. (2021) · Perspectives on Psychological Science · doi:10.1177/17456916211014183 · Cited by 95
“Objectivity can mean being true to nature from God’s eye point of view, or it can mean restricting personal preferences or desires of the inquirer, or may amount to just following transparent and mechanical procedures (e.g., the open science movement; Nosek et al., 2015) .”
BackgroundSelf-citationRenton, A., Dao, T. T., Johnstone, T., Civier, O., Sullivan, R. P., White, D., Lyons, P., Slade, B., Abbott, D. F., Amos, T. J., et al. (2024) · Nature Methods · doi:10.1038/s41592-023-02145-x · Cited by 83
“The transparency and openness promotion guidelines, which have over 5,000 journals, publishers and other related organizations as signatories, state that all reported results should be independently reproduced before publication 15 .”
Nullius in Verba1: Advancing Data Transparency in Industrial Ecology
Hertwich, E. G., Heeren, N., Kuczenski, B., Majeau‐Bettez, G., Myers, R. J., Pauliuk, S., Stadler, K., & Lifset, R. (2018) · Journal of Industrial Ecology · doi:10.1111/jiec.12738 · Cited by 69
“TOP: Transparency and Openness Promotion The COS recently published a set of TOP guidelines to promote open science practices (Nosek et al. 2015).”
InfluentialA Standard for the Scholarly Citation of Archaeological Data as an Incentive to Data Sharing
Marwick, B., & Birch, S. E. P. (2018) · Advances in Archaeological Practice · doi:10.1017/aap.2018.3 · Cited by 71
A framework for learning from null results
Jacob, R. T., Doolittle, F., Kemple, J., & Somers, M.-A. (2019) · Educational Researcher · doi:10.3102/0013189x19891955 · Cited by 65
“A reading intervention that has positive impacts on decoding skills but not reading comprehension skills is effective at improving decoding skills but not an effective reading intervention overall; an intervention that has no main impacts but does have impacts for the boys in the sample is an effective intervention fo…”
BackgroundBronkhorst, B., Tummers, L. G., & Steijn, B. (2017) · Safety Science · doi:10.1016/j.ssci.2017.12.009 · Cited by 79
“In line with good practices for pre-registration and promoting an open research culture (Nosek et al., 2015), we indicate which reported outcomes were in line with the pre-registration and which were not.”
ResultRinse and Repeat: Understanding the Value of Replication across Different Ways of Knowing
Penders, B., Holbrook, J. B., & de Rijcke, S. (2019) · Publications · doi:10.3390/publications7030052 · Cited by 80
“Nosek et al. [42] do not explicitly seek to impose a replication drive on the humanities; however, they lament the fact that there is no central authority that can impose a replication mandate on science: “Unfortunately, there is no centralized means of aligning individual and communal incentives…”
BackgroundInfluentialReward-driven distraction: A meta-analysis.
Rusz, D., Pelley, M. E. L., Kompier, M. A. J., Mait, L., & Bijleveld, E. (2020) · Psychological Bulletin · doi:10.1037/bul0000296 · Cited by 69
“…the hypotheses and the specific analysis plan for a study would help cleanly distinguish between confirmatory and exploratory analyses, which is an important aspect of open science (Munafò et al., 2017; Nosek et al., 2015; Nosek, Ebersole, DeHaven, & Mellor, 2018; Wagenmakers & Dutilh, 2016).”
BackgroundWalkey, A. J., Sheldrick, R. C., Kashyap, R., Kumar, V. K., Boman, K., Bolesta, S., Zampieri, F. G., Bansal, V., Harhay, M. O., & Gajic, O. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004572 · Cited by 65
“They can also go further by embracing standards that reflect accountability and reason, such as full public transparency (12)—in this case concerning the rationale, rules, and results of the prioritization process—and also by offering opportunity for challenge and dissent (13).”
BackgroundMeta-analysis in vocational behavior: A systematic review and recommendations for best practices
Rudolph, C. W., Chang, C. K., Rauvola, R. S., & Zacher, H. (2020) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2020.103397 · Cited by 69
Rosenfeld, D. L., Balcetis, E., Bastian, B. B., Berkman, E. T., Bosson, J. K., Brannon, T. N., Burrow, A. L., Cameron, C. D., Chen, S., Cook, J., et al. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691621999374 · Cited by 72
“…for hard-to-reach populations (e.g., Klein et al., 2014, 2019; the ManyBabies Consortium; Psychological Science Accelerator)—will not only facilitate reproducible psychological science but also help build the community of science we desperately need (Nosek et al., 2015; Stevens et al., 2018).”
Self-citationA meta-analysis of the survival-processing advantage in memory
Scofield, J. E., Buchanan, E. M., & Kostic, B. N. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1346-0 · Cited by 70
“(2008) as a part of the Reproducibility Project (Nosek et al., 2015).”
BackgroundMethodTransparent Social Inquiry: Implications for Political Science
Elman, C., Kapiszewski, D., & Lupia, A. (2017) · Annual Review of Political Science · doi:10.1146/annurev-polisci-091515-025429 · Cited by 83
Self-citationCrisis or self-correction: Rethinking media narratives about the well-being of science
Jamieson, K. H. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708276114 · Cited by 78
Maggio, L. A., Dong, T., Driessen, E. W., & Artino, A. R. (2019) · Perspectives on Medical Education · doi:10.1007/s40037-019-0501-x · Cited by 71
Improving Transparency in Observational Social Science Research: A Pre-Analysis Plan Approach
Burlig, F. (2016) · Economics Letters · doi:10.1016/j.econlet.2018.03.036 · Cited by 79
López‐Nicolás, R., López-López, J. A., Rubio‐Aparicio, M., & Sánchez‐Meca, J. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01644-z · Cited by 80
“Open science initiatives are a major point here: preregistration, sharing open material and data, and sharing open analysis scripts offer several benefits (Federer et al., 2018; Hardwicke & Ioannidis, 2018b; Nelson et al., 2018; Nosek et al., 2015; Nosek et al., 2019; Nosek & Lindsay, 2018; Popkin, 2019).”
BackgroundElliot, A. J. (2018) · Review of General Psychology · doi:10.1037/gpr0000170 · Cited by 79
Smith, D. V., Gseir, M., Speer, M. E., & Delgado, M. R. (2016) · Human Brain Mapping · doi:10.1002/hbm.23216 · Cited by 61
Passing the Data Baton : A Retrospective Analysis on Data Science Work and Workers
Crisan, A., Fiore-Gartland, B., & Tory, M. K. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3030340 · Cited by 90
“We note that these challenges are also mirrored in the growing open science movement more generally [29, 62].”
BackgroundQuantitative Data From Rating Scales: An Epistemological and Methodological Enquiry
Uher, J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02599 · Cited by 85
“…which they aim to solve by scrutinizing the epistemological foundations of significance testing, confidence interval estimations and Bayesian approaches (Nosek et al., 2015; Open Science Collaboration, 2015; Wagenmakers et al., 2016; Zwaan et al., 2017)—thus, by improving issues of data analysis.”
BackgroundDuBois, J. M., & Antes, A. L. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001966 · Cited by 77
From FAIR research data toward FAIR and open research software
Hasselbring, W., Carr, L. A., Hettrick, S., Packer, H. S., & Tiropanis, T. (2020) · it - Information Technology · doi:10.1515/itit-2019-0040 · Cited by 61
Bošnjak, M., Fiebach, C. J., Mellor, D. T., Mueller, S., O'Connor, D. B., Oswald, F. L., & Sokol, R. I. (2021) · American Psychologist · doi:10.1037/amp0000879 · Cited by 77
Gupta, C., Chandrashekar, P. B., Jin, T., He, C., Khullar, S., Chang, Q., & Wang, D. (2022) · Journal of Neurodevelopmental Disorders · doi:10.1186/s11689-022-09438-w · Cited by 76
Gagné, N., & Franzen, L. (2023) · Swiss Psychology Open · doi:10.5334/spo.34 · Cited by 79
Behavioral Public Administration: Combining Insights from Public Administration and Psychology
Grimmelikhuijsen, S., Jilke, S., Olsen, A. L., & Tummers, L. G. (2016) · Public Administration Review · doi:10.1111/puar.12609 · Cited by 760
Biomedical text mining for research rigor and integrity: tasks, challenges, directions
Kilicoglu, H. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx057 · Cited by 61
Increasing the Impact of Materials in and beyond Bio-Nano Science
Björnmalm, M., Faria, M., & Caruso, F. (2016) · Journal of the American Chemical Society · doi:10.1021/jacs.6b08673 · Cited by 67
Fox, J., Pearce, K. E., Massanari, A. L., Riles, J. M., Szulc, Ł., Ranjit, Y. S., Trevisan, F., Soriano, C. R., Jessica Vitak, B., Arora, P., et al. (2021) · Journal of Communication · doi:10.1093/joc/jqab029 · Cited by 96
Héroux, M. E., Taylor, J. L., & Gandevia, S. C. (2015) · PLoS ONE · doi:10.1371/journal.pone.0144151 · Cited by 66
“Irreproducible research, publication bias, and questionable research practices have become increasingly worrisome to researchers, policy makers, and journal editors [21, 30, 43, 45, 46].”
BackgroundGenomic Methods Take the Plunge: Recent Advances in High-Throughput Sequencing of Marine Mammals
Cammen, K. M., Andrews, K. R., Carroll, E. L., Foote, A. D., Humble, E., Khudyakov, J. I., Louis, M., McGowen, M. R., Olsen, M. T., & Cise, A. M. V. (2016) · Journal of Heredity · doi:10.1093/jhered/esw044 · Cited by 71
WORCS: A workflow for open reproducible code in science
Lissa, C. J. V., Brandmaier, A. M., Brinkman, L., Lamprecht, A.-L., Peikert, A., Struiksma, M. E., & Vreede, B. M. I. (2021) · Data Science · doi:10.3233/ds-210031 · Cited by 70
Gehlbach, H., & Robinson, C. D. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1898961 · Cited by 67
How Powerful is the Evidence in Criminology? On Whether We Should Fear a Coming Crisis of Confidence
Barnes, J. C., TenEyck, M. F., Pratt, T. C., & Cullen, F. T. (2019) · Justice Quarterly · doi:10.1080/07418825.2018.1495252 · Cited by 69
“Thus, readers are free to reproduce the present findings or expand on the analysis as they see fit (see Nosek et al., 2015).”
BackgroundScanning the Horizon: Towards transparent and reproducible neuroimaging researchPreprint
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafo, M. R., Nichols, T. E., Poline, J., Vul, E., & Yarkoni, T. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/059188 · Cited by 74
Self-citationAll-hazards dataset mined from the US National Incident Management System 1999–2020
Denis, L. A. S., Short, K. C., McConnell, K., Cook, M., Mietkiewicz, N. P., Buckland, M., & Balch, J. K. (2023) · Scientific Data · doi:10.1038/s41597-023-01955-0 · Cited by 68
A new approach to data access and research transparency (DART)
Beugelsdijk, S., van Witteloostuijn, A. C., & Meyer, K. E. (2020) · Journal of International Business Studies · doi:10.1057/s41267-020-00323-z · Cited by 65
A Meta-Analysis of Test Scores in Proctored and Unproctored Ability Assessments
Steger, D., Schroeders, U., & Gnambs, T. (2018) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000494 · Cited by 71
Five common pitfalls in eye-tracking research
Godfroid, A., & Hui, B. (2020) · Second language Research · doi:10.1177/0267658320921218 · Cited by 69
“Scientific progress depends on both innovation and self-correction (Nosek et al., 2015).”
BackgroundFlake, J. K. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1898962 · Cited by 58
Hoffmann, S., Schönbrodt, F. D., Elsas, R., Wilson, R., Strasser, U., & Boulesteix, A. (2021) · Royal Society Open Science · doi:10.1098/rsos.201925 · Cited by 85
“Another prominent solution which has been proposed in response to the replication crisis in science is to increase transparency by promoting reproducible research practices [66,67].”
BackgroundSelf-citationNitschke, F., McKimmie, B. M., & Vanman, E. J. (2019) · Psychological Bulletin · doi:10.1037/bul0000206 · Cited by 84
Köhler, T., González‐Morales, M. G., Banks, G. C., O’Boyle, E. H., Allen, J. A., Sinha, R., Woo, S. E., & Gulick, L. M. V. (2020) · Industrial and Organizational Psychology · doi:10.1017/iop.2019.121 · Cited by 66
“TOP guidelines focus on, in part, providing suggestions for improving the peer-review process (Nosek et al., 2015).”
BackgroundSelf-citationBrown, A. W., Aslibekyan, S. W., Bier, D. M., da Silva, R. F., Hoover, A. W., Klurfeld, D. M., Loken, E., Mayo‐Wilson, E., Menachemi, N., Pavela, G., et al. (2021) · Critical Reviews in Food Science and Nutrition · doi:10.1080/10408398.2021.1985427 · Cited by 65
Self-citationTowards a Research Agenda for Promoting Responsible Research Practices
Tijdink, J. K., Horbach, S. P. J. M., Nuijten, M. B., & O’Neill, G. (2021) · Journal of Empirical Research on Human Research Ethics · doi:10.1177/15562646211018916 · Cited by 63
Status, use and impact of sharing individual participant data from clinical trials: a scoping review
Ohmann, C., Moher, D., Siebert, M., Motschall, E., & Naudet, F. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-049228 · Cited by 64
Seven steps toward more transparency in statistical practice
Wagenmakers, E., Sarafoglou, A., Aarts, S., Albers, C. J., Algermissen, J., Bahník, Š., van Dongen, N. N. N., Hoekstra, R., Moreau, D., van Ravenzwaaij, D., et al. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01211-8 · Cited by 68
Self-citationQCA in International Relations: A Review of Strengths, Pitfalls, and Empirical Applications
Ide, T., & Mello, P. A. (2022) · International Studies Review · doi:10.1093/isr/viac008 · Cited by 87
Good scientific practice in EEG and MEG research: Progress and perspectives
Niso, G., Krol, L. R., Combrisson, E., Dubarry, A., Elliott, M., Clément, F., Héjja‐Brichard, Y., Herbst, S. K., Jerbi, K., Ković, V., et al. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119056 · Cited by 75
“Larson, 2021; Piwowar & Vision, 2013), facilitates compliance with publishing guidelines (Nosek et al., 2015), increases job competitiveness (Nosek et al.”
BackgroundProsser, A. M. B., Hamshaw, R. J. T., Meyer, J., Bagnall, R., Blackwood, L. M., Huysamen, M., Jordan, A. L., Vasileiou, K., & Walter, Z. C. (2022) · British Journal of Social Psychology · doi:10.1111/bjso.12576 · Cited by 75
Scientific Testimony: Its roles in science and society
Gerken, M. (2022) · doi:10.1093/oso/9780198857273.001.0001 · Cited by 76
CivilServant: Community-Led Experiments in Platform Governance
Matias, J. N., & Mou, M. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173583 · Cited by 82
“By publishing all software and analyses, we are also able to contribute to an open research culture among other scholars who can query and replicate our findings [57].”
BackgroundMethodBeier, K., Schweda, M., & Schicktanz, S. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0799-7 · Cited by 66
Open Science Promotes Diverse, Just, and Sustainable Research and Educational Outcomes
Grahe, J. E., Cuccolo, K., Leighton, D. C., & Alvarez, L. D. C. (2019) · Psychology Learning & Teaching · doi:10.1177/1475725719869164 · Cited by 65
“io/rh7ta/, such as the Transparency and Openness Promotion guidelines (Nosek et al., 2015), Open Science Badges (Blohowiak et al.”
BackgroundMethodThe agora high-resolution galaxy simulations comparison project. Ii. Isolated disk test
Kim, J.-H., Agertz, O., Teyssier, R., al, E., & AGORA Collaboration (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-129935 · Cited by 56
Thermofeel: A python thermal comfort indices library
Brimicombe, C., Napoli, C. D., Quintino, T., Pappenberger, F., Cornforth, R., & Cloke, H. (2022) · SoftwareX · doi:10.1016/j.softx.2022.101005 · Cited by 63
Biosecurity in an age of open science
Smith, J. A., & Sandbrink, J. B. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001600 · Cited by 64
Unpacking the link between socioeconomic status and behavior problems: A second-order meta-analysis
Korous, K. M., Causadías, J. M., Bradley, R. H., & Luthar, S. S. (2018) · Development and Psychopathology · doi:10.1017/s0954579418001141 · Cited by 76
“io/2e8kc/) on March 23, 2018, before conducting analyses, to embrace best research practices (Nosek et al., 2015).”
BackgroundMethod
Metadata
- Authors
- Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E. L., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T.
- Affiliations
- Affiliations for the authors, all of whom are members of the TOP Guidelines Committee, are given in the supplementary materials.
- Corresponding author
- Nosek, B. A.
- ORCID
- Nosek, B. A.
- Alter, G.
- Banks, G. C.
- Borsboom, D.
- Bowman, S. D.
- Breckler, S. J.
- Buck, S.
- Chambers, C. D.
- Christensen, G.
- Dafoe, A.
- Eich, E.
- Glennerster, R.
- Goroff, D.
- Hesse, B.
- Humphreys, M.
- Ishiyama, J.
- Karlan, D.
- Lupia, A.
- Mabry, P.
- Madon, T.
- Malhotra, N.
- McNutt, M.
- Miguel, E.
- Paluck, E. Levy
- Simonsohn, U.
- Soderberg, C.
- Spellman, B. A.
- VandenBos, G.
- Vazire, S.
- Wagenmakers, E. J.
- Wilson, R.
- Yarkoni, T.
- Venue
- Science
- Published
- 2015-06-26
- Volume
- 348
- Issue
- 6242
- Pages
- 1422-1425
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC4550299
- OpenAlex
- W1767470961
- Cited by
- 2,708
- References
- 15
- Citations by year
- 2026: 214 · 2025: 203 · 2024: 200 · 2023: 232 · 2022: 278 · 2021: 306 · 2020: 360 · 2019: 276 · 2018: 256 · 2017: 211 · 2016: 134 · 2015: 38
- FWCI
- 418.98
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- PubMed Central (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open research culture, author guidelines, journal guidelines, transparency, research transparency, reproducibility
- MeSH terms
- Periodicals as Topic, Research, Reproducibility of Results, Guidelines as Topic, Peer Review, Research, Access to Information, Disclosure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.