Promoting an open research culture
Open accessPublished version of a preprint
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Author guidelines for journals could help to promote transparency, openness, and reproducibility
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, many are closed source applications with black box algorithms, which is problematic as transparency and data ownership are essential in the era of open science 22 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“Open science will clearly help in this regard (Nosek et al., 2015).”
From Intrusive memories of trauma: A target for research bridging cognitive science and its clinical application (2019) · cites this paper as Nosek et al., 2015
In the citing paper
“These issues may be strongly suspected and their magnitude can sometimes be estimated 63 , but cannot truly be corrected 64,65 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Self-citationNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Self-citationRecommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 823
Self-citationRegistered Reports: A new publishing initiative at Cortex
Chambers, C. D. (2013) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 570
Self-citationPromoting Transparency in Social Science Research
Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., Glennerster, R., Green, D. P., Humphreys, M., Imbens, G., et al. (2014) · Science · doi:10.1126/science.1245317 · Cited by 540
Self-citationPublication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Self-citation
Cited by
Items 401–500 of 1,016
The Future is a Moving Target: Predicting Political Instability
Bowlsby, D., Chenoweth, E., Hendrix, C. S., & Moyer, J. D. (2019) · British Journal of Political Science · doi:10.1017/s0007123418000443 · Cited by 69
Public Health and Cost Benefits of Successful Reperfusion After Thrombectomy for Stroke
Kunz, W. G., Almekhlafi, M., Menon, B. K., Saver, J. L., Hunink, M. G. M., Dippel, D. W. J., Majoie, C. B. L. M., Liebeskind, D. S., Jovin, T. G., Dávalos, A., et al. (2020) · Stroke · doi:10.1161/strokeaha.119.027874 · Cited by 57
Maximising value from a United Kingdom Biomedical Research Centre: study protocol
Greenhalgh, T., Ovseiko, P. V., Fahy, N., Shaw, S. E., Kerr, P., Rushforth, A., Channon, K. M., & Kiparoglou, V. (2017) · Health Research Policy and Systems · doi:10.1186/s12961-017-0237-1 · Cited by 61
“researchers conducting similar research [40].”
Envisioning the scientific paper of the future
Sopinka, N., Coristine, L. E., DeRosa, M. C., Rochman, C. M., Owens, B. L., & Cooke, S. J. (2020) · FACETS · doi:10.1139/facets-2019-0012 · Cited by 58
“Systematic reviews, meta-analyses, and centres for evidence synthesis can improve the potential for new research to expand upon and incorporate pre-existing scientific knowledge (Nosek et al. 2015; Gil et al. 2016; Cooke et al. 2017a; Piñero 2018).”
BackgroundPownall, M., Azevedo, F., König, L. M., Slack, H. R., Evans, T. R., Flack, Z. M., Grinschgl, S., Elsherif, M. M., Gilligan-Lee, K. A., Oliveira, C. M., et al. (2023) · Royal Society Open Science · doi:10.1098/rsos.221255 · Cited by 78
“While there is promising ongoing work in this area, including discussions surrounding the pedagogical value of open scholarship [46] and current Registered Reports that directly and empirically assess the impact of open scholarship on student attitudes [121,122], further empirical research should continue to investiga…”
MethodInfluentialRethinking Transparency and Rigor from a Qualitative Open Science Perspective
Steltenpohl, C. N., Lustick, H. A., Meyer, M. S., Lee, L. E., Stegenga, S. M., Reyes, L. S., & Renbarger, R. (2023) · Journal of Trial and Error · doi:10.36850/mr7 · Cited by 101
Hung, L., Kristiyanto, D., Lee, S. B., & Yeung, K. Y. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152686 · Cited by 61
Innovation with field experiments: Studying organizational behaviors in actual organizations
Hauser, O., Linos, E., & Rogers, T. (2017) · Research in Organizational Behavior · doi:10.1016/j.riob.2017.10.004 · Cited by 70
“As many others have argued before us (Abdul Latif Jameel Poverty Action Lab, n.d.; Munafò et al., 2017; Nosek et al., 2015), good experimentation requires careful planning, starting with building on strong theory, developing testable predictions, designing a clear test of the hypotheses,…”
BackgroundThe Weak Spots in Contemporary Science (and How to Fix Them)
Wicherts, J. M. (2017) · Animals · doi:10.3390/ani7120090 · Cited by 59
Mebane, C. A., Sumpter, J. P., Fairbrother, A., Augspurger, T. P., Canfield, T. J., Goodfellow, W. L., Guiney, P. D., LeHuray, A. P., Maltby, L., Mayfield, D. B., et al. (2019) · Integrated Environmental Assessment and Management · doi:10.1002/ieam.4119 · Cited by 62
“Science’s claim to self-correction and overall reliability is based on the ability of researchers to replicate the results of published studies (Nosek et al. 2015).”
BackgroundChalkia, A., Oudenhove, L. V., & Beckers, T. (2020) · Cortex · doi:10.1016/j.cortex.2020.03.031 · Cited by 63
“Methods, materials and data sharing, as well as pre-registration of designs, hypotheses, and statistical analysis plans are only some of the actions that have been recommended (Nosek et al., 2015; Nosek, Spies, & Motyl, 2012).”
Methodideal: an R/Bioconductor package for interactive differential expression analysis
Marini, F., Linke, J., & Binder, H. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03819-5 · Cited by 64
Spatially-defined motor deficits in people with unilateral complex regional pain syndrome
Reid, E., BRAITHWAITE, F., Wallwork, S. B., Harvie, D. S., Chalmers, J., Spence, C. J., Gallace, A., & Moseley, G. L. (2017) · Cortex · doi:10.1016/j.cortex.2017.06.024 · Cited by 55
A new structure for representing and tracking version information in a deep time knowledge graph
Ma, X., Ma, C., & Wang, C. (2020) · Computers & Geosciences · doi:10.1016/j.cageo.2020.104620 · Cited by 53
Woodland, L., Blignault, I., O’Callaghan, C., & Harris‐Roxas, B. (2021) · Health Research Policy and Systems · doi:10.1186/s12961-020-00657-y · Cited by 52
“It involves ensuring harmonized, mutual benefit between communities and researchers, investing time in respectful relationships, transparency, and processes to support this [49].”
BackgroundThe Power and the Pitfalls of Large-scale, Unreplicated Natural Experiments
Barley, S. C., & Meeuwig, J. J. (2016) · Ecosystems · doi:10.1007/s10021-016-0028-5 · Cited by 60
Accelerating Biomedical Discoveries through Rigor and Transparency
Hewitt, J., Brown, L. L., Murphy, S. J., Grieder, F. B., & Silberberg, S. D. (2017) · ILAR Journal · doi:10.1093/ilar/ilx011 · Cited by 55
Dougherty, M. R. P., Slevc, L. R., & Grand, J. A. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691618810693 · Cited by 80
Exploratory and confirmatory research in the open science era
Nilsen, E. B., Bowler, D. E., & Linnell, J. D. C. (2020) · Journal of Applied Ecology · doi:10.1111/1365-2664.13571 · Cited by 72
The Delaware Pain Database: a set of painful expressions and corresponding norming data
Mende‐Siedlecki, P., Qu-Lee, J. W., Lin, J., Drain, A., & Goharzad, A. (2020) · PAIN Reports · doi:10.1097/pr9.0000000000000853 · Cited by 66
Does Neonatal Imitation Exist? Insights From a Meta-Analysis of 336 Effect Sizes
Davis, J., Redshaw, J., Suddendorf, T., Nielsen, M., Kennedy‐Costantini, S., Oostenbroek, J., & Slaughter, V. P. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620959834 · Cited by 66
A Systems-Based Approach to Fostering Robust Science in Industrial-Organizational Psychology
Grand, J. A., Rogelberg, S. G., Allen, T. D., Landis, R. S., Reynolds, D. H., Scott, J. C., Tonidandel, S., & Truxillo, D. M. (2018) · Industrial and Organizational Psychology · doi:10.1017/iop.2017.55 · Cited by 52
“…(Grote, 2016); operationalize the meaning of robust science; and provide helpful guidelines for identifying, evaluating, and allocating resources (e.g., time, effort, money) toward any proposed changes intended to improve scientific practices (cf., Nosek&Bar-Anan, 2012; Nosek et al., 2015).”
BackgroundCashin, A. G., Bagg, M. K., Richards, G. C., Toomey, E. C., McAuley, J. H., & Lee, H. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2019-111296 · Cited by 65
Open science, the replication crisis, and environmental public health
Hicks, D. (2021) · Accountability in Research · doi:10.1080/08989621.2021.1962713 · Cited by 63
“For example, the Transparency and Openness Promotion (TOP) Guidelines (Nosek et al. 2015) have been adopted by many major academic publishers, including the American Association for the Advancement of Science CONTACT Daniel J. Hicks hicks.daniel.j@gmail.com University Of California, Merced, CA, USA…”
BackgroundMethodInfluentialSocial comparisons and social anxiety in daily life: An experience-sampling approach.
Goodman, F. R., Kelso, K. C., Wiernik, B. M., & Kashdan, T. B. (2021) · Journal of Abnormal Psychology · doi:10.1037/abn0000671 · Cited by 77
Hardwicke, T. E., Mathur, M. B., MacDonald, K. E., Nilsonne, G., Banks, G. C., Kidwell, M. C., Mohr, A. H., Clayton, E., Yoon, E. J., Tessler, M. H., et al. (2018) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/39cfb · Cited by 64
Self-citationA reproducible systematic map of research on the illusory truth effect
Henderson, E. L., Westwood, S. J., & Simons, D. J. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-021-01995-w · Cited by 69
Kepes, S., Keener, S. K., McDaniel, M. A., & Hartman, N. S. (2022) · Journal of Organizational Behavior · doi:10.1002/job.2623 · Cited by 71
“Overall, the objective of our sciences, to discover the truth about the world, is much better achieved with transparent scientific processes and an open research culture (Nosek et al., 2015).”
BackgroundData set mentions and citations: A content analysis of full‐text publications
Zhao, M., Yan, E., & Li, K. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23919 · Cited by 66
Personal Motivations and Systemic Incentives: Scientists on Questionable Research Practices
Bruton, S. V., Medlin, M. M., Brown, M., & Sacco, D. F. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00182-9 · Cited by 55
Kaplan, G., Mitman, K., & Violante, G. L. (2016) · National Bureau of Economic Research · doi:10.3386/w22232 · Cited by 52
“This view is in linewith the recent recognition of the importance of replicability in the physical sciences and other social sciences (Begley and Ioannidis, 2015; Munafò et al., 2017; Nosek et al., 2015).”
Handbook of field sampling for multi-taxon biodiversity studies in European forests
Burrascano, S., Trentanovi, G., Paillet, Y., Heilmann‐Clausen, J., Giordani, P., Bagella, S., Bravo‐Oviedo, A., Campagnaro, T., Campanaro, A., Chianucci, F., et al. (2021) · Ecological Indicators · doi:10.1016/j.ecolind.2021.108266 · Cited by 58
Logullo, P., de Beyer, J. A., Kirtley, S., Schlüssel, M. M., & Collins, P. G. S. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112126 · Cited by 64
Bradford, D. E., Starr, M. J., Shackman, A. J., & Curtin, J. J. (2015) · Psychophysiology · doi:10.1111/psyp.12545 · Cited by 51
“We support emerging open science guidelines (Nosek et al., 2015).”
The functional‐cognitive framework for psychological research: Controversies and resolutions
Hughes, S., Houwer, J. D., & Perugini, M. (2015) · International Journal of Psychology · doi:10.1002/ijop.12239 · Cited by 59
“Therefore, just as certain practices increase the likelihood of replicating psychological research (Nosek et al., 2015), the FC framework increase the likelihood of open-mindedness and cumulative growth when carrying out that research.”
BackgroundA Replication by Any Other Name: A Systematic Review of Replicative Intervention Studies
Cook, B. G., Collins, L. W., Cook, S. C., & Cook, L. (2016) · Remedial and Special Education · doi:10.1177/0741932516637198 · Cited by 56
A Bayesian bird's eye view of ‘Replications of important results in social psychology’
Marsman, M., Schönbrodt, F. D., Morey, R. D., Yao, Y., Gelman, A., & Wagenmakers, E. (2017) · Royal Society Open Science · doi:10.1098/rsos.160426 · Cited by 61
“[73,74]): a stronger focus on replication studies, more use of high-powered designs, standard adoption of preregistration and data sharing by default [75].”
BackgroundSelf-citationA brief guide to the science and art of writing manuscripts in biomedicine
Forero, D. A., López‐León, S., & Perry, G. L. (2020) · Journal of Translational Medicine · doi:10.1186/s12967-020-02596-2 · Cited by 57
Mieczkowski, H., Lee, A. Y., & Hancock, J. T. (2020) · Social Media + Society · doi:10.1177/2056305120961784 · Cited by 50
“This advice is in line with other suggestions that promote open science best practices (Nosek et al., 2015).”
ResultToribio‐Flórez, D., Anneser, L., Lopes, F. N. D. O., Pallandt, M. M. T. A., Tunn, I., Windel, H., & on behalf of Max Planck PhDnet Open Science Group (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2020.586992 · Cited by 63
“Examples for guidelines for OA publishing (Nosek et al., 2015; Schiltz, 2018), FAIR data management (Wilkinson et al., 2016), as well as OS collaborations (Gold et al., 2019) are already existing, and their use should be promoted by governments and funding agencies, as well as integrated in the…”
BackgroundTransparency, Reproducibility, and the Credibility of Economics ResearchPreprint
Christensen, G., & Miguel, E. A. (2017) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/9a3rw · Cited by 59
Self-citationWaltemath, D., & Wolkenhauer, O. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2555481 · Cited by 49
“A lack of transparency and openness led to what many consider a reproducibility crisis [1]–[3].”
BackgroundFostering responsible research practices is a shared responsibility of multiple stakeholders
Bouter, L. M. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.016 · Cited by 53
Out-of-the-Box Reproducibility: A Survey of Machine Learning Platforms
Isdahl, R., & Gundersen, O. E. (2019) · eScience · doi:10.1109/escience.2019.00017 · Cited by 80
“In psychology, the reproducibility project was only able to reproduce 36 out of 100 psychology research articles with statistically significant results [3].”
BackgroundGiner‐Sorolla, R. (2018) · European Review of Social Psychology · doi:10.1080/10463283.2018.1542902 · Cited by 56
Preregistration of Analyses of Preexisting Data
Mertens, G., & Krypotos, A.-M. (2019) · Psychologica Belgica · doi:10.5334/pb.493 · Cited by 48
Bloomfield, R. J., Rennekamp, K. M., & Steenhoven, B. A. (2018) · Journal of Accounting Research · doi:10.1111/1475-679x.12208 · Cited by 54
Recommendations for Reviewing Meta-Analyses in Organizational Research
DeSimone, J. A., Brannick, M. Τ., O’Boyle, E. H., & Ryu, J. W. (2020) · Organizational Research Methods · doi:10.1177/1094428120967089 · Cited by 54
“Our primary focus on transparency aligns with recent efforts to encourage open science, encouraging openness, integrity, and reproducibility in organizational research (Banks et al., 2019; Nosek et al., 2015).”
BackgroundTowards computational reproducibility: researcher perspectives on the use and sharing of software
AlNoamany, Y., & Borghi, J. (2018) · PeerJ Computer Science · doi:10.7717/peerj-cs.163 · Cited by 60
“At the level of scholarly publications, the TOP (Transparency and Openness Promotion) guidelines ( Nosek et al., 2015 ) address citation standards and the availability of research materials including data and software.”
BackgroundStrengthening causal inference from randomised controlled trials of complex interventions
Leroy, J. L., Frongillo, E. A., Kase, B. E., Alonso, S., Chen, M., Dohoo, I. R., Huybregts, L., Kadiyala, S., & Saville, N. M. (2022) · BMJ Global Health · doi:10.1136/bmjgh-2022-008597 · Cited by 61
Open Science: What, Why, and HowPreprint
Spellman, B., Gilbert, E. A., & Corker, K. S. (2017) · doi:10.31234/osf.io/ak6jr · Cited by 50
Self-citationHockerts, K. N., & Searcy, C. (2023) · Journal of Business Ethics · doi:10.1007/s10551-023-05386-0 · Cited by 56
“Next, as called for by our co-editors at JBE, Roloff and Zyphur (2019), we would also like to encourage more authors to consider pre-registering (Nosek et al., 2015) their corporate sustainability research.”
BackgroundSelective Inference: The Silent Killer of Replicability
Benjamini, Y. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.fc62b261 · Cited by 80
Quantitative Political Science Research Is Greatly Underpowered
Arel‐Bundock, V., Briggs, R. C., Doucouliagos, H., Aviña, M. M., & Stanley, T. D. (2024) · The Journal of Politics · doi:10.1086/734279 · Cited by 65
Janke, S., Daumiller, M. H., & Rudert, S. C. (2018) · Social Psychological and Personality Science · doi:10.1177/1948550618790227 · Cited by 55
The Reproducibility Crisis and Academic Libraries
Sayre, F., & Riegelman, A. (2018) · College & Research Libraries · doi:10.5860/crl.79.1.2 · Cited by 59
Liu, D. M., & Salganik, M. J. (2019) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023119849803 · Cited by 47
“The TOP Guidelines provide a framework journals can use to assess and improve their efforts to make scientific communication more open.”
BackgroundInfluentialTowards reproducible radiomics research: introduction of a database for radiomics studies
Akinci D’Antonoli, T., Cuocolo, R., Baessler, B., & Pinto dos Santos, D. (2023) · European Radiology · doi:10.1007/s00330-023-10095-3 · Cited by 49
“Moreover, this could be as simple as requesting a statement to do so [6].”
BackgroundMethodAn Audit of Political Behavior Research
Robison, J., Stevenson, R. T., Druckman, J. N., Jackman, S. D., Katz, J. N., & Vavreck, L. (2018) · SAGE Open · doi:10.1177/2158244018794769 · Cited by 49
“Furthermore, the discipline, more broadly, transformed due to a push for open science practices (e.g., Nosek et al., 2015) as well as the emergence of using social media to study political behavior (e.g., Murphy et al., 2014).”
BackgroundBüttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2019-101863 · Cited by 47
Vanelli, F. M., Kobiyama, M., & de Brito, M. M. (2022) · Hydrology and earth system sciences · doi:10.5194/hess-26-2301-2022 · Cited by 60
Gallegos-Riofrío, C. A., Arab, H., Carrasco-Torrontegui, A., & Gould, R. K. (2022) · Current Research in Environmental Sustainability · doi:10.1016/j.crsust.2022.100148 · Cited by 55
Why research integrity matters and how it can be improved
Bouter, L. M. (2023) · Accountability in Research · doi:10.1080/08989621.2023.2189010 · Cited by 62
Transparency in design science research
Hevner, A. R., Parsons, J., Brendel, B., Lukyanenko, R., Tiefenbeck, V., Tremblay, M. C., & Brocke, J. V. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114236 · Cited by 87
On the dimensionality of crystallized intelligence: A smartphone-based assessment
Steger, D., Schroeders, U., & Wilhelm, O. (2018) · Intelligence · doi:10.1016/j.intell.2018.12.002 · Cited by 44
Gilroy, S. P., & Kaplan, B. A. (2019) · Perspectives on Behavior Science · doi:10.1007/s40614-019-00202-5 · Cited by 52
“…studies and clinical trials, preregistration would offer similar benefits to behavior analytic research as well, as preregistration serves to better distinguish research that is confirmatory or hypothesis testing from research that is exploratory or hypothesis generating (Nosek et al., 2015).”
BackgroundInfluentialMayo‐Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. T. (2021) · Research Integrity and Peer Review · doi:10.1186/s41073-021-00112-8 · Cited by 57
Self-citationPackaging data analytical work reproducibly using R (and friends)Preprint
Marwick, B., Boettiger, C., & Mullen, L. A. (2017) · doi:10.7287/peerj.preprints.3192v1 · Cited by 50
Gaba, J. F., Siebert, M., Dupuy, A., Moher, D., & Naudet, F. (2020) · PLOS ONE · doi:10.1371/journal.pone.0237464 · Cited by 46
“It also ensures “transparency, openness, and reproducibility” [2] and evaluates the risk taken by patients participating in clinical trials.”
BackgroundKrafczyk, M. S., Shi, A., Bhaskar, A., Marinov, D., & Stodden, V. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0069 · Cited by 61
“sh and check the internal structure of software packages that the authors submit to be published alongside their article, thus extending existing standards [7].”
BackgroundChopra, F., Haaland, I., Roth, C., & Stegmann, A. (2023) · The Economic Journal · doi:10.1093/ej/uead060 · Cited by 85
The Creativity-Verification Cycle in Psychological Science: New Methods to Combat Old Idols
Wagenmakers, E.-J., Dutilh, G., & Sarafoglou, A. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691618771357 · Cited by 46
“Now, preregistration (Nosek et al., 2015) that more than 5,000 journals and societies have under consideration (for more information, see https://cos .io/our-services/top-guidelines/); •• is facilitated by online resources such as the Open Science Framework and aspredicted.org; and •• has been…”
Self-citationKim, J. S., Relyea, J. E., Burkhauser, M., Scherer, E., & Rich, P. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09609-6 · Cited by 75
“There is strong evidence that domain knowledge is a strong predictor of a student’s argument writing quality (Graham et al. 2018; Olinghouse et al. 2015).”
BackgroundQuestions and answers about conducting systematic reviews in sport and exercise psychology
Gunnell, K., Poitras, V. J., & Tod, D. (2020) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2019.1695141 · Cited by 45
“‘Open science’ describes strategies to increase research transparency and replicability (Nosek et al., 2015) and has been recommended for sport and exercise psychology researchers (Tamminen & Poucher, 2018).”
BackgroundCritical measurement issues in the assessment of social media influence on body image
Jarman, H. K., McLean, S. A., Griffiths, S., Teague, S. J., Rodgers, R. F., Paxton, S. J., Austen, E., Harris, E., Steward, T., Shatte, A., et al. (2022) · Body Image · doi:10.1016/j.bodyim.2021.12.007 · Cited by 46
Siebert, M., Gaba, J. F., Caquelin, L., Gouraud, H., Dupuy, A., Moher, D., & Naudet, F. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-038887 · Cited by 45
Reproducibility in Evolutionary Computation
López‐Ibáñez, M., Branke, J., & Paquete, L. (2021) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3466624 · Cited by 64
(Why) Are Open Research Practices the Future for the Study of Language Learning?
Marsden, E. J., & Morgan‐Short, K. (2023) · Language Learning · doi:10.1111/lang.12568 · Cited by 56
Świątkowski, W., Surret, F. L., Henry, J. L. A., Buchs, C., Visintin, E. P., & Butera, F. (2024) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2024.102295 · Cited by 75
Who Believes in the Storybook Image of the Scientist?
Veldkamp, C. L. S., Hartgerink, C. H. J., van Assen, M. A. L. M., & Wicherts, J. M. (2016) · Accountability in Research · doi:10.1080/08989621.2016.1268922 · Cited by 47
Too Good to be False: Nonsignificant Results Revisited
Hartgerink, C. H. J., Wicherts, J. M., & van Assen, M. A. L. M. (2017) · Collabra: Psychology · doi:10.1525/collabra.71 · Cited by 45
Improving the Replicability of Psychological Science Through Pedagogy
Hawkins, R. X. D., Smith, E. N., Au, C., Arias, J. M., Catapano, R., Hermann, E., Keil, M., Lampinen, A. K., Raposo, S., Reynolds, J., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917740427 · Cited by 48
“We strongly believe that methods and materials should be open, to allow for replication and productive science (Nosek et al., 2015), and hoped to empower the students to engage with and build on the published literature directly—without personal relationships or even personal contact.”
BackgroundMolecular Frame Reconstruction Using Time-Domain Photoionization Interferometry
Marceau, C., Makhija, V. S., Platzer, D., Naumov, A. Y., Corkum, P. B., Stolow, A., Villeneuve, D. M., & Hockett, P. (2017) · Physical Review Letters · doi:10.1103/physrevlett.119.083401 · Cited by 45
Practical Considerations for Navigating Registered Reports
Kiyonaga, A., & Scimeca, J. M. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2019.07.003 · Cited by 45
Flexible yet fair: blinding analyses in experimental psychology
Dutilh, G., Sarafoglou, A., & Wagenmakers, E. (2019) · Synthese · doi:10.1007/s11229-019-02456-7 · Cited by 55
“Preregistration is an important component of the Transparency andOpenness Promotion guidelines (Nosek et al. 2015; see alsoMunafò et al. 2017) that over 800 journals and societies currently have under consideration.”
Self-citationChen, M., Lv, G., Zhou, C., Lin, H., Ma, Z., Yue, S., Wen, Y., Zhang, F., Wang, J., Zhu, Z., et al. (2021) · Science China Earth Sciences · doi:10.1007/s11430-020-9759-0 · Cited by 48
Goals in Nutrition Science 2015–2020
Allison, D. B., Bassaganya-Riera, J., Burlingame, B., Brown, A. W., le Coutre, J., Dickson, S. L., van Eden, W., Garssen, J., Hontecillas, R., Khoo, C. S. H., et al. (2015) · Frontiers in Nutrition · doi:10.3389/fnut.2015.00026 · Cited by 45
“Second, there is a general movement in science for “Transparency and Openness Promotion,” formalized in “The TOP Guidelines” (30).”
Schulz, J. B., Cookson, M. R., & Hausmann, L. (2016) · Journal of Neurochemistry · doi:10.1111/jnc.13844 · Cited by 52
Strengthening a Weak Link: Transparency of Qualitative Modeling Tools
Jalali, M. S., & Beaulieu, É. (2022) · System Dynamics Review · doi:10.1002/sdr.1753 · Cited by 48
“Reproducibility is upheld as a characteristic of rigorous scienti fi c research that goes hand in hand with transparency (Nosek et al ., 2015).”
MethodA Meta-Analysis of Odor Thresholds and Odor Identification in Autism Spectrum Disorders
Larsson, M. U., Tirado, C., & Wiens, S. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00679 · Cited by 46
“Last, the raw data should be available online to facilitate future meta-analyses (Nosek et al., 2015; Munafò et al., 2017).”
MethodAntes, A. L., Kuykendall, A., & DuBois, J. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214595 · Cited by 50
Dennis, S. J., Garrett, P. M., Yim, H., Hamm, J., Osth, A. F., Sreekumar, V., & Stone, B. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01259-5 · Cited by 53
“Driven by the specter of the replication crisis (Open Science Collaboration, 2015), a deepening concern for the integrity of scientific evidence has led to the open science movement and the construction of best-practice guidelines for research (Nosek et al., 2015).”
BackgroundJournal Peer Review and Editorial Evaluation: Cautious Innovator or Sleepy Giant?
Horbach, S. P. J. M., & Halffman, W. (2019) · Minerva · doi:10.1007/s11024-019-09388-z · Cited by 58
Data management and sharing: Practices and perceptions of psychology researchers
Borghi, J., & Gulick, A. E. V. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252047 · Cited by 51
“In conjunction with the adoption of publication guidelines like TOP (Transparency and Openness Promotion) [18] and the use of open science badges to demarcate articles featuring pre-registered protocols and available materials, an increasing number of journals in and outside the field of psychology now require authors…”
BackgroundRyan, J. C., & Tipu, S. A. A. (2021) · Research Policy · doi:10.1016/j.respol.2021.104408 · Cited by 44
Mozersky, J., McIntosh, T., Walsh, H. A., Parsons, M. V., Goodman, M., & DuBois, J. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0261719 · Cited by 57
Ten (mostly) simple rules to future‐proof trait data in ecological and evolutionary sciences
Keller, A., Ankenbrand, M. J., Bruelheide, H., Dekeyzer, S., Enquist, B. J., Erfanian, M. B., Falster, D. S., Gallagher, R. V., Hammock, J., Kattge, J., et al. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14033 · Cited by 50
“Recent efforts to expand trait knowledge across the Tree of Life (Gallagher et al., 2020) call for datasets that are open and FAIR (Findable, Accessible, Interoperable, Reusable; Wilkinson et al., 2016), fundamental principles at the heart of the emergent Open Science movement (Nosek et al., 2015).”
BackgroundNiksirat, K. S., Goswami, L., Rao, P. S. B., Tyler, J., Silacci, A., Aliyu, S., Aebli, A., Wacharamanotham, C., & Cherubini, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580848 · Cited by 74
“Transparency is closely related to openness and is often mentioned together, such as in The Center of Open Science’s Transparency and Openness Promotion Guideline [65].”
BackgroundFilling in the Gaps: Memory Implications for Inferring Missing Content in Graphic Narratives
Magliano, J. P., Kopp, K. J., Higgs, K., & Rapp, D. N. (2016) · Discourse Processes · doi:10.1080/0163853x.2015.1136870 · Cited by 47
How stakeholders can respond to the rise of predatory journals
Lalu, M. M., Shamseer, L., Cobey, K. D., & Moher, D. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0257-4 · Cited by 43
Metadata
- Authors
- Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E. L., Simonsohn, U., Soderberg, C., Spellman, B. A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T.
- Affiliations
- Affiliations for the authors, all of whom are members of the TOP Guidelines Committee, are given in the supplementary materials.
- Corresponding author
- Nosek, B. A.
- ORCID
- Nosek, B. A.
- Alter, G.
- Banks, G. C.
- Borsboom, D.
- Bowman, S. D.
- Breckler, S. J.
- Buck, S.
- Chambers, C. D.
- Christensen, G.
- Dafoe, A.
- Eich, E.
- Glennerster, R.
- Goroff, D.
- Hesse, B.
- Humphreys, M.
- Ishiyama, J.
- Karlan, D.
- Lupia, A.
- Mabry, P.
- Madon, T.
- Malhotra, N.
- McNutt, M.
- Miguel, E.
- Paluck, E. Levy
- Simonsohn, U.
- Soderberg, C.
- Spellman, B. A.
- VandenBos, G.
- Vazire, S.
- Wagenmakers, E. J.
- Wilson, R.
- Yarkoni, T.
- Venue
- Science
- Published
- 2015-06-26
- Volume
- 348
- Issue
- 6242
- Pages
- 1422-1425
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- PMC
- PMC4550299
- OpenAlex
- W1767470961
- Cited by
- 2,708
- References
- 15
- Citations by year
- 2026: 214 · 2025: 203 · 2024: 200 · 2023: 232 · 2022: 278 · 2021: 306 · 2020: 360 · 2019: 276 · 2018: 256 · 2017: 211 · 2016: 134 · 2015: 38
- FWCI
- 418.98
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (publishedVersion)
- PubMed Central (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- open research culture, author guidelines, journal guidelines, transparency, research transparency, reproducibility
- MeSH terms
- Periodicals as Topic, Research, Reproducibility of Results, Guidelines as Topic, Peer Review, Research, Access to Information, Disclosure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.