Open science challenges, benefits and tips in early career and beyond
Open accessAbstract
The movement towards open science is a consequence of seemingly pervasive failures to replicate previous research. This transition comes with great benefits but also significant challenges that are likely to affect those who carry out the research, usually early career researchers (ECRs). Here, we describe key benefits, including reputational gains, increased chances of publication, and a broader increase in the reliability of research. The increased chances of publication are supported by exploratory analyses indicating null findings are substantially more likely to be published via open registered reports in comparison to more conventional methods. These benefits are balanced by challenges that we have encountered and that involve increased costs in terms of flexibility, time, and issues with the current incentive structure, all of which seem to affect ECRs acutely. Although there are major obstacles to the early adoption of open science, overall open science practices should benefit both the ECR and improve the quality of research. We review 3 benefits and 3 challenges and provide suggestions from the perspective of ECRs for moving towards open science practices, which we believe scientists and institutions at all levels would do well to consider.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The hypothesis that RRs receive greater scrutiny and ultimate rejection for negative than positive results was supported in two prior investigations.
“This hypothesis was supported in two prior investigations 26,27”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [26]
In “Benefit 1: Greater faith in research” (page not provided), the paper reports that 60.5% of hypotheses in its surveyed published RRs were unsupported and concludes that RRs increase the chances of publishing null findings. It says publication is guaranteed irrespective of statistical significance, rather than reporting two investigations finding greater rejection of negative results. (LLM, conf=0.83)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredMaking acceptance decisions prior to knowing the outcomes should and does mitigate publication bias favoring positive results.
“Making acceptance decisions prior to knowing the outcomes should and does mitigate publication bias favoring positive results 26,27 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [26]
The Introduction says registered reports can receive in-principle acceptance before data collection. In “Benefit 1: Greater faith in research,” the authors report more unsupported hypotheses in registered reports than estimated for traditional literature, but call the comparison exploratory and say it suggests increased publication of null findings. The unqualified claim that this process *does* mitigate publication bias is stronger than the paper’s empirical conclusion. (LLM, conf=0.83)
Experimental — accuracy not yet measuredFLOSS/OSS is an important form of digital infrastructure and technical career pathway for many developers.
“Free/libre open source software (FLOSS/OSS) is an important form of digital infrastructure and technical career pathway for many developers [4,20].”
From Codes of Conduct in Open Source (2024) · cites this paper as [4]
In “Benefit 2: New helpful systems,” the paper describes open-source software and repositories as tools that support research workflows and collaboration. “Benefit 3: Investment in your future” discusses career benefits for early-career researchers who adopt open-science practices. It does not establish FLOSS/OSS as a technical career pathway for many developers. (LLM, conf=0.82)
Experimental — accuracy not yet measuredPreregistration, and especially registered reports, helps to increase the reporting of null findings.
“Preregistration, and especially registered reports, helps to increase the reporting of null findings”
From Does preregistration improve the credibility of research findings? (2020) · cites this paper as Allen & Mehler, 2019
Initial evidence suggests that Registered Reports are effective at mitigating publication bias with a sharp increase in publishing negative results compared to the standard model.
“Initial evidence suggests that RRs are (1) effective at mitigating publication bias with a sharp increase in publishing negative results compared to the standard model 26,27 ,”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001779 · Cited by 12,728 · see citing works
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 2× · see citing works
“These approaches require hypotheses and analysis pipelines to be declared publicly before data collection [2,14,28,29] (although protocols can be embargoed).”
BackgroundPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 3× · see citing works
“…data, and research materials [2,18]; valuing replications and reanalyses [2,5,6,19,20]; changes in statistical approaches with regards to power [21,22] and how evidence is assessed [23]; interactive and more transparent ways of presenting data graphically [24,25]; potentially the use of…”
BackgroundInfluential1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · see citing works
Jupyter Notebooks – a publishing format for reproducible computational workflows
Kluyver, T. A., Ragan-Kelley, B., Fernando, P., Granger, B., Bussonnier, M., Frederic, J., Kelley, K., Hamrick, J. B., Grout, J., Corlay, S., et al. (2016) · IOS Press eBooks · doi:10.3233/978-1-61499-649-1-87 · Cited by 3,191 · see citing works
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708 · see citing works
The Human Connectome Project: A data acquisition perspective
Essen, D. C. V., Uǧurbil, K., Auerbach, E. J., Barch, D. M., Behrens, T. E. J., Bucholz, R. D., Chang, A., Chen, L., Corbetta, M., Curtiss, S. W., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.018 · Cited by 2,605 · see citing works
Extinction risk and conservation of the world’s sharks and rays
Dulvy, N. K., Fowler, S. L., Musick, J. A., Cavanagh, R. D., Kyne, P. M., Harrison, L. R., Carlson, J. K., Davidson, L. N., Fordham, S. V., Francis, M. P., et al. (2014) · eLife · doi:10.7554/elife.00590 · Cited by 2,047 · see citing works
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · see citing works
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
A century of trends in adult human height
NCD Risk Factor Collaboration (NCD-RisC) (2016) · eLife · doi:10.7554/elife.13410 · Cited by 1,558 · see citing works
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500 · see citing works
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301 · see citing works
The economics of reproducibility in preclinical research
Freedman, L. P., Cockburn, I. M., & Simcoe, T. S. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002165 · Cited by 1,055 · see citing works
The ENIGMA Consortium: large-scale collaborative analyses of neuroimaging and genetic data
the Alzheimer’s Disease Neuroimaging Initiative, E. C., Thompson, P. M., Stein, J. L., Medland, S. E., Hibar, D. P., Vasquez, A. A., Renteria, M. E., Toro, R., Jahanshad, N., Schumann, G., et al. (2014) · Brain Imaging and Behavior · doi:10.1007/s11682-013-9269-5 · Cited by 838 · see citing works
How Open Science Helps Researchers Succeed
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., McDougall, D., Nosek, B. A., Ram, K., Soderberg, C. K., et al. (2016) · eLife · doi:10.7554/elife.16800 · Cited by 798 · see citing works
Szucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761 · see citing works
The Development of Open Access Journal Publishing from 1993 to 2009
Laakso, M., Welling, P., Bukvova, H., Nyman, L., Björk, B.-C., & Hedlund, T. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020961 · Cited by 755 · see citing works
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., et al. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002456 · Cited by 700 · see citing works
Data reuse and the open data citation advantage
Piwowar, H., & Vision, T. (2013) · PeerJ · doi:10.7717/peerj.175 · Cited by 629 · see citing works
Reviewer bias in single- versus double-blind peer review
Tomkins, A., Zhang, M., & Heavlin, W. D. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1707323114 · Cited by 605 · see citing works
Registered Reports: A new publishing initiative at Cortex
Chambers, C. (2012) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 574 · see citing works
The prevalence of statistical reporting errors in psychology (1985–2013)
Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0664-2 · Cited by 510 · cited 1× · see citing works
“Another useful checking tool is statcheck, an R toolbox that scans documents for inconsistencies in reported statistical values [64].”
MethodThe Availability of Research Data Declines Rapidly with Article Age
Vines, T. H., Albert, A. Y. K., Andrew, R. L., Débarre, F., Bock, D. G., Franklin, M. T., Gilbert, K. J., Moore, J.-S., Renaut, S., & Rennison, D. J. (2014) · Current Biology · doi:10.1016/j.cub.2013.11.014 · Cited by 482 · see citing works
Small is beautiful: In defense of the small-N design
Smith, P. L., & Little, D. R. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1451-8 · Cited by 466 · see citing works
Assessing scientists for hiring, promotion, and tenure
Moher, D., Naudet, F., Cristea, I. A., Miedema, F., Ioannidis, J. P. A., & Goodman, S. N. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004089 · Cited by 462 · see citing works
Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference
Dienes, Z. P. (2008) · Cited by 418 · see citing works
Enhancing reproducibility for computational methods
Stodden, V., Mcnutt, M. K., Bailey, D. H., Deelman, E., Gil, Y., Hanson, B., Heroux, M. A., Ioannidis, J. P. A., & Taufer, M. (2016) · Science · doi:10.1126/science.aah6168 · Cited by 414 · see citing works
Moshontz, H., Campbell, L., Ebersole, C. R., IJzerman, H., Urry, H. L., Forscher, P. S., Grahe, J. E., McCarthy, R. J., Musser, E. D., Antfolk, J., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918797607 · Cited by 377 · see citing works
Meta-assessment of bias in science
Fanelli, D., Costas, R., & Ioannidis, J. P. A. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618569114 · Cited by 364 · see citing works
Hormonal Contraception and the Risk of HIV Acquisition: An Individual Participant Data Meta-analysis
MORRISON, C. S., Chen, P., Kwok, C., Baeten, J. M., Brown, J., Crook, A. M., Damme, L. V., Delany‐Moretlwe, S., Francis, S. C., Friedland, B., et al. (2015) · PLoS Medicine · doi:10.1371/journal.pmed.1001778 · Cited by 349 · see citing works
Most published research findings are false-but a little replication goes a long way.
Moonesinghe, R., Khoury, M. J., & Janssens, A. C. J. W. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040028 · Cited by 341 · see citing works
The Allen Brain Atlas: 5 years and beyond
Jones, A. R., Overly, C. C., & Sunkin, S. M. (2009) · Nature reviews. Neuroscience · doi:10.1038/nrn2722 · Cited by 322 · see citing works
An open investigation of the reproducibility of cancer biology research
Errington, T. M., Iorns, E., Gunn, W., Tan, F. E., Lomax, J., & Nosek, B. A. (2014) · eLife · doi:10.7554/elife.04333 · Cited by 312 · see citing works
Gorgolewski, K. J., Alfaro-Almagro, F., Auer, T., Bellec, P., Capotă, M., Chakravarty, M. M., Churchill, N. W., Cohen, A. L., Craddock, R. C., Devenyi, G. A., et al. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005209 · Cited by 311 · see citing works
Reproducible Research Practices and Transparency across the Biomedical Literature
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002333 · Cited by 310 · see citing works
How scientists fool themselves – and how they can stop
Nuzzo, R. (2015) · Nature · doi:10.1038/526182a · Cited by 296 · see citing works
Our path to better science in less time using open data science tools
Lowndes, J. S., Best, B. D., Scarborough, C., Afflerbach, J. C., Frazier, M. R., O’Hara, C. C., Jiang, N., & Halpern, B. S. (2017) · Nature Ecology & Evolution · doi:10.1038/s41559-017-0160 · Cited by 283 · see citing works
Wallach, J. D., Boyack, K. W., & Ioannidis, J. P. A. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2006930 · Cited by 269 · see citing works
An Open Resource for Non-human Primate Imaging
Milham, M. P., Ai, L., Koo, B., Xu, T., Amiez, C., Balezeau, F., Baxter, M. G., Blezer, E. L. A., Brochier, T. G., Chen, A., et al. (2018) · Neuron · doi:10.1016/j.neuron.2018.08.039 · Cited by 247 · see citing works
Nosek, B. A., & Errington, T. M. (2017) · eLife · doi:10.7554/elife.23383 · Cited by 242 · see citing works
Taichman, D. B., Sahni, P., Pinborg, A. B., Peiperl, L., Laine, C. M., James, A., Hong, S., Haileamlak, A., Gollogly, L., Godlee, F., et al. (2017) · PLoS Medicine · doi:10.1371/journal.pmed.1002315 · Cited by 220 · see citing works
The Preregistration RevolutionPreprint
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2017) · doi:10.31219/osf.io/2dxu5 · Cited by 220 · see citing works
Developing Global Norms for Sharing Data and Results during Public Health Emergencies
Modjarrad, K., Moorthy, V., Millett, P. D., Gsell, P., Roth, C. E., & Kiény, M.-P. (2016) · PLoS Medicine · doi:10.1371/journal.pmed.1001935 · Cited by 193 · see citing works
Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions
Higginson, A. D., & Munafò, M. R. (2016) · PLOS Biology · doi:10.1371/journal.pbio.2000995 · Cited by 192 · see citing works
On the value of preprints: An early career researcher perspective
Sarabipour, S., Debat, H., Emmott, E., Burgess, S., Schwessinger, B., & Hensel, Z. (2019) · PLoS Biology · doi:10.1371/journal.pbio.3000151 · Cited by 184 · see citing works
The seven deadly sins of psychology a manifesto for reforming the culture of scientific practice
Kaplan, D. M., Sowman, P. F., Abel, L., Arbige, S., Chandler, C. B., Chen, C., Chard, T., Higgins, W. C., Jones, S., Murray, L., et al. (2020) · Philosophical Psychology · doi:10.1080/09515089.2020.1837360 · Cited by 173 · see citing works
Taichman, D. B., Sahni, P., Pinborg, A., Peiperl, L., Laine, C., James, A., Hong, S.-T., Haileamlak, A., Gollogly, L., Godlee, F., et al. (2017) · New England Journal of Medicine · doi:10.1056/nejme1705439 · Cited by 156 · see citing works
Mapping the universe of registered reports
Hardwicke, T. E., & Ioannidis, J. P. A. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0444-y · Cited by 147 · see citing works
Data sharing: Make outbreak research open access
Yozwiak, N. L., Schaffner, S. F., & Sabeti, P. C. (2015) · Nature · doi:10.1038/518477a · Cited by 139 · see citing works
R Markdown: Integrating A Reproducible Analysis Tool into Introductory Statistics
Baumer, B. S., Çetinkaya-Rundel, M., Bray, A. P., Loi, L., & Horton, N. J. (2014) · Technology Innovations in Statistics Education · doi:10.5070/t581020118 · Cited by 136 · see citing works
Irreproducibility in Preclinical Biomedical Research: Perceptions, Uncertainties, and Knowledge Gaps
Jarvis, M. F., & Williams, M. (2016) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2015.12.001 · Cited by 129 · see citing works
The Seven Deadly Sins of Psychology: A Manifesto for Reforming the Culture of Scientific Practice
Chambers, C. (2019) · Cited by 125 · see citing works
The Postdoctoral Experience Revisited
National Academy of Sciences, N. A. O. E. (2014) · National Academies Press eBooks · doi:10.17226/18982 · Cited by 124 · see citing works
Perceptions of Emergency Department Crowding in the Commonwealth of Pennsylvania
Pines, J. M., Isserman, J. A., & Kelly, J. J. (2013) · Western Journal of Emergency Medicine · doi:10.5811/westjem.2011.5.6700 · Cited by 119 · see citing works
Five selfish reasons to work reproducibly
Markowetz, F. M. (2015) · Genome biology · doi:10.1186/s13059-015-0850-7 · Cited by 108 · see citing works
Toward standard practices for sharing computer code and programs in neuroscience
Eglen, S. J., Marwick, B., Halchenko, Y. O., Hanke, M., Sufi, S., Gleeson, P., Silver, R. A., Davison, A. P., Lanyon, L., Abrams, M. B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4550 · Cited by 108 · see citing works
Research integrity: Don't let transparency damage science
Lewandowsky, S., & Bishop, D. (2016) · Nature · doi:10.1038/529459a · Cited by 103 · see citing works
Fontanarosa, P., Bauchner, H., & Flanagin, A. (2017) · JAMA · doi:10.1001/jama.2017.19341 · Cited by 102 · see citing works
Bishop, D. V. M., & Thompson, P. A. (2016) · PeerJ · doi:10.7717/peerj.1715 · Cited by 90 · see citing works
Open Science challenges, benefits and tips in early career and beyondPreprint
Allen, C. P. G., & Mehler, D. M. A. (2018) · doi:10.31234/osf.io/3czyt · Cited by 89 · see citing works
Self-citationExploratory reports: A new article type for Cortex
McIntosh, R. D. (2017) · Cortex · doi:10.1016/j.cortex.2017.07.014 · Cited by 87 · see citing works
Data Sharing from Clinical Trials — A Research Funder’s Perspective
Kiley, R., Peatfield, T., Hansen, J., & Reddington, F. (2017) · New England Journal of Medicine · doi:10.1056/nejmsb1708278 · Cited by 85 · see citing works
Poldrack, R. A. (2019) · Neuron · doi:10.1016/j.neuron.2018.11.030 · Cited by 84 · see citing works
Ramsey, S. D., & Scoggins, J. F. (2008) · The Oncologist · doi:10.1634/theoncologist.2008-0133 · Cited by 83 · see citing works
Digital open science—Teaching digital tools for reproducible and transparent research
Toelch, U., & Ostwald, D. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2006022 · Cited by 80 · see citing works
Personality in the chimpanzees of Gombe National Park
Weiß, A., Wilson, M. L., Collins, D., Mjungu, D. C., Kamenya, S. M., Foerster, S., & Pusey, A. E. (2017) · Scientific Data · doi:10.1038/sdata.2017.146 · Cited by 79 · see citing works
Algermissen, J., & Mehler, D. M. A. (2018) · Journal of Neurophysiology · doi:10.1152/jn.00765.2017 · Cited by 73 · see citing works
Self-citationSeven Selfish Reasons for Preregistration
Dutilh, E.-J. W. A. G. (2016) · APS observer · Cited by 68 · see citing works
Hanel, P. H. P., & Mehler, D. M. (2019) · Public Understanding of Science · doi:10.1177/0963662519834193 · Cited by 66 · see citing works
Self-citationPickett, C. L., Corb, B., Matthews, C. R., Sundquist, W. I., & Berg, J. M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1509901112 · Cited by 60 · see citing works
Rigor or mortis: best practices for preclinical research in neuroscience.
Steward, O., & Balice-Gordon, R. (2014) · Neuron · doi:10.1016/j.neuron.2014.10.042 · Cited by 58 · see citing works
Poupon, V., Seyller, A., & Rouleau, G. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.07.026 · Cited by 36 · see citing works
Faculty promotion must assess reproducibility
Flier, J. S. (2017) · Nature · doi:10.1038/549133a · Cited by 35 · see citing works
(2017) · eLife · doi:10.7554/elife.23693 · Cited by 27 · see citing works
First analysis of ‘pre-registered’ studies shows sharp rise in null findings
Warren, M. (2018) · Nature · doi:10.1038/d41586-018-07118-1 · Cited by 26 · see citing works
archivist: An R Package for Managing, Recording and Restoring Data Analysis Results
Biecek, P., & Kosiński, M. (2017) · Journal of Statistical Software · doi:10.18637/jss.v082.i11 · Cited by 21 · see citing works
Many junior scientists need to take a hard look at their job prospects
(2017) · Nature · doi:10.1038/550429a · Cited by 21 · see citing works
Addgene: Making Materials Sharing “Science As Usual”
Kamens, J. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001991 · Cited by 18 · see citing works
Fallibility in science: Responding to errors in the work of oneself and othersPreprint
Bishop, D. (2017) · doi:10.7287/peerj.preprints.3486v1 · Cited by 15 · see citing works
Maizey, L., & Tzavella, L. (2018) · Cortex · doi:10.1016/j.cortex.2018.12.015 · Cited by 11 · see citing works
Mehler, D. M. A. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-816179-1.00010-4 · Cited by 8 · see citing works
Self-citationFallibility in science: Responding to errors in the work of oneself and othersPreprint
Bishop, D. (2017) · doi:10.7287/peerj.preprints.3486 · Cited by 6 · see citing works
Harris, R. (2017) · C&EN Global Enterprise · doi:10.1021/cen-09547-editorial · Cited by 4 · see citing works
Referees should exercise their rights
(2018) · Nature · doi:10.1038/d41586-018-06006-y · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [21]Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013; 14: 365– 376. 10.1038/nrn3475 23571845 · DOI · Paper page
- [24]Introducing eLife’s first computationally reproducible article [Internet]. 2019. Available from: https://elifesciences.org/labs/ad58f08d/introducing-elife-s-first-computationally-reproducible-article. [cited 2019 March 25].
- [26]Tomkins A, Zhang M, Heavlin WD. Reviewer bias in single- versus double-blind peer review. Proc Natl Acad Sci U S A. National Academy of Sciences; 2017; 114: 12708– 12713. 10.1073/pnas.1707323114 29138317 PMC5715744 · DOI · Paper page
Cited by
Sorted by citations
Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0
Percie du Sert, N., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000411 · Cited by 2,677 · see citing works
Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01142-4 · Cited by 289 · see citing works
Protecting against researcher bias in secondary data analysis: challenges and potential solutions
Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R. (2022) · European Journal of Epidemiology · doi:10.1007/s10654-021-00839-0 · Cited by 178 · see citing works
What’s next for Registered Reports?
Chambers, C. (2019) · Nature · doi:10.1038/d41586-019-02674-6 · Cited by 154 · see citing works
Morrison, A. B., & Richmond, L. L. (2020) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-019-0201-4 · Cited by 87 · see citing works
O'Daffer, A., Colt, S. F., Wasil, A. R., & Lau, N. (2022) · JMIR Mental Health · doi:10.2196/40924 · Cited by 72 · see citing works
Integrating Qualitative Methods and Open Science: Five Principles for More Trustworthy Research*
Humphreys, L., Lewis, N. A., Sender, K., & Won, A. S. (2021) · Journal of Communication · doi:10.1093/joc/jqab026 · Cited by 69 · see citing works
Refining animal research: The Animal Study Registry
Bert, B., Heinl, C., Chmielewska, J., Schwarz, F., Grune, B., Hensel, A., Greiner, M., & Schönfelder, G. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000463 · Cited by 67 · see citing works
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81 · see citing works
Does preregistration improve the credibility of research findings?
Rubin, M. (2020) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.16.4.p376 · Cited by 61 · see citing works
The Registered Reports Revolution Lessons in Cultural Reform
Chambers, C. (2019) · Significance · doi:10.1111/j.1740-9713.2019.01299.x · Cited by 43 · see citing works
Messing with Merton: The intersection between Open Science practices and Mertonian values
Hosseini, M., Senabre Hidalgo, E., Horbach, S. P. J. M., Guttinger, S., & Penders, B. (2022) · doi:10.31235/osf.io/9nyh8 · Cited by 30 · see citing works
Research transparency in dental research: A programmatic analysis
Raittio, E., Sofi‐Mahmudi, A., & Uribe, S. E. (2023) · European Journal of Oral Sciences · doi:10.1111/eos.12908 · Cited by 15 · see citing works
Transparency of COVID-19-Related Research in Dental Journals
Sofi-Mahmudi, A., & Raittio, E. (2022) · Frontiers in Oral Health · doi:10.3389/froh.2022.871033 · Cited by 15 · see citing works
Forensic DNA Interpretation and Human Factors: Improving the Practice Through a Systems Approach
Taylor, M. K. (2024) · doi:10.6028/nist.ir.8503 · Cited by 14 · see citing works
Cross-Sectional Evaluation of Open Science Practices at Imaging Journals: A Meta-Research Study
Kashif Al-Ghita, M., Cobey, K., Moher, D., Leeflang, M. M. G., Ebrahimzadeh, S., Lam, E., Rooprai, P., Khalil, A. A., Islam, N., Algodi, H., et al. (2024) · Canadian Association of Radiologists Journal · doi:10.1177/08465371231211290 · Cited by 8 · see citing works
Endorsement of open science practices by dental journals: A meta-research study
Santos, W. V. D. O., Dotto, L., Ferreira, T. D. G. M., & Sarkis-Onofre, R. (2024) · Journal of Dentistry · doi:10.1016/j.jdent.2024.104869 · Cited by 8 · see citing works
Use of Open Science Practices in Surgical Journals
Pathak, K., Marwaha, J. S., Chen, H. W., Krumholz, H. M., & Matthews, J. B. (2024) · JAMA Surgery · doi:10.1001/jamasurg.2023.5389 · Cited by 6 · see citing works
Codes of Conduct in Open Source
Frluckaj, H., & Howison, J. (2024) · Equity, Diversity, and Inclusion in Software Engineering · doi:10.1007/978-1-4842-9651-6_17 · Cited by 3 · see citing works
- Authors
- Allen, C., & Mehler, D. M. A.
- Affiliations
- Cardiff University (GB)
- University of Münster (DE)
- Corresponding author
- Allen, Christopher
- Mehler, David M. A.
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-05-01
- Volume
- 17
- Issue
- 5
- Pages
- e3000246
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC6513108
- OpenAlex
- W2943216811
- Cited by
- 608
- References
- 86
- Citations by year
- 2026: 28 · 2025: 71 · 2024: 76 · 2023: 88 · 2022: 94 · 2021: 128 · 2020: 82 · 2019: 37 · 2018: 4
- FWCI
- 100.65
- Type
- journal article
- Language
- en
- Update notices
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ORCA Online Research @Cardiff (Cardiff University) (acceptedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Durham Research Online (Durham University) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Wellcome Trust (104943/Z/14/Z, 104943)
- Health and Care Research Wales (HS/14/20, 104943/Z/14/Z)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
- Keywords
- open science, early-career researchers, research reproducibility, replication failure, negative results, publication bias, research incentives, research reliability, academic reputation
- MeSH terms
- Career Choice, Science, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
