CoChat
Sign Up
Cited by 28Academic Publishing and Open AccessResearch, Science, and AcademiaAcademic integrity and plagiarism

Framework for advancing rigorous research

Open access
Koroshetz, Walter J, Behrman, Shannon, Brame, Cynthia J, Branchaw, Janet L, Brown, Emery N, Clark, Erin A, Dockterman, David, Elm, Jordan J +19 more·2020·eLife
DOI 10.7554/elife.55915PMC PMC7056268
Open this paper in CoChat — free account

Abstract

There is a pressing need to increase the rigor of research in the life and biomedical sciences. To address this issue, we propose that communities of 'rigor champions' be established to campaign for reforms of the research culture that has led to shortcomings in rigor. These communities of rigor champions would also assist in the development and adoption of a comprehensive educational platform that would teach the principles of rigorous science to researchers at all career stages.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7554/elife.55915

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Numerous reports that decry failures to replicate findings in behavioral and clinical neuroscience have triggered increased emphasis on the importance of replication.

    “Numerous reports that decry failures to replicate findings in behavioral and clinical neuroscience have triggered increased emphasis on the importance of replication [1-5].”

    From A feature and not a bug: Replication failures can lead to new biology (2026) · cites this paper as [3]

    The Abstract (page not provided) calls for greater rigor in life and biomedical research and describes a neuroscience-focused workshop on improving research practices. It does not say that numerous reports of failed replications in behavioral and clinical neuroscience triggered an increased emphasis on replication. (LLM, conf=0.73)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. Alberts B Cicerone RJ Fienberg SE Kamb A McNutt M Nerem RM Schekman R Shiffrin R Stodden V Suresh S Zuber MT Pope BK Jamieson KH 2015 Self-correction in science at work Science 348 1420 1422 10.1126/science.aab3847 26113701 · DOI
  2. Begley CG Buchan AM Dirnagl U 2015 Robust Research: Institutions must do their part for reproducibility Nature 525 25 27 10.1038/525025a 26333454 · DOI
  3. Bertuzzi S Jamaleddine Z 2016 Capturing the value of biomedical research Cell 165 9 12 10.1016/j.cell.2016.03.004 27015300 · DOI
  4. Bosch G 2018 Train PhD students to be thinkers not just specialists Nature 554 277 10.1038/d41586-018-01853-1 32094741 · DOI
  5. Bosch G Casadevall A 2017 Graduate biomedical science education needs a new philosophy mBio 8 17 10.1128/mBio.01539-17 PMC5736909 29259084 · DOI
  6. Bradforth SE Miller ER Dichtel WR Leibovich AK Feig AL Martin JD Bjorkman KS Schultz ZD Smith TL 2015 University Learning: Improve undergraduate science education Nature 523 282 284 10.1038/523282a 26178951 · DOI
  7. Brown JS Adler RP 2008 Minds on fire: open education, the long tail, and learning 2.0 EDUCAUSE Review 43 16 32
  8. Button KS Chambers CD Lawrence N Munafò MR 2020 Grassroots training for reproducible science: a consortium-based approach to the empirical dissertation Psychology Learning & Teaching 19 77 90 10.1177/1475725719857659 · DOI
  9. Casadevall A Ellis LM Davies EW McFall-Ngai M Fang FC 2016 A framework for improving the quality of research in the biological sciences mBio 7 e01256 10.1128/mBio.01256-16 27578756 PMC4999552 · DOI · Paper page
  10. Casadevall A Fang FC 2016 Rigorous Science: A how-to guide mBio 7 e01902 10.1128/mBio.01902-16 27834205 PMC5111411 · DOI
  11. Casci T Adams E 2020 Setting the right tone eLife 9 e55543 10.7554/eLife.55543 32036857 PMC7010405 · DOI
  12. Collins FS Tabak LA 2014 NIH plans to enhance reproducibility Nature 505 612 613 10.1038/505612a 24482835 PMC4058759 · DOI · Paper page
  13. Corwin LA Graham MJ Dolan EL 2015 Modeling course-based undergraduate research experiences: an agenda for future research and evaluation CBE—Life Sciences Education 14 es1 10.1187/cbe.14-10-0167 25687826 PMC4353087 · DOI
  14. Cressey D 2015 UK funders demand strong statistics for animal studies Nature 520 271 272 10.1038/520271a 25877180 · DOI
  15. Dirnagl U Kurreck C Castaños-Vélez E Bernard R 2018 Quality management for academic laboratories: burden or boon? EMBO Reports 19 e47143 10.15252/embr.201847143 30341068 PMC6216282 · DOI · Paper page
  16. Dirnagl U 2019 Resolving the ttension between exploration and confirmation in preclinical biomedical research Handbook of Experimental Pharmacology Berlin, Heidelberg Springer 10.1007/164_2019_278 31696348 · DOI
  17. D’Mello S Lehman B Pekrun R Graesser A 2014 Confusion can be beneficial for learning Learning and Instruction 29 153 170 10.1016/j.learninstruc.2012.05.003 · DOI
  18. Handelsman J Ebert-May D Beichner R Bruns P Chang A DeHaan R Gentile J Lauffer S Stewart J Tilghman SM Wood WB 2004 Scientific teaching Science 304 521 522 10.1126/science.1096022 15105480 · DOI · Paper page
  19. Howitt SM Wilson AN 2014 Revisiting “Is the scientific paper a fraud?” EMBO Reports 15 481 484 10.1002/embr.201338302 24723688 PMC4210093 · DOI
  20. Ioannidis JPA Greenland S Hlatky MA Khoury MJ Macleod MR Moher D Schulz KF Tibshirani R 2014 Increasing value and reducing waste in research design, conduct, and analysis The Lancet 383 166 175 10.1016/S0140-6736(13)62227-8 PMC4697939 24411645 · DOI · Paper page
  21. Kelner KL 2013 Playing our part Science Translational Medicine 5 190ed7 10.1126/scitranslmed.3006661 · DOI
  22. Lundwall RA 2019 Changing institutional incentives to foster sound scientific practices: one department Infant Behavior and Development 55 69 76 10.1016/j.infbeh.2019.03.006 30933839 · DOI
  23. MacLeod MR Lawson McLean A Kyriakopoulou A Serghiou S de Wilde A Sherratt N Hirst T Hemblade R Bahor Z Nunes-Fonseca C Potluru A Thomson A Baginskaite J Baginskitae J Egan K Vesterinen H Currie GL Churilov L Howells DW Sena ES 2015 Risk of bias in reports of in vivo research: a focus for improvement PLOS Biology 13 e1002273 10.1371/journal.pbio.1002273 26460723 PMC4603955 · DOI · Paper page
  24. McNutt M 2014 Journals unite for reproducibility Science 346 679 10.1126/science.aaa1724 25383411 · DOI
  25. Minner DD Levy AJ Century J 2010 Inquiry-based science instruction-what is it and does it matter? results from a research synthesis years 1984 to 2002 Journal of Research in Science Teaching 47 474 496 10.1002/tea.20347 · DOI · Paper page
  26. Moher D Naudet F Cristea IA Miedema F Ioannidis JPA Goodman SN 2018 Assessing scientists for hiring, promotion, and tenure PLOS Biology 16 e2004089 10.1371/journal.pbio.2004089 29596415 PMC5892914 · DOI · Paper page
  27. Munafò MR Nosek BA Bishop DVM Button KS Chambers CD Percie du Sert N Simonsohn U Wagenmakers E-J Ware JJ Ioannidis JPA 2017 A manifesto for reproducible science Nature Human Behaviour 1 0021 10.1038/s41562-016-0021 PMC7610724 33954258 · DOI · Paper page
  28. Munafò MR Chambers CD Collins AM Fortunato L Macleod MR 2020 Research culture and reproducibility Trends in Cognitive Sciences 24 91 93 10.1016/j.tics.2019.12.002 31892459 · DOI
  29. Munafò MR Davey Smith G 2018 Robust research needs many lines of evidence Nature 553 399 401 10.1038/d41586-018-01023-3 29368721 · DOI · Paper page
  30. National Research Council 2015 Enhancing the Effectiveness of Team Science The National Academies Press 10.17226/19007 26247083 · DOI
  31. Nosek BA Spies JR Motyl M 2012 Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability Perspectives on Psychological Science 7 615 631 10.1177/1745691612459058 26168121 PMC10540222 · DOI
  32. Pfund C Maidl Pribbenow C Branchaw J Miller Lauffer S Handelsman J 2006 The merits of training mentors Science 311 473 474 10.1126/science.1123806 16439648 · DOI
  33. PLOS Biology 2018 Fifteen years in, what next for PLOS biology? PLOS Biology 16 e3000049 10.1371/journal.pbio.3000049 30321167 PMC6188623 · DOI
  34. Raman IM 2014 How to be a graduate advisee Neuron 81 9 11 10.1016/j.neuron.2013.12.030 24411728 · DOI
  35. Silberberg SD Crawford DC Finkelstein R Koroshetz WJ Blank RD Freeze HH Garrison HH Seger YR 2017 Shake up conferences Nature 548 153 154 10.1038/548153a 28796229 · DOI · Paper page
  36. Spencer KC McDaniels M Utzerath E Rogers JG Sorkness CA Asquith P Pfund C 2018 Building a sustainable national infrastructure to expand research mentor training CBE—Life Sciences Education 17 ar48 10.1187/cbe.18-03-0034 30153422 PMC6234808 · DOI
  37. Strech D Weissgerber T Dirnagl U QUEST Group 2020 Improving the trustworthiness, usefulness, and ethics of biomedical research through an innovative and comprehensive institutional initiative PLOS Biology 18 e3000576 10.1371/journal.pbio.3000576 32045410 PMC7012388 · DOI
  38. Walkington C Bernacki ML 2018 Personalization of instruction: design dimensions and implications for cognition The Journal of Experimental Education 86 50 68 10.1080/00220973.2017.1380590 · DOI
  39. Wasserstein RL Schirm AL Lazar NA 2019 Moving to a world beyond "p < 0.05" The American Statistician 73 1 19 10.1080/00031305.2019.1583913 · DOI · Paper page
  40. Yeager DS Henderson MD Paunesku D Walton GM D'Mello S Spitzer BJ Duckworth AL 2014 Boring but important: a self-transcendent purpose for learning fosters academic self-regulation Journal of Personality and Social Psychology 107 559 580 10.1037/a0037637 25222648 PMC4643833 · DOI

Cited by

Sorted by citations

Authors
Koroshetz, W. J., Behrman, S., Brame, C. J., Branchaw, J. L., Brown, E. N., Clark, E. A., Dockterman, D., Elm, J. J., Gay, P. L., Green, K. M., Hsi, S., Kaplitt, M. G., Kolber, B. J., Kolodkin, A. L., Lipscombe, D., MacLeod, M. R., McKinney, C. C., Munafò, M. R., Oakley, B., Olimpo, J. T., Percie du Sert, N., Raman, I. M., Riley, C., Shelton, A. L., Uzzo, S. M., Crawford, D. C., & Silberberg, S. D.
Affiliations
  • National Institute of Neurological Disorders and Stroke (US)
  • iBiology, San Francisco, United States
  • Vanderbilt University (US)
  • University of Wisconsin–Madison (US)
  • Massachusetts General Hospital (US)
  • Brandeis University (US)
  • Harvard University (US)
  • Medical University of South Carolina (US)
  • Planetary Science Institute (US)
  • University of Michigan (US)
  • Concord Consortium (US)
  • Weill Cornell Medicine (US)
  • Duquesne University (US)
  • Johns Hopkins University (US)
  • Brown University (US)
  • University of Edinburgh (GB)
  • Georgetown University Medical Center (US)
  • MRC Integrative Epidemiology Unit, School of Psychological Science, University of Bristol, Bristol, United Kingdom
  • Oakland University (US)
  • The University of Texas at El Paso (US)
  • National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), London, United Kingdom
  • Northwestern University (US)
  • Complexly, Missoula, United States
  • New York Hall of Science (US)
Corresponding author
  • Crawford, Devon C
  • Silberberg, Shai D
ORCID
Venue
eLife
Publisher
eLife Sciences Publications, Ltd
Published
2020-03-04
Volume
9
ISSN
2050-084X (ISSN-L)
DOI
10.7554/elife.55915
PMC
PMC7056268
OpenAlex
W3009484667
Cited by
28
References
4
Citations by year
2026: 1 · 2025: 2 · 2024: 5 · 2023: 7 · 2022: 4 · 2021: 3 · 2020: 6
FWCI
4.65
Type
journal article
Language
en
Open access
Licence
Locations
  • eLife (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, public-domain)
  • PubMed Central (publishedVersion, public-domain)
  • PubMed (publishedVersion)
  • Explore Bristol Research (publishedVersion, public-domain)
  • Bristol Research (University of Bristol) (acceptedVersion, public-domain)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Medical Research Council (MC_UU_00011/7)
  • National Institute of Neurological Disorders and Stroke (R01NS055251)
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
  • Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
  • Social Sciences › Safety Research › Academic integrity and plagiarism
Keywords
research rigor, research integrity, science education, biomedical sciences, research training
MeSH terms
Humans, Research Design, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review