CoChat
Sign Up
Cited by 25Academic Writing and PublishingPsychology Research and BibliometricsMeta-analysis and systematic reviews

From methods to monographs: fostering a culture of research quality

Open access
Crawford, Devon C., Hoye, Mariah L., Silberberg, Shai D.·2023·eneuro
DOI 10.1523/eneuro.0247-23.2023PMC PMC10411680
Open this paper in CoChat — free accountOpen PDF

Abstract

Al estudiar las monografías de la especialidad que se desee cultivar, debemos fijarnos sobre todo en dos cosas: en los métodos de investigación de que el autor se ha servido en sus pesquisas, y en los problemas que han quedado pendientes de solución."["When studying monographs from the specialty one wishes to cultivate, we should focus on two things above all: on the investigation methods that the author has used in their research, and on the problems that are still pending a solution."]

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1523/eneuro.0247-23.2023

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Numerous reports that decry failures to replicate findings in behavioral and clinical neuroscience have triggered increased emphasis on the importance of replication.

    “Numerous reports that decry failures to replicate findings in behavioral and clinical neuroscience have triggered increased emphasis on the importance of replication [1-5].”

    From A feature and not a bug: Replication failures can lead to new biology (2026) · cites this paper as [2]

    The Scientific Ecosystem > Publishing norms discusses inadequate reporting in neurologic research, and NINDS Efforts > Publications describes responses focused on reporting guidelines. The available passages do not say that numerous reports of failed replication in behavioral and clinical neuroscience prompted greater emphasis on replication itself (searched: failed replication, reproducibility, and renewed attention to replication). Page numbers were not available in our copy. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 52

  1. Begley CG, Buchan AM, Dirnagl U (2015) Robust research: institutions must do their part for reproducibility. Nature 525: 25– 27. 10.1038/525025a 26333454 · DOI
  2. Berg JM, et al. (2016) Preprints for the life sciences. Science 352: 899– 901. 10.1126/science.aaf9133 27199406 · DOI
  3. Bespalov A, et al. (2021) Introduction to the EQIPD quality system. Elife 10: e63294. 10.7554/eLife.63294 34028353 PMC8184207 · DOI · Paper page
  4. Bosch G, Casadevall A (2017) Graduate biomedical science education needs a new philosophy. mBio 8: e01539-17. 10.1128/mBio.01539-17 29259084 PMC5736909 · DOI
  5. Brumback RA (2008) Worshiping false idols: the impact factor dilemma. J Child Neurol 23: 365– 367. 10.1177/0883073808315170 18401031 · DOI
  6. Chambers CD (2013) Registered reports: a new publishing initiative at cortex. Cortex 49: 609– 610. 10.1016/j.cortex.2012.12.016 23347556 · DOI
  7. Chambers CD, Tzavella L (2022) The past, present and future of registered reports. Nat Hum Behav 6: 29– 42. 10.1038/s41562-021-01193-7 34782730 · DOI · Paper page
  8. Collins FS, Tabak LA (2014) Policy: NIH plans to enhance reproducibility. Nature 505: 612– 613. 10.1038/505612a 24482835 PMC4058759 · DOI · Paper page
  9. Cordero RJB, de León-Rodriguez CM, Alvarado-Torres JK, Rodriguez AR, Casadevall A (2016) Life science’s average publishable unit (APU) has increased over the past two decades. PLoS One 11: e0156983. 10.1371/journal.pone.0156983 27310929 PMC4911092 · DOI
  10. de Solla Price DJ (1963) Little science, big science. New York: Columbia University Press.
  11. de Solla Price DJ (1975) Science since Babylon. New Haven and London: Yale University Press.
  12. Dirnagl U, Przesdzing I, Kurreck C, Major S (2016) A laboratory critical incident and error reporting system for experimental biomedicine. PLoS Biol 14: e2000705. 10.1371/journal.pbio.2000705 27906976 PMC5131907 · DOI
  13. Dougherty MR, Horne Z (2022) Citation counts and journal impact factors do not capture some indicators of research quality in the behavioural and brain sciences. R Soc Open Sci 9: 220334. 10.1098/rsos.220334 35991336 PMC9382220 · DOI
  14. Falagas ME, Alexiou VG (2008) The top-ten in journal impact factor manipulation. Arch Immunol Ther Exp (Warsz) 56: 223– 226. 10.1007/s00005-008-0024-5 18661263 · DOI
  15. Forscher BK (1963) Chaos in the Brickyard. Science 142: 339. 10.1126/science.142.3590.339 17799464 · DOI
  16. Fyfe A, McDougall-Waters J, Moxham N (2015) 350 years of scientific periodicals. Notes Rec R Soc Lond 69: 227– 239. 10.1098/rsnr.2015.0036 26495575 PMC4528406 · DOI
  17. Garfield E (1963) Citation indexes in sociological and historical research. Am Doc 14: 289– 291. 10.1002/asi.5090140405 · DOI
  18. Garfield E, Sher IH (1963) New factors in the evaluation of scientific literature through citation indexing. Am Doc 14: 195– 201. 10.1002/asi.5090140304 · DOI
  19. Gross PL, Gross EM (1927) College libraries and chemical education. Science 66: 385– 389. 10.1126/science.66.1713.385 17782476 · DOI · Paper page
  20. Hicks D, Wouters P, Waltman L, de Rijcke S, Rafols I (2015) The Leiden manifesto for research metrics. Nature 520: 429– 431. 10.1038/520429a 25903611 · DOI · Paper page
  21. Hoskin K (1996) The ‘awful idea of accountability’: inscribing people into the measurement of objects. In: Accountability: power, ethos and the technologies of managing (Mouritsen RMaJ, ed), pp 265– 282. London: International Thomson Business Press.
  22. Kelner KL (2013) Playing our part. Sci Transl Med 5: 190ed197. 10.1126/scitranslmed.3006661 · DOI
  23. Koroshetz WJ, et al. (2020) Framework for advancing rigorous research. Elife 9: e55915. 10.7554/eLife.55915 32127131 PMC7056268 · DOI · Paper page
  24. Kyzas PA, Denaxa-Kyza D, Ioannidis JPA (2007) Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer 43: 2559– 2579. 10.1016/j.ejca.2007.08.030 17981458 · DOI
  25. Laakso M, Welling P, Bukvova H, Nyman L, Björk BC, Hedlund T (2011) The development of open access journal publishing from 1993 to 2009. PLoS One 6: e20961. 10.1371/journal.pone.0020961 21695139 PMC3113847 · DOI · Paper page
  26. Larivière V, Sugimoto CR (2019) The J impact factor: a brief history, critique, and discussion of adverse effects. In: Springer handbook of science and technology indicators (Glänzel W, Moed HF, Schmoch U, and Thelwall M, eds), pp 3– 24. Cham: Springer International Publishing.
  27. Liu M, Choy V, Clarke P, Barnett A, Blakely T, Pomeroy L (2020) The acceptability of using a lottery to allocate research funding: a survey of applicants. Res Integr Peer Rev 5: 3. 10.1186/s41073-019-0089-z 32025338 PMC6996170 · DOI
  28. Macleod MR, McLean AL, Kyriakopoulou A, Serghiou S, de Wilde A, Sherratt N, Hirst T, Hemblade R, Bahor Z, Nunes-Fonseca C, Potluru A, Thomson A, Baginskaite J, Egan K, Vesterinen H, Currie GL, Churilov L, Howells DW, Sena ES (2015) Risk of bias in reports of in vivo research: a focus for improvement. PLoS Biol 13: e1002273. 10.1371/journal.pbio.1002273 26460723 PMC4603955 · DOI · Paper page
  29. Marcus E; whole Cell team (2016) A STAR is born. Cell 166: 1059– 1060. 10.1016/j.cell.2016.08.021 27565332 · DOI
  30. McGuigan GS, Russell RD (2008) The business of academic publishing: a strategic analysis of the academic journal publishing industry and its impact on the future of scholarly publishing. E-JASL 9(3).
  31. Mckiernan EC, Schimanski LA, Nieves CM, Matthias L, Niles MT, Alperin JP (2019) Use of the journal impact factor in academic review, promotion, and tenure evaluations. Elife 8: e47338. 10.7554/eLife.47338 31364991 PMC6668985 · DOI
  32. McNutt M (2014a) Journals unite for reproducibility. Science 346: 679. 10.1126/science.aaa1724 25383411 · DOI
  33. Menke J, Roelandse M, Ozyurt B, Martone M, Bandrowski A (2020) The rigor and transparency index quality metric for assessing biological and medical science methods. iScience 23: 101698. 10.1016/j.isci.2020.101698 33196023 PMC7644557 · DOI · Paper page
  34. Menke J, Eckmann P, Ozyurt IB, Roelandse M, Anderson N, Grethe J, Gamst A, Bandrowski A (2022) Establishing institutional scores with the rigor and transparency index: large-scale analysis of scientific reporting quality. J Med Internet Res 24: e37324. 10.2196/37324 35759334 PMC9274430 · DOI · Paper page
  35. Moher D, Naudet F, Cristea IA, Miedema F, Ioannidis JPA, Goodman SN (2018) Assessing scientists for hiring, promotion, and tenure. PLoS Biol 16: e2004089. 10.1371/journal.pbio.2004089 29596415 PMC5892914 · DOI · Paper page
  36. Moher D, Bouter L, Kleinert S, Glasziou P, Sham MH, Barbour V, Coriat AM, Foeger N, Dirnagl U (2020) The Hong Kong principles for assessing researchers: fostering research integrity. PLoS Biol 18: e3000737. 10.1371/journal.pbio.3000737 32673304 PMC7365391 · DOI
  37. Nature Neuroscience (2013) Raising standards. Nat Neurosci 16: 517. 23619751
  38. Orben A (2019) A journal club to fix science. Nature 573: 465. 10.1038/d41586-019-02842-8 31551562 · DOI
  39. Perrin S (2014) Preclinical research: make mouse studies work. Nature 507: 423– 425. 10.1038/507423a 24678540 · DOI · Paper page
  40. Reich ES (2013) Science publishing: the golden club. Nature 502: 291– 293. 10.1038/502291a 24132273 · DOI
  41. Reif F (1961) The competitive world of the pure scientist: the quest for prestige can cause conflict between the goals of science and the goals of the scientist. Science 134: 1957– 1962. 10.1126/science.134.3494.1957 17744407 · DOI
  42. Rice DB, Raffoul H, Ioannidis JPA, Moher D (2020) Academic criteria for promotion and tenure in biomedical sciences faculties: cross sectional analysis of international sample of universities. BMJ 369: m2081. 10.1136/bmj.m2081 32586791 PMC7315647 · DOI · Paper page
  43. Rosenthal R (1979) The file drawer problem and tolerance for null results. Psychol Bull 86: 638– 641. 10.1037/0033-2909.86.3.638 · DOI · Paper page
  44. Rowlands I, Nicholas D (2005) Scholarly communication in the digital environment: The 2005 survey of journal author behaviour and attitudes. Aslib Proceedings 57: 481– 497.
  45. Schmid SL (2017) Five years post-DORA: promoting best practices for research assessment. Mol Biol Cell 28: 2941– 2944. 10.1091/mbc.E17-08-0534 29084913 PMC5662254 · DOI
  46. Sena ES, Currie GL, McCann SK, Macleod MR, Howells DW (2014) Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically. J Cereb Blood Flow Metab 34: 737– 742. 10.1038/jcbfm.2014.28 24549183 PMC4013765 · DOI · Paper page
  47. Silberberg SD, Crawford DC, Finkelstein R, Koroshetz WJ, Blank RD, Freeze HH, Garrison HH, Seger YR (2017) Shake up conferences. Nature 548: 153– 154. 10.1038/548153a 28796229 · DOI · Paper page
  48. Spier R (2002) The history of the peer-review process. Trends Biotechnol 20: 357– 358. 10.1016/s0167-7799(02)01985-6 12127284 · DOI
  49. Stewart SLK, Pennington CR, da Silva GR, Ballou N, Butler J, Dienes Z, Jay C, Rossit S, Samara A; U. K. Reproducibility Network (UKRN) Local Network Leads (2022) Reforms to improve reproducibility and quality must be coordinated across the research ecosystem: the view from the UKRN Local Network Leads. BMC Res Notes 15: 58. 10.1186/s13104-022-05949-w 35168675 PMC8845353 · DOI
  50. Strech D, Weissgerber T, Dirnagl U; QUEST Group (2020) Improving the trustworthiness, usefulness, and ethics of biomedical research through an innovative and comprehensive institutional initiative. PLoS Biol 18: e3000576. 10.1371/journal.pbio.3000576 32045410 PMC7012388 · DOI
  51. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252– 260. 10.1056/NEJMsa065779 18199864 · DOI · Paper page
  52. Vale RD (2012) Evaluating how we evaluate. Mol Biol Cell 23: 3285– 3289. 10.1091/mbc.E12-06-0490 22936699 PMC3431935 · DOI

Cited by

Sorted by citations

Authors
Crawford, D. C., Hoye, M. L., & Silberberg, S. D.
Affiliations
National Institutes of Health (US)
Corresponding author
Crawford, Devon C.
ORCID
Venue
eneuro
Publisher
Society for Neuroscience
Published
2023-08
Volume
10
Issue
8
Pages
ENEURO.0247-23.2023
ISSN
2373-2822 (ISSN-L)
DOI
10.1523/eneuro.0247-23.2023
PMC
PMC10411680
OpenAlex
W4385655002
Cited by
25
References
10
Citations by year
2026: 5 · 2025: 10 · 2024: 10
FWCI
22.55
Type
journal article
Language
en
Open access
Licence
Locations
  • eNeuro (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords
research quality, research methodology, research challenges

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review