Reforming Science: Methodological and Cultural Reforms
Open accessThe DOI matches its Crossref record; 44 of the 46 references it cites were found in a registry or an index.
Abstract
Contemporary science has brought about technological advances and an unprecedented understanding of the natural world. However, there are signs of dysfunction in the scientific community as well as threats from diverse antiscience and political forces. Incentives in the current system place scientists under tremendous stress, discourage cooperation, encourage poor scientific practices, and deter new talent from entering the field. It is time for a discussion of how the scientific enterprise can be reformed to become more effective and robust. Serious reform will require more consistent methodological rigor and a transformation of the current hypercompetitive scientific culture.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The impact of plagiarism and duplicate publication on science is therefore limited; although these practices are to be discouraged, the present study suggests that the reduction of fraud and error should be a higher priority (32).”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [32]
In the citing paper
“Theoretical analyses point to a system of incentives that prioritizes innovation over verification, leading to infrequent efforts to replicate findings and to behaviors that could reduce the replicability of published findings such as selective reporting, presenting exploratory findings as confirmatory tests, and failures of documentation, transparency, and sharing”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Casadevall and Fang, 2012
In the citing paper
“there is accumulating evidence of the deleterious impacts of low power and small sample sizes, ignoring null results, failures of transparency of methodology, failing to publish all experimental data collected, and questionable research practices such as p-hacking on the inflation of false positives in the published literature”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Casadevall and Fang, 2012
In the citing paper
“The rise in the rate of retractions raises concern about the health of the scientific enterprise itself (32).”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [32]
In the citing paper
“The modest correlation between impact factor and time-toretraction argues against an explanation based on increased scrutiny alone, but the higher proportion of fraud in highly prestigious journals is consistent with the suggestion that the benefits of publishing in such venues are powerful incentives for fraud (4,6,32).”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [32]
Citations
Cites
4 retracted
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
RetractedLancet, T. E. O. T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 465
RetractedRetracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405
Self-citationRetractedAdditive Genetic Breeding Values Correlate with the Load of Partially Deleterious Mutations
Tomkins, J. L., Penrose, M. A., Greeff, J. C., & LeBas, N. R. (2010) · Science · doi:10.1126/science.1188013 · Cited by 29
RetractedWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The Matthew Effect in Science: The reward and communication systems of science are considered.
Merton, R. K. (1968) · Science · doi:10.1126/science.159.3810.56 · Cited by 6,720
A surgical safety checklist to reduce morbidity and mortality in a global population
Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A.-H. S., Dellinger, E. P., Herbosa, T., Joseph, S., Kibatala, P. L., Lapitan, M. C. M., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0810119 · Cited by 5,688 · cited 1×
“There is conclusive evidence that the use of checklists can reduce errors in human activities ranging from aviation to surgery (17).”
Background(1968) · PubMed · Cited by 4,803
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 2×
“However, competition in excess can be a compelling incentive for scientific misconduct ranging from egregious fraud to more subtle transgressions, such as selective reporting of results (12).”
BackgroundHarzing, A.-W. K. (2007) · Cited by 969
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805 · cited 1×
“These factors, as well as the difficulty publishing negative results (publication bias), serve to undermine the reliability of the scientific literature (13, 18).”
BackgroundThe Double Helix: A Personal Account of the Discovery of the Structure of DNA
Watson, J. D., & Devons, S. (1968) · Physics Today · doi:10.1063/1.3035117 · Cited by 638
Climategate, Public Opinion, and the Loss of Trust
Leiserowitz, A. A., Maibach, E. W., Roser-Renouf, C., Smith, N., & Dawson, E. (2013) · American Behavioral Scientist · doi:10.1177/0002764212458272 · Cited by 406 · cited 1×
“Nevertheless, this trust is precious and must not be taken for granted, as illustrated by the rapid loss of trust in climate science and scientists that has followed the release of emails and computer files in the incident known as “climategate” (25).”
BackgroundRetractions in the scientific literature: is the incidence of research fraud increasing?
Steen, R. G. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2010.040923 · Cited by 405 · cited 1×
“Although concerns about the relationship between pressure to publish and research fraud are not new (38), the frequency of retracted papers is increasing (32).”
BackgroundArticle 50 million: an estimate of the number of scholarly articles in existence
JINHA, A. E. (2010) · Learned Publishing · doi:10.1087/20100308 · Cited by 404 · cited 1×
“Jinha has estimated that 50 million scholarly articles have been published since 1665, when the first modern journal appeared, with more than half of these published during just the last 25 years (21).”
BackgroundThe Role of the Priority Rule in Science
Strevens, M., & Journal of Philosophy, I. (2003) · Journal of Philosophy · doi:10.5840/jphil2003100224 · Cited by 345
Retractions in the scientific literature: do authors deliberately commit research fraud?
Steen, R. G. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2010.038125 · Cited by 217
Casadevall, A., & Pirofski, L. (2006) · Advances in immunology · doi:10.1016/s0065-2776(06)91001-3 · Cited by 160 · cited 1×
“For example, the conclusion that humoral immunity had no role in protection against many intracellular pathogens was based on an inability to demonstrate the efficacy of antibody with the methodology used (7).”
Self-citationMortality and immortality: The Nobel Prize as an experiment into the effect of status upon longevity
Rablen, M. D., & Oswald, A. J. (2008) · Journal of Health Economics · doi:10.1016/j.jhealeco.2008.06.001 · Cited by 151
Concentration of the Most-Cited Papers in the Scientific Literature: Analysis of Journal Ecosystems
Ioannidis, J. P. A. (2006) · PLoS ONE · doi:10.1371/journal.pone.0000005 · Cited by 122 · cited 1×
“The result is an unwieldy scientific literature in which the most highly cited papers are concentrated in a small number of journals (19) while a large proportion of papers are cited infrequently or not at all (2).”
BackgroundWinner Takes all: An Evaluation of Adventure-Based Experiential Training
Ibbetson, A., & Newell, S. (1996) · Management Learning · doi:10.1177/1350507696272001 · Cited by 107
Wilson, D. S. (2007) · Cited by 97
A historical reflection on the discovery of human retroviruses
Vahlne, A. G. (2009) · Retrovirology · doi:10.1186/1742-4690-6-40 · Cited by 80 · cited 1×
“A more recent example is provided by the dispute over priority in the discovery of HIV (34).”
BackgroundKohut, A. (2009) · Cited by 80
Ioannidis, J. P. A. (2011) · Scientific American · doi:10.1038/scientificamerican0611-16 · Cited by 67
Reforming Science: Structural Reforms
Fang, F. C., & Casadevall, A. (2012) · Infection and Immunity · doi:10.1128/iai.06184-11 · Cited by 60 · cited 1×
“This article will focus on methodological and cultural reforms, and the accompanying paper (15) will address structural problems.”
BackgroundSelf-citationWe must stop the avalanche of low-quality research
Bauerlein, M. (2010) · The chronicle of higher education · Cited by 60
Korpela, K. M. (2010) · Current Medical Research and Opinion · doi:10.1185/03007991003603804 · Cited by 58 · cited 1×
“Thus, even retracted papers continue to be cited and used as evidence to buttress false claims (23).”
BackgroundPressure to Publish and Fraud in Science
WOOLF, P. K. (1986) · Annals of Internal Medicine · doi:10.7326/0003-4819-104-2-254 · Cited by 57 · cited 1×
“Although concerns about the relationship between pressure to publish and research fraud are not new (38), the frequency of retracted papers is increasing (32).”
BackgroundIoannidis, J. P. A. (2011) · PubMed · Cited by 56
Uhlich, G. A., Gunther, N. W., IV, Bayles, D. O., & Mosier, D. A. (2009) · Infection and Immunity · doi:10.1128/iai.00949-08 · Cited by 56
Climategate, Public Opinion, and the Loss of TrustPreprint
Leiserowitz, A., Maibach, E. W., Roser‐Renouf, C., et al. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1633932 · Cited by 53
Web of deception: misinformation on the internet
(2002) · The Journal of Academic Librarianship · doi:10.1016/s0099-1333(02)00360-9 · Cited by 50
Casadevall, A., & Fang, F. C. (2008) · Infection and Immunity · doi:10.1128/iai.00743-08 · Cited by 47
Self-citationEinstein, Heisenberg, Kant: methodological distinction and conditions of possibilities
Monod, E. (2004) · Information and Organization · doi:10.1016/j.infoandorg.2003.12.001 · Cited by 26 · cited 1×
“Einstein credited the philosopher Immanuel Kant with inspiration that led to the theory of relativity, and Einstein’s scientific contributions have in turn influenced philosophy (29).”
BackgroundHow to Build a Motivated Research Group
Alon, U. S. (2010) · Molecular Cell · doi:10.1016/j.molcel.2010.01.011 · Cited by 25 · cited 1×
“The systems biologist Uri Alon has written a thought-provoking essay on how to build a motivated research group (1).”
BackgroundCasadevall, A., & Fang, F. C. (2009) · Infection and Immunity · doi:10.1128/iai.00623-09 · Cited by 24 · cited 1×
“A stronger foundation in epistemology, supplemented by an awareness of philosophical issues involving language, might also avoid some of the problems created in science from the misuse of such terms as “descriptive” (5) and “mechanistic” (6) in the categorization of projects and papers.”
BackgroundSelf-citationWeb of Deception: Misinformation on the Internet
(2003) · Online Information Review · doi:10.1108/14684520310481517 · Cited by 21
Patrimony and the Evolution of Risk-Taking
Stern, M. D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0011656 · Cited by 13 · cited 1×
“Winner-take-all strategies have been proposed to be male evolutionary strategies that disproportionally reward risk taking in the production of offspring (33).”
BackgroundPostma, E. (2011) · Science · doi:10.1126/science.1200996 · Cited by 6 · cited 1×
“For example, a recent paper in Science was retracted because the authors assumed that two correlated variables were independent while these variables were in fact additive and dependent (30).”
BackgroundWhy did the Lancet take so long?
Greenhalgh, T. (2010) · BMJ · doi:10.1136/bmj.c644 · Cited by 6
Cole, M. W. (2009) · ACADEME University of Bohol Graduate School and Professional Studies · Cited by 4
McHale, J. E. (1962) · Journal of the American Oil Chemists Society · doi:10.1007/bf02633361 · Cited by 2
Research works printed but not linked: 3
- Carlyle T. 1858. The collected works of Thomas Carlyle. Chapman and Hall, London, United Kingdom.
- Lewis S. 1925. Arrowsmith. Harcourt Brace and Co., New York, NY.
- Machiavelli N. 1532. The Prince. Antonio Blado d'Asola, Florence, Italy.
Cited by
Misconduct accounts for the majority of retracted scientific publications
Fang, F. C., Steen, R. G., & Casadevall, A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1212247109 · Cited by 1,232
“The rise in the rate of retractions raises concern about the health of the scientific enterprise itself (32).”
BackgroundInfluentialSelf-citationReproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035
“It is probably relevant to note the evidence that investigators who are assured of more consistent, stable laboratory funding are less likely to succumb to the pressures of fraud.(53,54) It seems reasonable to extrapolate and suggest that may also be the case for investigators who enjoy similar guaranteed research fun…”
Deep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441
“…2012; Kriegeskorte et al., 2012; Lee, 2012; Nosek and Bar-Anan, 2012; Pöschl, 2012; Priem and Hemminger, 2012; Sandewall, 2012; Walther and Van den Bosch, 2012; Wicherts et al., 2012; Yarkoni, 2012), but the need for an alternative system is clearly pressing (Casadevall and Fang, 2012).”
Why has the number of scientific retractions increased?
Steen, R. G., Casadevall, A., & Fang, F. C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068397 · Cited by 415
“Better understanding of the underlying causes for retractions can potentially inform efforts to change the culture of science [41] and to stem a loss of trust in science among the lay public [42,43].”
BackgroundSelf-citationInvestigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 462
Metadata
- Authors
- Casadevall, A., & Fang, F. C.
- Affiliations
- Albert Einstein College of Medicine (US)
- University of Washington (US)
- Venue
- Infection and Immunity
- Publisher
- American Society for Microbiology
- Published
- 2012-03
- Volume
- 80
- Issue
- 3
- Pages
- 891-896
- ISSN
- 0019-9567 (ISSN-L)
- 1098-5522
- PMC
- PMC3294645
- OpenAlex
- W2148405837
- Cited by
- 156
- References
- 46
- Citations by year
- 2025: 6 · 2024: 8 · 2023: 5 · 2022: 9 · 2021: 11 · 2020: 10 · 2019: 24 · 2018: 16 · 2017: 10 · 2016: 15 · 2015: 13 · 2014: 9 · 2013: 13 · 2012: 6
- FWCI
- 22.13
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- journals.asm.org/non-commercial-tdm-license (tdm)
- Locations
- Infection and Immunity (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Research, Science, and Academia
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- scientific reform, anti-science, political interference in science, research culture, methodological quality, scientific cooperation, scientific integrity
- MeSH terms
- Competitive Behavior, Economic Competition, Humans, Science, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.