Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
The growing competition and "publish or perish" culture in academia might conflict with the objectivity and integrity of research, because it forces scientists to produce "publishable" results at all costs. Papers are less likely to be published and to be cited if they report "negative" results (results that fail to support the tested hypothesis). Therefore, if publication pressures increase scientific bias, the frequency of "positive" results in the literature should be higher in the more competitive and "productive" academic environments. This study verified this hypothesis by measuring the frequency of positive results in a large random sample of papers with a corresponding author based in the US. Across all disciplines, papers were more likely to support a tested hypothesis if their corresponding authors were working in states that, according to NSF data, produced more academic papers per capita. The size of this effect increased when controlling for state's per capita R&D expenditure and for study characteristics that previous research showed to correlate with the frequency of positive results, including discipline and methodology. Although the confounding effect of institutions' prestige could not be excluded (researchers in the more productive universities could be the most clever and successful in their experiments), these results support the hypothesis that competitive academic environments increase not only scientists' productivity but also their bias. The same phenomenon might be observed in other countries where academic competition and pressures to publish are high.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Earlier studies that compared the scientific productivity of countries offered some support for such a link (38, 39).”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The academic system and peer-review process tolerates and perhaps even inadvertently encourages such conduct 5 .”
From Raise standards for preclinical cancer research (2012) · cites this paper as [5]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pharmacological interventions for somatoform disorders in adults
Kleinstäuber, M., Witthöft, M., Steffanowski, A., van Marwijk, H., Hiller, W., & Lambert, M. J. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010628.pub2 · Cited by 11,335
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
The cochrane database of systematic reviews
Geddes, J. (1996) · Journal of Psychosomatic Research · doi:10.1016/s0022-3999(96)90063-x · Cited by 5,989
Science and Engineering Indicators
Washington, E., & Warren, M. E. (2004) · Cited by 2,822
Journal of International Business Studies
(1975) · Maritime Studies and Management · doi:10.1080/03088837500000054 · Cited by 2,436
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147 · cited 1×
“Quantitative studies have repeatedly shown that financial interests can influence the outcome of biomedical research [27,28] but they appear to have neglected the much more widespread conflict of interest created by scientists’ need to publish.”
BackgroundHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1×
“fabrication and falsification are probably rare, but other questionable research practices might be relatively common [26].”
BackgroundSelf-citationChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A Systematic Review
Bekelman, J. E., Li, Y., & Gross, C. P. (2003) · JAMA · doi:10.1001/jama.289.4.454 · Cited by 1,816
Wacholder, S., Chanock, S., Garcia-Closas, M., El ghormli, L., & Rothman, N. (2004) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djh075 · Cited by 1,754
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Martinson, B. C., Anderson, M. S., & de Vries, R. (2005) · Nature · doi:10.1038/435737a · Cited by 1,096
The Academic Ranking of World Universities
Liu, N. C., & Cheng, Y. (2005) · Higher Education in Europe · doi:10.1080/03797720500260116 · Cited by 1,070
Beyond significance testing: Reforming data analysis methods in behavioral research.
Kline, R. B. (2004) · doi:10.1037/10693-000 · Cited by 1,034
Full publication of results initially presented in abstracts.
Scherer, R. W., Langenberg, P., & von Elm, E. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000005.pub3 · Cited by 997
Information and Software Technology
Achimugu, P., & Selamat, A. (2014) · Cited by 786
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734
Self-citationThe Empire of chance: how probability changed science and everyday life
(1989) · Choice Reviews Online · doi:10.5860/choice.27-2084 · Cited by 597
Publication and Related Biases
Song, F., Eastwood, A., Gilbody, S., Duley, L., & Sutton, A. (2001) · The Advanced Handbook of Methods in Evidence Based Healthcare · doi:10.4135/9781848608344.n21 · Cited by 455
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453
(1952) · Physics Today · doi:10.1063/1.3067448 · Cited by 445
The Perverse Effects of Competition on Scientists’ Work and Relationships
Anderson, M. S., Ronning, E. A., De Vries, R., & Martinson, B. C. (2007) · Science and Engineering Ethics · doi:10.1007/s11948-007-9042-5 · Cited by 384
What determines the citation frequency of ecological papers?
LEIMU, R., & KORICHEVA, J. (2005) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2004.10.010 · Cited by 367
The Scientific Life: A Moral History of a Late Modern Vocation
Labinger, J. A. (2009) · Physics Today · doi:10.1063/1.3099581 · Cited by 316
A systematic review of statistical power in software engineering experiments
Dybå, T., Kampenes, V. B., & Sjøberg, D. I. K. (2006) · Information and Software Technology · doi:10.1016/j.infsof.2005.08.009 · Cited by 289
A survey of the statistical power of research in behavioral ecology and animal behavior
Jennions, M. D. (2003) · Behavioral Ecology · doi:10.1093/beheco/14.3.438 · Cited by 278 · cited 1×
“But negative results are virtually inevitable, unless all the hypotheses tested were true, experiments were designed and conducted perfectly, and the statistical power available were always 100% – which it rarely is, and is usually much lower [42,43,44,45,46].”
BackgroundNormal Misbehavior: Scientists Talk about the Ethics of Research
De Vries, R., Anderson, M. S., & Martinson, B. C. (2006) · Journal of Empirical Research on Human Research Ethics · doi:10.1525/jer.2006.1.1.43 · Cited by 254
Gerber, A. S., & Malhotra, N. (2008) · Sociological Methods & Research · doi:10.1177/0049124108318973 · Cited by 246
Time to publication for results of clinical trials
Hopewell, S., Clarke, M. J., Stewart, L., & Tierney, J. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000011.pub2 · Cited by 243
Reflections on the Decline of Science in England, and on Some of its Causes
Babbage, C. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9781139381048 · Cited by 237
Almost all articles on cancer prognostic markers report statistically significant results
Kyzas, P. A., Denaxa-Kyza, D., & Ioannidis, J. P. A. (2007) · European Journal of Cancer · doi:10.1016/j.ejca.2007.08.030 · Cited by 231
Publication bias in ecology and evolution: an empirical assessment using the ‘trim and fill’ method
JENNIONS, M. D., & MØLLER, A. P. (2002) · Biological Reviews · doi:10.1017/s1464793101005875 · Cited by 230
Normative Dissonance in Science: Results from a National Survey of U.S. Scientists
Anderson, M. S., Martinson, B. C., & De Vries, R. (2007) · Journal of Empirical Research on Human Research Ethics · doi:10.1525/jer.2007.2.4.3 · Cited by 208
The "File Drawer Problem" of Non-Significant Results: Does It Apply to Biological Research?
Csada, R. D., James, P. C., & Espie, R. H. M. (1996) · Oikos · doi:10.2307/3546355 · Cited by 201
Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research
KLEIN, D. F. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.3.643-a · Cited by 167
Sterling, T. D., Rosenbaum, W. L., & Jj, W. (1995) · The American Statistician · doi:10.2307/2684823 · Cited by 158
International ranking systems for universities and institutions: a critical appraisal
Ioannidis, J. P. A., Patsopoulos, N. A., Kavvoura, F. K., Tatsioni, A., Εvangelou, Ε., Kouri, I., Contopoulos‐Ioannidis, D. G., & Liberopoulos, G. (2007) · BMC Medicine · doi:10.1186/1741-7015-5-30 · Cited by 152
Causal Factors Implicated in Research Misconduct: Evidence from ORI Case Files
Davis, M. S., Riske‐Morris, M., & Díaz, S. (2007) · Science and Engineering Ethics · doi:10.1007/s11948-007-9045-2 · Cited by 151
The Empire of Chance: How Probability Changed Science and Everyday Life.
Guttorp, P. M., Gigerenzer, G., Swijtink, Z. G., Porter, T. M., Daston, L., Beatty, J., & Krüger, L. (1990) · Journal of the American Statistical Association · doi:10.2307/2289806 · Cited by 151
Giles, J. (2004) · Nature · doi:10.1038/news040301-9 · Cited by 127
JOURNAL QUALITY, EFFECT SIZE, AND PUBLICATION BIAS IN META-ANALYSIS
Murtaugh, P. A. (2002) · Ecology · doi:10.1890/0012-9658(2002)083[1162:jqesap]2.0.co;2 · Cited by 126
Disciplinary and Departmental Effects on Observations of Faculty and Graduate Student Misconduct
Anderson, M. S., Louis, K. S., & Earle, J. (1994) · The Journal of Higher Education · doi:10.1080/00221546.1994.11778504 · Cited by 123
Country development and manuscript selection bias: a review of published studies
Nooraie, R. Y., Shakiba, B., & Mortaz-Hejri, S. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-37 · Cited by 115
Irreproducibility of the results of the Shanghai academic ranking of world universities
Florian, R. V. (2007) · Scientometrics · doi:10.1007/s11192-007-1712-1 · Cited by 106
Citation bias of hepato-biliary randomized clinical trials
Gluud, L. L., & Gluud, C. N. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00513-3 · Cited by 84
A meta-analysis of the export growth hypothesis
Mookerjee, R. (2006) · Economics Letters · doi:10.1016/j.econlet.2005.12.020 · Cited by 74
Experimenter effects in behavioral research, Enlarged ed.
Rosenthal, R. (1976) · Cited by 67
Disciplinary and Departmental Effects on Observations of Faculty and Graduate Student Misconduct
Anderson, M. S., Louis, K. S., & Earle, J. (1994) · The Journal of Higher Education · doi:10.2307/2943971 · Cited by 55
Seeing What We Want to See: Confirmation Bias in Animal Behavior Research
Marsh, D. M., & Hanlon, T. J. (2007) · Ethology · doi:10.1111/j.1439-0310.2007.01406.x · Cited by 53
The ‘power’ of international business research
Brock, J. K. (2003) · Journal of International Business Studies · doi:10.1057/palgrave.jibs.8400006 · Cited by 49
Statistical power of articles published in three health-psychology related journals.
Maddock, J. E., & Rossi, J. S. (2001) · Health Psychology · doi:10.1037/0278-6133.20.1.76 · Cited by 46
Inadequate Statistical Power of Negative Clinical Trials in Urological Literature
Breau, R. H., Carnat, T. A., & Gaboury, I. (2006) · The Journal of Urology · doi:10.1016/s0022-5347(06)00505-2 · Cited by 45
Academic Misconduct and Values: The Department’s Influence
Louis, K. S., Anderson, M. S., & Rosenberg, L. (1995) · Review of higher education/The review of higher education · doi:10.1353/rhe.1995.0007 · Cited by 44
Etter, J., & Stapleton, J. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.09.015 · Cited by 43
Fanelli, D. (2009) · PsycEXTRA Dataset · doi:10.1037/e521122012-010 · Cited by 39
Self-citationJournal Quality, Effect Size, and Publication Bias in Meta-Analysis
Murtaugh, P. A. (2002) · Ecology · doi:10.2307/3071921 · Cited by 26
Shakiba, B., Salmasian, H., Nooraie, R. Y., & Rohanizadegan, M. (2008) · PubMed · Cited by 14
Giles, J. (2007) · Nature · doi:10.1038/445242a · Cited by 14
Causal Factors Implicated in Research Misconduct: Evidence from ORI Case Files
Davis, M. S., Riske‐Morris, M., & Díaz, S. (2008) · Science and Engineering Ethics · doi:10.1007/s11948-008-9070-9 · Cited by 11
Research works printed but not linked: 2
- [20]Rosenthal R 1976 Experimenter effects in behavioural research. Enlarged edition New York Irvington Publishers, Inc
- [22]Mahoney MJ 1979 Psychology of the scientist: An evaluative review. Social Studies of Science 1979 3
Cited by
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248
Self-citationA call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22784 · Cited by 1,000
Metadata
- Authors
- Fanelli, D.
- Affiliations
- University of Edinburgh (GB)
- Corresponding author
- Fanelli, Daniele
- ORCID
- Fanelli, Daniele
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2010-04-21
- Volume
- 5
- Issue
- 4
- Pages
- e10271
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2858206
- OpenAlex
- W1966297423
- Cited by
- 805
- References
- 60
- Citations by year
- 2026: 34 · 2025: 47 · 2024: 37 · 2023: 61 · 2022: 60 · 2021: 64 · 2020: 62 · 2019: 83 · 2018: 58 · 2017: 52 · 2016: 64 · 2015: 40 · 2014: 43 · 2013: 42 · 2012: 36
- FWCI
- 82.69
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- DigitalGeorgetown (Georgetown University Library) (submittedVersion)
- Funders
- European Commission (221441)
- Economic and Social Research Council (ES/F028180/1)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publish or perish, publication pressure, research bias, positive results, negative results, academic competition, research productivity, R&D investment, research integrity, hypothesis testing
- MeSH terms
- Ethics, Professional, Publishing, United States, Bias, Conflict of Interest, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.