Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
The growing competition and "publish or perish" culture in academia might conflict with the objectivity and integrity of research, because it forces scientists to produce "publishable" results at all costs. Papers are less likely to be published and to be cited if they report "negative" results (results that fail to support the tested hypothesis). Therefore, if publication pressures increase scientific bias, the frequency of "positive" results in the literature should be higher in the more competitive and "productive" academic environments. This study verified this hypothesis by measuring the frequency of positive results in a large random sample of papers with a corresponding author based in the US. Across all disciplines, papers were more likely to support a tested hypothesis if their corresponding authors were working in states that, according to NSF data, produced more academic papers per capita. The size of this effect increased when controlling for state's per capita R&D expenditure and for study characteristics that previous research showed to correlate with the frequency of positive results, including discipline and methodology. Although the confounding effect of institutions' prestige could not be excluded (researchers in the more productive universities could be the most clever and successful in their experiments), these results support the hypothesis that competitive academic environments increase not only scientists' productivity but also their bias. The same phenomenon might be observed in other countries where academic competition and pressures to publish are high.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Earlier studies that compared the scientific productivity of countries offered some support for such a link (38, 39).”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [39]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The academic system and peer-review process tolerates and perhaps even inadvertently encourages such conduct 5 .”
From Raise standards for preclinical cancer research (2012) · cites this paper as [5]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pharmacological interventions for somatoform disorders in adults
Kleinstäuber, M., Witthöft, M., Steffanowski, A., van Marwijk, H., Hiller, W., & Lambert, M. J. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010628.pub2 · Cited by 11,335
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
The cochrane database of systematic reviews
Geddes, J. (1996) · Journal of Psychosomatic Research · doi:10.1016/s0022-3999(96)90063-x · Cited by 5,989
Science and Engineering Indicators
Washington, E., & Warren, M. E. (2004) · Cited by 2,822
Journal of International Business Studies
(1975) · Maritime Studies and Management · doi:10.1080/03088837500000054 · Cited by 2,436
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147 · cited 1×
“Quantitative studies have repeatedly shown that financial interests can influence the outcome of biomedical research [27,28] but they appear to have neglected the much more widespread conflict of interest created by scientists’ need to publish.”
BackgroundHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1×
“fabrication and falsification are probably rare, but other questionable research practices might be relatively common [26].”
BackgroundSelf-citationChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A Systematic Review
Bekelman, J. E., Li, Y., & Gross, C. P. (2003) · JAMA · doi:10.1001/jama.289.4.454 · Cited by 1,816
Research works printed but not linked: 2
- [20]Rosenthal R 1976 Experimenter effects in behavioural research. Enlarged edition New York Irvington Publishers, Inc
- [22]Mahoney MJ 1979 Psychology of the scientist: An evaluative review. Social Studies of Science 1979 3
Cited by
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248
Self-citationA call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22784 · Cited by 1,000
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1166120 · Cited by 708
Reducing implicit racial preferences: I. A comparative investigation of 17 interventions.
Lai, C. K., Marini, M., Lehr, S. A., Cerruti, C., Shin, J.-E. L., Joy-Gaba, J. A., Ho, A. K., Teachman, B. A., Wojcik, S. P., Koleva, S. P., et al. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0036260 · Cited by 679
Engagement in language learning: a systematic review of 20 years of research methods and definitions
Hiver, P., Al-Hoorie, A. H., Vitta, J. P., & Wu, J. (2024) · Language Teaching Research · doi:10.1177/13621688211001289 · Cited by 677
“Such comprehensiveness is in line with Norris and Ortega (2000), and the rationale for such comprehensiveness includes the mitigation of selection and publication biases where large and significant effects are favored to be published (Fanelli, 2010).”
BackgroundFerguson, C. J., & Heene, M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459059 · Cited by 603
Why Science Is Not Necessarily Self-Correcting
Ioannidis, J. P. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612464056 · Cited by 563
Do people who feel connected to nature do more to protect it? A meta-analysis
Mackay, C. M. L., & Schmitt, M. T. (2019) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2019.101323 · Cited by 538
Dealing with the positive publication bias: Why you should really publish your negative results
Mlinarić, A., Horvat, M., & Šupak Smolčić, V. (2017) · Biochemia Medica · doi:10.11613/bm.2017.030201 · Cited by 514
Deep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441
Intersectional HCI: Engaging Identity through Gender, Race, and Class
Schlesinger, A., Edwards, W. K., & Grinter, R. E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025766 · Cited by 449
What is open peer review? A systematic review
Ross-Hellauer, T. (2017) · F1000Research · doi:10.12688/f1000research.11369.2 · Cited by 449
Is science really facing a reproducibility crisis, and do we need it to?
Fanelli, D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708272114 · Cited by 416
Self-citationPre-registration in social psychology—A discussion and suggested template
van 't Veer, A. E., & Giner-Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 439
Star Wars: The Empirics Strike Back
Brodeur, A., Lé, M., Sangnier, M., & Zylberberg, Y. (2016) · American Economic Journal: Applied Economics · doi:10.1257/app.20150044 · Cited by 430
Using publication metrics to highlight academic productivity and research impact
Carpenter, C. R., Cone, D. C., & Sarli, C. C. (2014) · Academic Emergency Medicine · doi:10.1111/acem.12482 · Cited by 377
Meta-assessment of bias in science
Fanelli, D., Costas, R., & Ioannidis, J. P. A. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618569114 · Cited by 364
Self-citationIntended and unintended consequences of a publish‐or‐perish culture: A worldwide survey
van Dalen, H. P., & Henkens, K. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22636 · Cited by 355
A Meta-Analysis of State-of-the-Art Electoral Prediction From Twitter Data
Gayo-Avello, D. (2013) · Social Science Computer Review · doi:10.1177/0894439313493979 · Cited by 315
Attention bias modification for social anxiety: A systematic review and meta-analysis
Heeren, A., Mogoașe, C., Philippot, P., & McNally, R. J. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.06.001 · Cited by 312
Pickering, C., Grignon, J., Steven, R., Guitart, D., & Byrne, J. (2015) · Studies in Higher Education · doi:10.1080/03075079.2014.914907 · Cited by 285
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
Fanelli, D., Costas, R., & Larivière, V. (2015) · PLOS ONE · doi:10.1371/journal.pone.0127556 · Cited by 268
Self-citationWhy Growing Retractions Are (Mostly) a Good Sign
Fanelli, D. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001563 · Cited by 272
Self-citationMeta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265
The Chrysalis Effect: How Ugly Initial Results Metamorphosize Into Beautiful Articles
O’Boyle, E. H., Banks, G. C., & Gonzalez-Mulé, E. (2017) · Journal of Management · doi:10.1177/0149206314527133 · Cited by 246
What is open peer review? A systematic review
Ross-Hellauer, T. (2017) · F1000Research · doi:10.12688/f1000research.11369.1 · Cited by 232
“, 2017) mean that editors and reviewers seem primed to prefer positive results over negative or neutral ones (Bardy, 1998; Dickersin et al., 1992; Fanelli, 2010; Ioannidis, 1998), and to disfavour replication studies (Campanario, 1998; Kerr et al.”
Researchers’ Individual Publication Rate Has Not Increased in a Century
Fanelli, D., & Larivière, V. (2016) · PLOS ONE · doi:10.1371/journal.pone.0149504 · Cited by 205
Self-citationReforming Science: Methodological and Cultural Reforms
Casadevall, A., & Fang, F. C. (2012) · Infection and Immunity · doi:10.1128/iai.06183-11 · Cited by 156
“These factors, as well as the difficulty publishing negative results (publication bias), serve to undermine the reliability of the scientific literature (13, 18).”
BackgroundScience and the sources of hype
Caulfield, T., & Condit, C. (2012) · Public Health Genomics · doi:10.1159/000336533 · Cited by 152
Reproducible and replicable pain research: a critical review
Lee, H., Lamb, S. E., Bagg, M. K., Toomey, E., Cashin, A. G., & Moseley, G. L. (2018) · Pain · doi:10.1097/j.pain.0000000000001254 · Cited by 150
How the publish-or-perish principle divides a science: the case of economists
van Dalen, H. P. (2021) · Scientometrics · doi:10.1007/s11192-020-03786-x · Cited by 128
Rising complexity and falling explanatory power in ecology
Low-Décarie, E., Chivers, C., & Granados, M. (2014) · Frontiers in Ecology and the Environment · doi:10.1890/130230 · Cited by 111
ChatGPT and the Rise of Large Language Models: The New AI-Driven Infodemic Threat in Public Health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · doi:10.2139/ssrn.4352931 · Cited by 708
Calibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98
Commensuration Bias in Peer Review
Lee, C. J. (2015) · Philosophy of Science · doi:10.1086/683652 · Cited by 90
Reasons for article retraction in anesthesiology: a comprehensive analysis
Nair, S., Yean, C., Yoo, J., Leff, J., Delphin, E., & Adams, D. C. (2020) · Canadian Journal of Anesthesia/Journal canadien d'anesthésie · doi:10.1007/s12630-019-01508-3 · Cited by 90
Plain language summaries: A systematic review of theory, guidelines and empirical research
Stoll, M., Kerwer, M., Lieb, K., & Chasiotis, A. (2022) · PLOS ONE · doi:10.1371/journal.pone.0268789 · Cited by 89
Understanding the onset of hot streaks across artistic, cultural, and scientific careers
Liu, L., Dehmamy, N., Chown, J., Giles, C. L., & Wang, D. (2021) · Nature Communications · doi:10.1038/s41467-021-25477-8 · Cited by 81
Reproducing Statistical Results
Stodden, V. (2015) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-010814-020127 · Cited by 77
The researchers’ view of scientific rigor—survey on the conduct and reporting of in vivo research
Reichlin, T. S., Vogt, L., & Würbel, H. (2016) · PLOS ONE · doi:10.1371/journal.pone.0165999 · Cited by 72
Gijzen, M. W. M., Rasing, S. P. A., Creemers, D. H. M., Engels, R. C. M. E., & Smit, F. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.10.062 · Cited by 61
Be positive about negatives–recommendations for the publication of negative (or null) results
Bespalov, A., Steckler, T., & Skolnick, P. (2019) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2019.10.007 · Cited by 59
A Systems-Based Approach to Fostering Robust Science in Industrial-Organizational Psychology
Grand, J. A., Rogelberg, S. G., Allen, T. D., Landis, R. S., Reynolds, D. H., Scott, J. C., Tonidandel, S., & Truxillo, D. M. (2018) · Industrial and Organizational Psychology · doi:10.1017/iop.2017.55 · Cited by 52
Researcher's Perceptions on Publishing “Negative” Results and Open Access
Echevarría, L., Malerba, A., & Arechavala-Gomeza, V. (2021) · Nucleic Acid Therapeutics · doi:10.1089/nat.2020.0865 · Cited by 45
“Many reasons could be at the origin of this bias toward the preferential publication of positive results: one of them is the pressure to publish high-impact data, particularly among academics who are led to believe that their work needs to be groundbreaking to get published [3].”
BackgroundResults Blind Science Publishing
Locascio, J. J. (2017) · Basic and Applied Social Psychology · doi:10.1080/01973533.2017.1336093 · Cited by 45
The Effect of Open Access on Research Quality
van Vlokhoven, H. (2019) · Journal of Informetrics · doi:10.1016/j.joi.2019.04.001 · Cited by 36
Authentication: A Standard Problem or a Problem of Standards?
Capes-Davis, A., & Neve, R. M. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002477 · Cited by 33
Martin, M. M., Baker, N. C., Boyes, W. K., Carstens, K. E., Culbreth, M. E., Gilbert, M. E., Harrill, J. A., Nyffeler, J., Padilla, S., Friedman, K. P., et al. (2022) · Neurotoxicology and Teratology · doi:10.1016/j.ntt.2022.107117 · Cited by 29
Examining the predictive validity of NIH peer review scores
Lindner, M. D., & Nakamura, R. K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0126938 · Cited by 24
Grant writing and grant peer review as questionable research practices
Conix, S., De Block, A., & Vaesen, K. (2021) · F1000Research · doi:10.12688/f1000research.73893.1 · Cited by 23
Publication practice in Taxonomy: Global inequalities and potential bias against negative results
Salvador, R. B., Cavallari, D. C., Rands, D., & Tomotani, B. M. (2022) · PLOS ONE · doi:10.1371/journal.pone.0269246 · Cited by 21
Research transparency in dental research: A programmatic analysis
Raittio, E., Sofi‐Mahmudi, A., & Uribe, S. E. (2023) · European Journal of Oral Sciences · doi:10.1111/eos.12908 · Cited by 15
Preregistration of Studies with Existing Data
Mertens, G., & Krypotos, A.-M. (2022) · Integrity of Scientific Research · doi:10.1007/978-3-030-99680-2_36
Metadata
- Authors
- Fanelli, D.
- Affiliations
- University of Edinburgh (GB)
- Corresponding author
- Fanelli, Daniele
- ORCID
- Fanelli, Daniele
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2010-04-21
- Volume
- 5
- Issue
- 4
- Pages
- e10271
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2858206
- OpenAlex
- W1966297423
- Cited by
- 805
- References
- 60
- Citations by year
- 2026: 34 · 2025: 47 · 2024: 37 · 2023: 61 · 2022: 60 · 2021: 64 · 2020: 62 · 2019: 83 · 2018: 58 · 2017: 52 · 2016: 64 · 2015: 40 · 2014: 43 · 2013: 42 · 2012: 36
- FWCI
- 82.69
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- DigitalGeorgetown (Georgetown University Library) (submittedVersion)
- Funders
- European Commission (221441)
- Economic and Social Research Council (ES/F028180/1)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publish or perish, publication pressure, research bias, positive results, negative results, academic competition, research productivity, R&D investment, research integrity, hypothesis testing
- MeSH terms
- Ethics, Professional, Publishing, United States, Bias, Conflict of Interest, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.