Scientific citations favor positive results: a systematic review and meta-analysis.
Open accessAbstract
Objectives Citation bias concerns the selective citation of scientific articles based on their results. We brought together all available evidence on citation bias across scientific disciplines and quantified its impact.Study design and setting An extensive search strategy was applied to the Web of Science Core Collection and Medline, yielding 52 studies in total. We classified these studies on scientific discipline, selection method, and other variables. We also performed random-effects meta-analyses to pool the effect of positive vs. negative results on subsequent citations. Finally, we checked for other determinants of citation as reported in the citation bias literature.Results Evidence for the occurrence of citation bias was most prominent in the biomedical sciences and least in the natural sciences. Articles with statistically significant results were cited 1.6 (95% confidence interval [CI] 1.3-1.8) times more often than articles with nonsignificant results. Articles in which the authors explicitly conclude to have found support for their hypothesis were cited 2.7 (CI 2.0-3.7) times as often. Article results and journal impact factor were associated with citation more often than any other reported determinant.Conclusion Similar to what we already know on publication bias, also citation bias can lead to an overrepresentation of positive results and unfounded beliefs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The tendency of journals to favor AI studies with positive results-especially those that demonstrate superior performance-skews the published literature, giving an incomplete view of the AI landscape.
“Furthermore, the tendency of journals to favor AI studies with positive results—especially those that demonstrate superior performance—skews the published literature, giving an incomplete view of the AI landscape [109].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [109]
Positive studies receive more citations from others than null or negative studies (citation bias).
“and cited more frequently by others (citation bias) 19”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1× · see citing works
“According to a meta-analysis of surveys [3], researchers report to engage in QRPs (about 34%) much more often than in research misconduct (about 2%).”
BackgroundThe Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483 · see citing works
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465 · see citing works
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · see citing works
Callaham, M. (2002) · JAMA · doi:10.1001/jama.287.21.2847 · Cited by 644 · see citing works
How citation distortions create unfounded authority: analysis of a citation network
Greenberg, S. A. (2009) · BMJ · doi:10.1136/bmj.b2680 · Cited by 499 · see citing works
Fostering integrity in research: Definitions, current knowledge, and future directions
Steneck, N. H. (2006) · Science and Engineering Ethics · doi:10.1007/pl00022268 · Cited by 407 · see citing works
What determines the citation frequency of ecological papers?
Leimu, R., & Koricheva, J. (2004) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2004.10.010 · Cited by 368 · see citing works
Cholesterol lowering trials in coronary heart disease: Frequency of citation and outcome.
Ravnskov, U. (1992) · BMJ · doi:10.1136/bmj.305.6844.15 · Cited by 313 · see citing works
Aguinis, H., Dalton, D. R., Bosco, F. A., Pierce, C. A., & Dalton, C. M. (2011) · Journal of Management · doi:10.1177/0149206310377113 · Cited by 302 · see citing works
Schrag, M., Mueller, C., Oyoyo, U. E., Smith, M. A., & Kirsch, W. M. (2011) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2011.05.001 · Cited by 266 · see citing works
A case for a duty to feed the hungry: GM plants and the third world
Carter, L. (2007) · Science and Engineering Ethics · doi:10.1007/s11948-006-0006-y · Cited by 265 · see citing works
Ioannidis, J. P. A. (2011) · JAMA · doi:10.1001/jama.2011.713 · Cited by 253 · see citing works
Minimizing the Three Stages of Publication Bias
Chalmers, T. C. (1990) · JAMA · doi:10.1001/jama.1990.03440100104016 · Cited by 226 · see citing works
Bouter, L. M., Tijdink, J., Axelsen, N., Martinson, B. C., & ter Riet, G. (2016) · Research Integrity and Peer Review · doi:10.1186/s41073-016-0024-5 · Cited by 211 · see citing works
Self-citationCitation bias favoring statistically significant studies was present in medical research
Jannot, A.-S., Agoritsas, T., Gayet-Ageron, A., & Perneger, T. V. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.09.015 · Cited by 193 · see citing works
The citation bias: Fad and fashion in the judgment and decision literature.
Christensen-Szalanski, J. J. J., & Beach, L. R. (1984) · American Psychologist · doi:10.1037/0003-066x.39.1.75 · Cited by 189 · see citing works
The relationship between quality of research and citation frequency
Nieminen, P. A., Carpenter, J. R., Rücker, G., & Schumacher, M. C. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-42 · Cited by 185 · see citing works
Characteristics Associated with Citation Rate of the Medical Literature
Kulkarni, A. V., Busse, J. W., & Shams, I. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000403 · Cited by 165 · see citing works
Factors associated with citation rates in the orthopedic literature.
Bhandari, M., Busse, J. W., Devereaux, P. J., Montori, V. M., Swiontkowski, M. F., Tornetta, P., Einhorn, T. A., Khera, V., & Schemitsch, E. H. (2007) · PubMed Central · Cited by 120 · see citing works
Useem, J., Brennan, A., LaValley, M., Vickery, M., Ameli, O., Reinen, N., & Gill, C. J. (2015) · PLOS ONE · doi:10.1371/journal.pone.0144980 · Cited by 105 · see citing works
West, R., & McIlwaine, A. (2002) · Addiction · doi:10.1046/j.1360-0443.2002.00104.x · Cited by 89 · see citing works
Citation Bias in Reported Smoking Prevalence in People with Schizophrenia
Chapman, S., Ragg, M., & McGeechan, K. (2009) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/00048670802653372 · Cited by 83 · see citing works
Statistically significant papers in psychiatry were cited more often than others
Nieminen, P. A., Rücker, G., Miettunen, J., Carpenter, J. R., & Schumacher, M. C. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.11.014 · Cited by 78 · see citing works
Why do we think we know what we know? A metaknowledge analysis of the salt controversy
Trinquart, L., Johns, D. M., & Galea, S. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyv184 · Cited by 77 · see citing works
Dissemination biases in ecology: effect sizes matter more than quality
Barto, E. K., & Rillig, M. C. (2011) · Oikos · doi:10.1111/j.1600-0706.2011.19401.x · Cited by 74 · see citing works
Is There a Citation Bias in the Judgment and Decision Literature?
Robins, R. W., & Craik, K. H. (1993) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1993.1011 · Cited by 73 · see citing works
Citation bias of hepato-biliary randomized clinical trials
Gluud, L. L., & Gluud, C. N. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00513-3 · Cited by 72 · see citing works
Ohta, N., Ishiguro, S., Kawabata, A., Uppalapati, D., Pyle, M. M., Troyer, D. L., De, S., Zhang, Y., Becker, K. G., & Tamura, M. (2015) · PLoS ONE · doi:10.1371/journal.pone.0123756 · Cited by 70 · see citing works
The difference between highly and poorly cited medical articles in the journal Lancet
Kostoff, R. N. (2007) · Scientometrics · doi:10.1007/s11192-007-1573-7 · Cited by 67 · see citing works
Positive results receive more citations, but only in some disciplines
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-012-0757-y · Cited by 67 · see citing works
Determinants of the citation rate of medical research publications from a developing country
Annalingam, A., Damayanthi, H., Jayawardena, R., & Ranasinghe, P. (2014) · SpringerPlus · doi:10.1186/2193-1801-3-140 · Cited by 64 · see citing works
Voracek, M. (2014) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2014.05.009 · Cited by 63 · see citing works
Meta-analysis selection bias in marketing research
Eisend, M., & Tarrahi, F. (2014) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2014.03.006 · Cited by 53 · see citing works
The Scientific Impact of Positive and Negative Phase 3 Cancer Clinical Trials
Unger, J. M., Barlow, W. E., Ramsey, S. D., LeBlanc, M. L., Blanke, C. D., & Hershman, D. L. (2016) · JAMA Oncology · doi:10.1001/jamaoncol.2015.6487 · Cited by 53 · see citing works
Etter, J., & Stapleton, J. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.09.015 · Cited by 40 · see citing works
Understanding belief using citation networks
Greenberg, S. A. (2011) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2011.01646.x · Cited by 38 · see citing works
Andrade, N. S., Flynn, J. P., & Bartanusz, V. (2013) · The Spine Journal · doi:10.1016/j.spinee.2013.06.071 · Cited by 38 · see citing works
Differential Citation Rates of Major Cardiovascular Clinical Trials According to Source of Funding
Conen, D., Torres, J., & Ridker, P. M. (2008) · Circulation · doi:10.1161/circulationaha.108.794016 · Cited by 32 · see citing works
Sawin, V. I., & Robinson, K. A. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.026 · Cited by 29 · see citing works
de Vries, Y. A., Roest, A. M., Franzen, M., Munafò, M. R., & Bastiaansen, J. A. (2016) · Psychological Medicine · doi:10.1017/s0033291716000805 · Cited by 28 · see citing works
Lortie, C., Aarssen, L., Budden, A., & Leimu, R. (2012) · Scientometrics · doi:10.1007/s11192-012-0822-6 · Cited by 27 · see citing works
A retrospective analysis of dissemination biases in the brief alcohol intervention literature.
Tanner-Smith, E. E., & Polanin, J. R. (2015) · Psychology of Addictive Behaviors · doi:10.1037/adb0000014 · Cited by 26 · cited 1× · see citing works
“their selection on a previously published review or reviews [6,11,15,16,25,27,30,33,35,36,38,39,41,44,48,50,51].”
BackgroundMika Kivimaki, B., Batty, G. D., Kawachi, I. I., Virtanen, M., Singh‐Manoux, A., & Brunner, E. J. (2014) · American Journal of Epidemiology · doi:10.1093/aje/kwu164 · Cited by 24 · see citing works
Bastiaansen, J. A., de Vries, Y. A., & Munafò, M. R. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.12.007 · Cited by 24 · see citing works
Kumar, G. K., Kumar, K. K., Baumgarten, G., & Ramkumar, G. (2014) · Journal of Atmospheric and Solar-Terrestrial Physics · doi:10.1016/j.jastp.2014.12.001 · Cited by 22 · see citing works
Misemer, B. S., Platts‐Mills, T. F., & Jones, C. W. (2016) · Trials · doi:10.1186/s13063-016-1595-7 · Cited by 21 · see citing works
Evidence of Pro-Homosexual Bias in Social Science: Citation Rates and Research on Lesbian Parenting
Schumm, W. R. (2010) · Psychological Reports · doi:10.2466/pr0.106.2.374-380 · Cited by 20 · see citing works
Poor Citation of Prior Evidence in Hip Fracture Trials
Sheth, U., Simunovic, N., Tornetta, P., Einhorn, T. A., & Bhandari, M. (2011) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.j.01274 · Cited by 20 · see citing works
Koren, G., & Nickel, C. (2011) · PubMed · Cited by 19 · see citing works
Nieri, M., Clauser, C., Franceschi, D., Pagliaro, U., Saletta, D., & Pini‐Prato, G. (2007) · Clinical Oral Implants Research · doi:10.1111/j.1600-0501.2007.01350.x · Cited by 19 · see citing works
Parameter identification of the observed citation distribution
Yu, G., & Li, Y. (2007) · Scientometrics · doi:10.1007/s11192-007-1662-7 · Cited by 19 · see citing works
Hutchison, B. G., Oxman, A. D., & Lloyd, S. (1995) · PubMed Central · Cited by 16 · see citing works
Perino, A. C., Hoang, D. D., Holmes, T. H., Santangeli, P., Heidenreich, P. A., Perez, M. V., Wang, P. J., & Turakhia, M. P. (2014) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.114.000912 · Cited by 15 · see citing works
Kim, M., & Lee, H.-S. (2015) · Journal of Sensory Studies · doi:10.1111/joss.12152 · Cited by 14 · see citing works
Peritz, B. C. (1994) · Scientometrics · doi:10.1007/bf02017221 · Cited by 12 · see citing works
Citations alone were enough to predict favorable conclusions in reviews of neuraminidase inhibitors
Zhou, X., Wang, Y., Tsafnat, G., Coiera, E. W., Bourgeois, F. T., & Dunn, A. G. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.09.014 · Cited by 11 · see citing works
Association between Article Citation Rate and Level of Evidence in the Companion Animal Literature
Giuffrida, M. A., & Brown, D. C. (2012) · Journal of Veterinary Internal Medicine · doi:10.1111/j.1939-1676.2011.00869.x · Cited by 11 · see citing works
Methodological quality in clinical trials and bibliometric indicators: no evidence of correlations
Akcan, D., Axelsson, S., Bergh, C., Davidson, T., & Rosén, M. (2013) · Scientometrics · doi:10.1007/s11192-013-0949-0 · Cited by 11 · see citing works
Valachis, A., Mauri, D. N., Neophytou, C., Polyzos, N. P., Tsali, L., Garras, A., & Papanikolau, E. G. (2011) · International Journal of Medical Sciences · doi:10.7150/ijms.8.492 · Cited by 11 · see citing works
M-free: Mask-independent scoring of the reference bias
Kunz, M., Zhou, Y., & Frangakis, A. S. (2015) · Journal of Structural Biology · doi:10.1016/j.jsb.2015.08.016 · Cited by 10 · see citing works
Zambon, M., Biondi‐Zoccai, G., Elena, B., Ruggeri, L., Zangrillo, A., & Landoni, G. (2012) · Journal of Anesthesia · doi:10.1007/s00540-012-1372-z · Cited by 10 · see citing works
Allareddy, V., Shah, A., Lin, C., Elangovan, S., Lee, M. K., Chung, K., Vasudavan, S., & Karimbux, N. Y. (2010) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2009.12.001 · Cited by 9 · see citing works
Promotion of “Low T” and citation bias in testosterone studies
Zhao, J. V., & Schooling, C. M. (2015) · International Journal of Cardiology · doi:10.1016/j.ijcard.2015.03.010 · Cited by 6 · see citing works
Etwel, F., & Koren, G. (2015) · Clinical and investigative medicine · doi:10.25011/cim.v38i5.25684 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Duyx B, Urlings MJE, Swaen G, Bouter LM, Zeegers MP. Protocol for systematic review of citation bias 2015. Available at http://hdl. handle.net/10411/20710. Accessed September 30, 2016.
- Peritz BC. The use of meta-analyses for citation studies: an illustra- tive example. Informetrics 1992;91:358e76.
Cited by
Sorted by citations
Bias in medical AI: Implications for clinical decision-making
Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000651 · Cited by 607 · see citing works
Dealing with the positive publication bias: Why you should really publish your negative results
Mlinarić, A., Horvat, M., & Smolčić, V. Š. (2017) · Biochemia Medica · doi:10.11613/bm.2017.030201 · Cited by 518 · see citing works
Misinformation in and about science
West, J. D., & Bergstrom, C. T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912444117 · Cited by 391 · see citing works
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1468 · Cited by 294 · see citing works
Research Weaving: Visualizing the Future of Research Synthesis
Nakagawa, S., Samarasinghe, G., Haddaway, N., Westgate, M. J., O’Dea, R. E., Noble, D. W. A., & Lagisz, M. (2018) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2018.11.007 · Cited by 260 · see citing works
Considering bias and conflicts of interest among the included studies
Boutron, I., Page, M. J., Higgins, J. P., Altman, D. G., Lundh, A., Hróbjartsson, A., & on behalf of the Cochrane Bias Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch7 · Cited by 258 · see citing works
Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important?
Robinson, K. A., Brunnhuber, K., Ciliska, D., Juhl, C. B., Christensen, R., & Lund, H. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.020 · Cited by 146 · see citing works
Bishop, D. (2019) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021819886519 · Cited by 109 · see citing works
Methodology over metrics: current scientific standards are a disservice to patients and society
Van Calster, B., Wynants, L., Riley, R. D., van Smeden, M., & Collins, G. S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.018 · Cited by 105 · see citing works
de Vries, Y. A., Roest, A. M., de Jonge, P., Cuijpers, P. J., Munafo, M. R., & Bastiaansen, J. A. (2018) · Psychological Medicine · doi:10.1017/s0033291718001873 · Cited by 100 · see citing works
Researcher and Author Impact Metrics: Variety, Value, and Context
Gasparyan, A. Y., Yessirkepov, M., Duisenova, A., Trukhachev, V. I., Kostyukova, E. I., & Kitas, G. D. (2018) · Journal of Korean Medical Science · doi:10.3346/jkms.2018.33.e139 · Cited by 97 · see citing works
Urlings, M. J. E., Duyx, B., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.11.019 · Cited by 102 · see citing works
Self-citationEEG datasets for seizure detection and prediction— A review
Wong, S., Simmons, A., Rivera‐Villicana, J., Barnett, S., Sivathamboo, S., Perucca, P., Ge, Z., Kwan, P., Kuhlmann, L., Vasa, R., et al. (2023) · Epilepsia Open · doi:10.1002/epi4.12704 · Cited by 95 · see citing works
Embracing higher education leadership in sustainability: A systematic review
Sanchez-Carrillo, J. C., Cadarso, M., & Gómez, M. Á. T. (2021) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2021.126675 · Cited by 92 · see citing works
Magliano, D. J., Chen, L., Carstensen, B., Gregg, E. W., Pavkov, M. E., Salim, A., Andes, L. J., Balicer, R., Baviera, M., Chan, J. C. N., et al. (2022) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(21)00327-2 · Cited by 91 · see citing works
The Dark Side of Radiomics: On the Paramount Importance of Publishing Negative Results
Buvat, I., & Orlhac, F. (2019) · Journal of Nuclear Medicine · doi:10.2967/jnumed.119.235325 · Cited by 79 · see citing works
Reeves, R., Curran, D., Gleeson, A., & Hanna, D. (2022) · Behavior Modification · doi:10.1177/0145445521991102 · Cited by 69 · see citing works
“frequently published in scientific literature (Duyx et al., 2017).”
Scientific Hypotheses: Writing, Promoting, and Predicting Implications
Gasparyan, A. Y., Ayvazyan, L., Mukanova, U., Yessirkepov, M., & Kitas, G. D. (2019) · Journal of Korean Medical Science · doi:10.3346/jkms.2019.34.e300 · Cited by 64 · see citing works
van den Akker, O. R., van Assen, M. A. L. M., Bakker, M., Elsherif, M. M., Wong, T. K., & Wicherts, J. M. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02277-0 · Cited by 78 · see citing works
Top-cited articles in medical professionalism: a bibliometric analysis versus altmetric scores
Azer, S. A., & Azer, S. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029433 · Cited by 57 · see citing works
Citation bias, diversity, and ethics
Ray, K. S., Zurn, P., Dworkin, J. D., Bassett, D. S., & Resnik, D. B. (2022) · Accountability in Research · doi:10.1080/08989621.2022.2111257 · Cited by 60 · see citing works
Discontinuing Long-Term PPI Therapy: Why, With Whom, and How?
Targownik, L. E. (2018) · The American Journal of Gastroenterology · doi:10.1038/ajg.2018.29 · Cited by 54 · see citing works
Interneuron Dysfunction and Inhibitory Deficits in Autism and Fragile X Syndrome
Nomura, T. (2021) · Cells · doi:10.3390/cells10102610 · Cited by 48 · see citing works
Salivary biomarkers in breast cancer diagnosis: A systematic review and diagnostic meta‐analysis
Koopaie, M., Kolahdooz, S., Fatahzadeh, M., & Manifar, S. (2022) · Cancer Medicine · doi:10.1002/cam4.4640 · Cited by 49 · see citing works
Scott, M., Shaver, N., Lapenskie, J., Isenberg, S. R., Saunders, S., Hsu, A. T., & Tanuseputro, P. (2019) · Palliative Medicine · doi:10.1177/0269216319870649 · Cited by 47 · see citing works
Yuan, Z.-Z., Fan, Y.-Z., Cheng, S.-J., Wei, F.-J., Gao, J., Wang, C., Song, B.-S., Tan, S.-L., Gao, S.-L., Kang, J.-J., et al. (2025) · Journal of Nanobiotechnology · doi:10.1186/s12951-025-03090-x · Cited by 44 · see citing works
Research fraud: a long-term problem exacerbated by the clamour for research grants
Harvey, L. (2020) · Quality in Higher Education · doi:10.1080/13538322.2020.1820126 · Cited by 43 · see citing works
Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis
Nuijten, M. B., van Assen, M. A. L. M., Augusteijn, H. E. M., Crompvoets, E. A. V., & Wicherts, J. M. (2020) · Journal of Intelligence · doi:10.3390/jintelligence8040036 · Cited by 42 · see citing works
Kirkpatrick, R. H., Munoz, D. P., Khalid-Khan, S., & Booij, L. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2020.11.023 · Cited by 41 · see citing works
Duyx, B., Swaen, G. M. H., Urlings, M. J. E., Bouter, L. M., & Zeegers, M. P. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1082-9 · Cited by 40 · see citing works
Self-citationPhaf, R. H. (2020) · Theory & Psychology · doi:10.1177/0959354319898250 · Cited by 38 · see citing works
The right to refuse unwanted citations: rethinking the culture of science around the citation
da Silva, J. A. T., & Vuong, Q. (2021) · Scientometrics · doi:10.1007/s11192-021-03960-9 · Cited by 37 · see citing works
Citation bias: questionable research practice or scientific misconduct?
Gøtzsche, P. C. (2022) · Journal of the Royal Society of Medicine · doi:10.1177/01410768221075881 · Cited by 37 · see citing works
Promoting learning from null or negative results in prevention science trials
Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K. (2022) · Prevention Science · doi:10.1007/s11121-020-01140-4 · Cited by 34 · see citing works
Doleman, B., Mathiesen, O., Jakobsen, J. C., Sutton, A. J., Freeman, S. C., Lund, J. N., & Williams, J. P. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2021.01.004 · Cited by 34 · see citing works
Vitamin D in early life and later risk of multiple sclerosis—A systematic review, meta-analysis
Ismailova, K., Poudel, P., Parlesak, A., Frederiksen, P., & Heitmann, B. L. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221645 · Cited by 31 · see citing works
Schneider, J. W., Allum, N., Andersen, J. P., Petersen, M. B., Madsen, E. B., Mejlgaard, N., & Zachariae, R. H. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304342 · Cited by 30 · see citing works
Frank, R. A., Sharifabadi, A. D., Salameh, J., McGrath, T. A., Kraaijpoel, N., Dang, W., Li, N., Gauthier, I. D., Wu, M. Z., Bossuyt, P. M., et al. (2018) · European Radiology · doi:10.1007/s00330-018-5801-8 · Cited by 30 · see citing works
Khan, M. S., Shahid, I., Asad, N., Greene, S. J., Khan, S. U., Doukky, R., Metra, M., Anker, S. D., Filippatos, G. S., Fonarow, G. C., et al. (2020) · ESC Heart Failure · doi:10.1002/ehf2.13099 · Cited by 27 · see citing works
Fernández, N., Niño, A. P., Azuero, A. J., O’Kelly, F., Hannick, J. H., Rickard, M., Kirsch, A. J., Caldamone, A. A., & Koyle, M. A. (2018) · Journal of Pediatric Urology · doi:10.1016/j.jpurol.2018.04.004 · Cited by 26 · see citing works
Biases in scholarly recommender systems: impact, prevalence, and mitigation
Färber, M., Coutinho, M., & Yuan, S. (2023) · Scientometrics · doi:10.1007/s11192-023-04636-2 · Cited by 28 · see citing works
Yilmaz, T., Jutten, R. J., Santos, C. Y., Hernandez, K. A., & Snyder, P. J. (2018) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1016/j.trci.2018.03.005 · Cited by 24 · see citing works
Page, L., NOUSSAIR, C. N., & Slonim, R. L. (2021) · Journal of the Economic Science Association · doi:10.1007/s40881-021-00107-7 · Cited by 23 · see citing works
Bass, S. B., Brajuha, J. A., Kelly, P. J. A., D’Avanzo, P. A., Lambertini, E., Nordhagen, S., & Monterrosa, E. C. (2022) · Foodborne Pathogens and Disease · doi:10.1089/fpd.2021.0056 · Cited by 23 · see citing works
How to Handle Co-authorship When Not Everyone’s Research Contributions Make It into the Paper
Helgesson, G., Master, Z., & Bülow, W. (2021) · Science and Engineering Ethics · doi:10.1007/s11948-021-00303-y · Cited by 22 · see citing works
Leng, G., & Leng, R. (2020) · The MIT Press eBooks · doi:10.7551/mitpress/12228.001.0001 · Cited by 24 · see citing works
Hughes, Z. D., Ball, L. J., Richardson, C., & Judge, J. (2023) · Psychonomic Bulletin & Review · doi:10.3758/s13423-023-02327-w · Cited by 21 · see citing works
Salivary noncoding RNA in the diagnosis of pancreatic cancer: Systematic review and meta‐analysis
Koopaie, M., Kolahdooz, S., Fatahzadeh, M., & Aleedawi, Z. A. (2022) · European Journal of Clinical Investigation · doi:10.1111/eci.13848 · Cited by 19 · see citing works
The 50 most-cited articles on temporomandibular disorders: A bibliometric analysis
Ferrillo, M., Gallo, V., Lippi, L., Bruni, A., Montrella, R., Curci, C., Calafiore, D., Invernizzi, M., Migliario, M., & Alessandro de, S. I. R. E. (2022) · Journal of Back and Musculoskeletal Rehabilitation · doi:10.3233/bmr-220152 · Cited by 19 · see citing works
Bast, A., Semen, K. O., & Drent, M. (2021) · Current Opinion in Pulmonary Medicine · doi:10.1097/mcp.0000000000000777 · Cited by 19 · see citing works
Bestiary of Questionable Research Practices in Psychology
Nagy, T., Hergert, J., Elsherif, M. M., Wallrich, L., Schmidt, K., Waltzer, T., Payne, J. W., Gjoneska, B., Seetahul, Y., Wang, Y. A., et al. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251348431 · Cited by 24 · see citing works
Through the quality kaleidoscope: reflections on research in dentomaxillofacial imaging
Rohlin, M., Horner, K., Lindh, C., & Wenzel, A. (2020) · Dentomaxillofacial Radiology · doi:10.1259/dmfr.20190484 · Cited by 18 · see citing works
Significance and implications of accurate and proper citations in clinical research studies
Ngatuvai, M., Autrey, C., McKenny, M., & Elkbuli, A. (2021) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2021.102841 · Cited by 24 · see citing works
Highly cited educational technology journal articles: a descriptive and critical analysis
Mertala, P., Moens, E., & Teräs, M. (2022) · Learning Media and Technology · doi:10.1080/17439884.2022.2141253 · Cited by 22 · see citing works
Questionable Research Practices
O’Boyle, E. H., & Götz, M. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780190938550.003.0010 · Cited by 17 · see citing works
Scientific integrity and fraud in radiology research
Kwee, R. M., Almaghrabi, M. T., & Kwee, T. C. (2022) · European Journal of Radiology · doi:10.1016/j.ejrad.2022.110553 · Cited by 17 · see citing works
A good idea or just an idea: Which adaptation strategies for conservation are tested?
Hansen, L. J., Braddock, K. N., & Rudnick, D. A. (2023) · Biological Conservation · doi:10.1016/j.biocon.2023.110276 · Cited by 16 · see citing works
Lajus, D., Головин, П. В., Yurtseva, A., Иванова, Т. С., Dorgham, A. S., & Иванов, М. В. (2019) · Evolutionary ecology research · Cited by 15 · see citing works
Krivzov, J., Notaerts, L., Nieuwenhove, K. V., Meganck, R., Truijens, F. L., & Goossens, A. (2021) · European Journal of Psychotherapy & Counselling · doi:10.1080/13642537.2021.1923051 · Cited by 14 · see citing works
Xie, S., & Mi, C. (2023) · Learned Publishing · doi:10.1002/leap.1515 · Cited by 14 · see citing works
Christl, H., Morilla, J., Hoen, T., & Zumkier, U. (2019) · Integrated Environmental Assessment and Management · doi:10.1002/ieam.4115 · Cited by 13 · see citing works
Salandra, R., Criscuolo, P., & Salter, A. J. (2020) · Research Policy · doi:10.1016/j.respol.2020.104130 · Cited by 14 · see citing works
Perski, O., Simons, D., Shahab, L., & Brown, J. (2021) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntab214 · Cited by 14 · see citing works
Treanor, L. M., Frank, R. A., Salameh, J., Sharifabadi, A. D., McGrath, T. A., Kraaijpoel, N., Bossuyt, P. M., & McInnes, M. D. F. (2019) · American Journal of Roentgenology · doi:10.2214/ajr.18.20977 · Cited by 13 · see citing works
Resilience Metrics for Socio-Ecological and Socio-Technical Systems: A Scoping Review
Steinmann, P., Tobi, H., & van Voorn, G. A. K. (2024) · Systems · doi:10.3390/systems12090357 · Cited by 11 · see citing works
Overcoming citation bias is necessary for true inclusivity in Plant Science
Pandey, S., & Burch‐Smith, T. M. (2023) · The Plant Cell · doi:10.1093/plcell/koad248 · Cited by 11 · see citing works
Duyx, B., Urlings, M. J. E., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026518 · Cited by 11 · see citing works
Self-citationCitation bias in the literature on dietary trans fatty acids and serum cholesterol
Urlings, M. J. E., Duyx, B., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.10.008 · Cited by 11 · see citing works
Self-citationDini, C., Pereira, M. M. A., Souza, J. G. S., Shibli, J. A., de Avila, E. D., & Barão, V. A. R. (2022) · Clinical Implant Dentistry and Related Research · doi:10.1111/cid.13065 · Cited by 9 · see citing works
Dewig, H., Cohen, J. N., Renaghan, E., Leary, M. E., Leary, B. K., Au, J. S., & Tenan, M. S. (2024) · Sports Medicine · doi:10.1007/s40279-024-02066-5 · Cited by 10 · see citing works
Uttley, L., Falzon, L., Byrne, J. A., Tricco, A. C., Munafo, M. R., Moher, D., Stoeger, T., Matandika, L., Labbé, C., & Naudet, F. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111616 · Cited by 10 · see citing works
Citation culture: Citing authors behaviour vs trust in research results
Тихонова, Е. В., & Kirillova, O. V. (2023) · Science Editor and Publisher · doi:10.24069/sep-22-58 · Cited by 11 · see citing works
The Value of Systematic Reviews and Meta-Analyses in Surgery
Knight, S. R. (2021) · European Surgical Research · doi:10.1159/000519593 · Cited by 10 · see citing works
Wu, X., Yan, Q., Riley, P., Hua, F., Shi, B., Glenny, A., & Tu, Y. (2020) · Clinical Oral Implants Research · doi:10.1111/clr.13620 · Cited by 10 · see citing works
Brown, A. C., Ross, P. P., Brown, S. M., & Mulcahey, M. K. (2021) · Arthroscopy Sports Medicine and Rehabilitation · doi:10.1016/j.asmr.2021.09.013 · Cited by 9 · see citing works
Is the Psychopathic Brain an Artifact of Coding Bias? A Systematic Review
Jalava, J., Griffiths, S., Larsen, R. R., & Alcott, B. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.654336 · Cited by 5 · see citing works
Investigating and Dealing with Publication Bias and Other Reporting Biases
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch5 · Cited by 9 · see citing works
Magama, N., Ojemaye, M. O., Manene, N. C., Okoh, O. O., & Okoh, A. I. (2024) · Ionics · doi:10.1007/s11581-024-05666-3 · Cited by 8 · see citing works
Movahed, F., Hosseini, F. H., Heidari, A., Dehbozorgi, M., Ataei, M., Vahidi, F., Abyaneh, R., Bajelan, B., Bakhtiyari, M., & Shafiee, A. (2025) · Journal of Infection and Public Health · doi:10.1016/j.jiph.2025.102825 · Cited by 8 · see citing works
Mapping Orthopedic Research Trends in Nepal (2000-2024): A Scopus-Based Bibliometric Study
Regmi, A., & Niraula, B. B. (2025) · Indian Journal of Orthopaedics · doi:10.1007/s43465-025-01528-w · Cited by 8 · see citing works
Positive Outcome Bias in the Influential Otolaryngology Clinical Trial Literature
Yang, H., Tsai, M., Mukdad, L. A., & John, M. A. S. (2023) · Otolaryngology · doi:10.1002/ohn.562 · Cited by 8 · see citing works
Duyx, B., Urlings, M. J. E., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P. (2017) · Research Integrity and Peer Review · doi:10.1186/s41073-017-0041-z · Cited by 8 · see citing works
Self-citationSystematic review of biological therapies for atrial fibrillation
McRae, C., Kapoor, A., Kanda, P., Hibbert, B. M., & Davis, D. R. (2019) · Heart Rhythm · doi:10.1016/j.hrthm.2019.03.021 · Cited by 8 · see citing works
Packer, M. (2019) · Journal of the American Heart Association · doi:10.1161/jaha.119.013779 · Cited by 7 · see citing works
Der Testungseffekt beim Lernen mit Texten
Rummer, R. (2021) · Psychologische Rundschau · doi:10.1026/0033-3042/a000518 · Cited by 8 · see citing works
An Introduction to Systematic Reviews and Meta-Analyses
Rao, S. C. (2023) · APIK Journal of Internal Medicine · doi:10.4103/ajim.ajim_36_23 · Cited by 8 · see citing works
Spin in randomized controlled trials of pharmacology in COVID-19: A systematic review
Oh, Y.-K., Jung, Y.-J., Purja, S., Kim, M., Yon, D. K., Lee, S. W., Cho, K., Koyanagi, A., Dai, Z., Smith, L., et al. (2023) · Accountability in Research · doi:10.1080/08989621.2023.2269083 · Cited by 7 · see citing works
POSITIVE RESULTS BIAS AND IMPACT FACTOR IN RETINA CLINICAL TRIALS 2016 to 2019
Parikh, A., Markle, J. C., Venincasa, M. J., Kuriyan, A. E., Patel, M. M., & Sridhar, J. S. (2021) · Retina · doi:10.1097/iae.0000000000003107 · Cited by 7 · see citing works
Christl, H., Hoen, T., & Zumkier, U. (2019) · Integrated Environmental Assessment and Management · doi:10.1002/ieam.4218 · Cited by 7 · see citing works
What do analyses of city size distributions have in common?
Cottineau, C. (2022) · Scientometrics · doi:10.1007/s11192-021-04256-8 · Cited by 7 · see citing works
Engidaw, A. E., Zou, W., Ning, J., Francesco, G., & Hanchen, J. (2025) · SAGE Open · doi:10.1177/21582440251321357 · Cited by 6 · see citing works
Zhang, L., Banihashemi, S., Zhang, Y., & Chen, S. (2025) · Project Leadership and Society · doi:10.1016/j.plas.2025.100181 · Cited by 6 · see citing works
Pattinson, D., & Currie, G. (2025) · Learned Publishing · doi:10.1002/leap.2012 · Cited by 20 · see citing works
Reflections and practices of citing papers in health care science -a focus group study
Herling, S. F., Jespersen, K. F., & Møller, A. M. (2021) · Nordisk Sygeplejeforskning · doi:10.18261/issn.1892-2021-03-06 · Cited by 6 · see citing works
Impact of podcasting on novel and conventional measures of academic impact
Levin, G., Harrison, R. F., Meyer, R., & Ramirez, P. T. (2023) · International Journal of Gynecological Cancer · doi:10.1136/ijgc-2022-004114 · Cited by 6 · see citing works
Schweinsberg, M., Thau, S., & Pillutla, M. M. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221144990 · Cited by 6 · see citing works
Sorjonen, K., Ghaderi, A., & Bo, M.-L. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1463701 · Cited by 5 · see citing works
Evaluation of “Spin” in the Abstracts of Randomized Controlled Trial Reports in Cardiology
Roberts, W., Cooper, C. M., Khattab, M., Neff, P., Wildes, D. E., Wayant, C., & Vassar, M. B. (2020) · Journal of Osteopathic Medicine · doi:10.7556/jaoa.2020.133 · Cited by 6 · see citing works
Misleading with citation statistics?
Bennet, L., Eisner, D. A., & Gunn, A. J. (2019) · The Journal of Physiology · doi:10.1113/jp277847 · Cited by 6 · see citing works
Yu, H. W. (2021) · The Journal of Antibiotics · doi:10.1038/s41429-021-00439-w · Cited by 6 · see citing works
- Authors
- Duyx, B., Urlings, M. J. E., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P.
- Affiliations
- Maastricht University (NL)
- Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Duyx, Bram
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2017-08
- Volume
- 88
- Pages
- 92-101
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2623032108
- Cited by
- 197
- References
- 66
- Citations by year
- 2026: 15 · 2025: 20 · 2024: 17 · 2023: 27 · 2022: 28 · 2021: 39 · 2020: 19 · 2019: 19 · 2018: 11 · 2017: 2
- FWCI
- 3.43
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Pure Amsterdam UMC (submittedVersion)
- VU Research Portal (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Funders
- European Chemical Industry Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- citation bias, publication bias, positive results, non-significant results, scientific citations, journal impact factor, hypothesis confirmation, scientific disciplines
- MeSH terms
- Humans, Periodicals as Topic, Publication Bias, Journal Impact Factor
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
