Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
OBJECTIVE To study variations in the number of times trials of nicotine replacement therapy (NRT) were cited, and which characteristics of trials predicted the number of citations and the impact factors of journals in which articles were published. STUDY DESIGN AND SETTING We used all 105 randomized controlled trials in the Cochrane review of NRT for smoking cessation. We obtained impact factors from the Journal Citation Reports and the number of citations from ISI Web of Knowledge and Google Scholar. RESULTS Trials were cited from 0 to 632 times (median 23 times). Trials were cited more often when results were statistically significant than when they were not (median=41 vs. 17 times, P<0.001), and when impact factors were higher (10.2 more citations per impact factor point, P<0.001). Patch trials were cited more often than gum trials (median=29 vs. 17 times, P=0.001), and trials funded by the pharmaceutical industry were cited more often than other trials (median=28 vs. 16.5 times, P=0.001). Trials with statistically significant results were published in journals with higher impact factors than trials with nonsignificant results (median impact factor=2.80 vs. 1.81, P=0.011). CONCLUSION Citations were biased toward trials with positive results and toward trials published in high-impact-factor journals.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770 · cited 2×
“It is calculated by dividing the number of citations to citable items published in a journal during the preceding 2 years by the number of articles published in those 2 years by that journal [12].”
BackgroundWhy the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447
A controlled trial of sustained-release bupropion, a nicotine patch, or both for smoking cessation
Jorenby, D. E., Leischow, S. J., Nides, M. A., Rennard, S. I., Johnston, J. A., Hughes, A. R., Smith, S. S., Muramoto, M. L., Daughton, D. M., Doan, K., et al. (1999) · New England Journal of Medicine · doi:10.1056/nejm199903043400903 · Cited by 1,585 · cited 1×
“Excluding one outlier (a comparison of NRT with bupropion, cited 475 times in ISI and 632 times in Google Scholar) [14], this correlation was reduced to r 5 0.”
BackgroundContradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Callaham, M. (2002) · JAMA · doi:10.1001/jama.287.21.2847 · Cited by 644
Simkin, M. V., Roychowdhury, V. P., & Department of Electrical Engineering, U. O. C. (2024) · Complex Systems · doi:10.25088/complexsystems.14.3.269 · Cited by 192
Citation indexes do not reflect methodological quality in lung cancer randomised trials.
Berghmans, T., Meert, A. P., Mascaux, C., Paesmans, M., Lafitte, J. J., & Sculier, J. P. (2003) · Annals of Oncology · doi:10.1093/annonc/mdg203 · Cited by 38
Cited by
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
Citation bias favoring statistically significant studies was present in medical research
Jannot, A.-S., Agoritsas, T., Gayet-Ageron, A., & Perneger, T. V. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.09.015 · Cited by 193
Scientific citations favor positive results: a systematic review and meta-analysis.
Duyx, B., Urlings, M. J. E., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.002 · Cited by 197
Falagas, M. E., Zarkali, A., Karageorgopoulos, D. E., Bardakas, V., & Mavros, M. N. (2013) · PLoS ONE · doi:10.1371/journal.pone.0049476 · Cited by 168
“While the effect of journal impact factor [5–10] and study design [11–16] on citations received has been established by different studies, the published evidence on other potentially relevant variables, such as open access to the full text of the article, [17–20] or article length, [13] seems conflicting.”
Background
Metadata
- Authors
- Etter, J.-F., & Stapleton, J.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2009-08
- Volume
- 62
- Issue
- 8
- Pages
- 831-837
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2088701056
- Cited by
- 40
- References
- 7
- FWCI
- 3.18
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. https://app.cochat.ai/papers/10.1016/j.jclinepi.2008.09.015
- The paper
- Etter, J.-F., & Stapleton, J. (2009). Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2008.09.015
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.