Citation of retracted publications: A challenging problem
Abstract
Scientific publications with compromised integrity should be retracted. Papers citing retracted publications might need correction if findings depend on the retracted publication. While many studies have reported on post-retraction citations, few have focused on citations made before the retraction. We investigated the citation profile for a research group with 113 published concerns regarding publication integrity (CRPI). We identified 376 of their source publications that were cited by 5577 articles, and whether the source publication had a published CRPI. Of 6926 references to a source publication in these citing articles, for 3925 (57%) the source article had a published CRPI, while for 3001 (43%) it did not. Of these 3925 references, 3688 were in citing articles published before the source article CRPI was published. 166 citing articles containing 198 references to source publications were published after the corresponding source article CRPI was published (range 1-5 such references/article; 19/166 (11%) articles had >1 reference). In summary, many articles cite retracted publications, with the majority of these references occurring before the retraction. However, very few publications assess the impact of the retracted citations, even though the findings of many might be altered, at least in part, by removal of the retracted citation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Retraction does not stop the diffusion of retracted papers; they continue to be cited after retraction.
“Although retraction is intended to remove published research from the citable literature, retraction does not stop the diffusion of the retracted paper. Empirical studies have shown that papers continue to be cited after being retracted”
From Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine (2021) · cites this paper as Bolland, Grey, & Avenell, 2021
Cites
The Continued Use of Retracted, Invalid Scientific Literature
Pfeifer, M. P. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100140020 · Cited by 188 · see citing works
Systematic review and statistical analysis of the integrity of 33 randomized controlled trials
Bolland, M. J., Avenell, A., Gamble, G. D., & Grey, A. (2016) · Neurology · doi:10.1212/wnl.0000000000003387 · Cited by 111 · cited 2× · see citing works
“To assess the magnitude of the problem of citations of retracted papers with a particular emphasis on the timing of the retraction in relation to the publication of the citing article, we investigated the citation profile of publications by a research group in the field of osteoporosis with 113 published concerns rega…”
BackgroundSelf-citationAvenell, A., Stewart, F., Grey, A., Gamble, G., & Bolland, M. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-031909 · Cited by 62 · cited 3× · see citing works
“%) would likely change if the retracted paper was removed, and in a further 8/68 (12%), it was unclear if findings would change (Avenell et al. 2019).”
BackgroundInfluentialSelf-citationPost retraction citations among manuscripts reporting a radiology-imaging diagnostic method
Bolboacă, S. D., Buhai, D.-V., Aluaș, M., & Bulboacă, A. E. (2019) · PLOS ONE · doi:10.1371/journal.pone.0217918 · Cited by 42 · see citing works
Mott, A., Fairhurst, C., & Torgerson, D. (2019) · Journal of Health Services Research & Policy · doi:10.1177/1355819618797965 · Cited by 36 · cited 1× · see citing works
“…retracted articles (Pfeifer and Snodgrass 1990; Budd, Sievert, and Schultz 1998; Sox and Rennie 2006; Neale, Dailey, and Abrams 2010; BornemannCimenti, Szilagyi, and Sandner-Kiesling 2016; Bar-Ilan and Halevi 2017; Bolboaca et al. 2019; Mott, Fairhurst, and Torgerson 2019; Dal-Re and Ayuso 2020).”
BackgroundDal-Ré, R., & Ayuso, C. (2021) · Accountability in Research · doi:10.1080/08989621.2020.1835479 · Cited by 21 · cited 1× · see citing works
“…retracted articles (Pfeifer and Snodgrass 1990; Budd, Sievert, and Schultz 1998; Sox and Rennie 2006; Neale, Dailey, and Abrams 2010; BornemannCimenti, Szilagyi, and Sandner-Kiesling 2016; Bar-Ilan and Halevi 2017; Bolboaca et al. 2019; Mott, Fairhurst, and Torgerson 2019; Dal-Re and Ayuso 2020).”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Hsiao, T.-K., & Schneider, J. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00155 · Cited by 98 · see citing works
Avenell, A., Bolland, M. J., Gamble, G. D., & Grey, A. (2024) · Accountability in Research · doi:10.1080/08989621.2022.2082290 · Cited by 26 · see citing works
Self-citationAn analysis of retracted papers in Computer Science
Shepperd, M., & Yousefi, L. (2023) · PLOS ONE · doi:10.1371/journal.pone.0285383 · Cited by 16 · see citing works
Sridharan, K., & Sivaramakrishnan, G. (2026) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2025.1737168 · Cited by 8 · see citing works
Nag, S. N., Roy, A., & Sudhier, K. G. (2025) · Global Knowledge, Memory and Communication · doi:10.1108/gkmc-09-2024-0582 · Cited by 4 · see citing works
- Authors
- Bolland, M. J., Grey, A., & Avenell, A.
- Affiliations
- University of Auckland (NZ)
- University of Aberdeen (GB)
- Corresponding author
- Bolland, Mark J.
- Venue
- Accountability in Research
- Publisher
- Informa UK Limited
- Published
- 2021-02-15
- Volume
- 29
- Issue
- 1
- Pages
- 18-25
- ISSN
- 0898-9621 (ISSN-L)
- 1545-5815
- OpenAlex
- W3128789760
- Cited by
- 78
- References
- 6
- Citations by year
- 2026: 15 · 2025: 14 · 2024: 13 · 2023: 18 · 2022: 10 · 2021: 8
- FWCI
- 15.33
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Accountability in Research (publishedVersion)
- PubMed (publishedVersion)
- Funders
- MB is a recipient of an HRC Clinical Practitioners Fellowship
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- retracted article, publication integrity, citation impact
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
