An analysis of retracted papers in Computer Science
Open accessAbstract
Context: The retraction of research papers, for whatever reason, is a growing phenomenon. However, although retracted paper information is publicly available via publishers, it is somewhat distributed and inconsistent. Objective: The aim is to assess: (i) the extent and nature of retracted research in Computer Science (CS) (ii) the post-retraction citation behaviour of retracted works and (iii) the potential impact upon systematic reviews and mapping studies. Method: We analyse the Retraction Watch database and take citation information from the Web of Science and Google scholar. Results: We find that of the 33,955 entries in the Retraction watch database (16 May 2022), 2,816 are classified as CS, i.e., ≈ 8%. For CS, 56% of retracted papers provide little or no information as to the reasons. This contrasts with 26% for other disciplines. There is also some disparity between different publishers, a tendency for multiple versions of a retracted paper to be available beyond the Version of Record (VoR), and for new citations long after a paper is officially retracted (median = 3; maximum = 18). Systematic reviews are also impacted with ≈ 30% of the retracted papers having one or more citations from a review. Conclusions: Unfortunately, retraction seems to be a sufficiently common outcome for a scientific paper that we as a research community need to take it more seriously, e.g., standardising procedures and taxonomies across publishers and the provision of appropriate research tools. Finally, we recommend particular caution when undertaking secondary analyses and meta-analyses which are at risk of becoming contaminated by these problem primary studies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Erroneous publications may continue to shape subsequent research even after corrections or retractions.
“erroneous publications may continue to shape subsequent research even after corrections or retractions (Shepperd and Yousefi, 2023)”
From FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers (2025) · cites this paper as Shepperd and Yousefi, 2023
The Summary and recommendations reports that 76 of 120 sampled computer-science papers were cited at least a year after retraction, supporting the claim about continued influence after retraction. Our copy does not establish that erroneous publications continued to shape research after correction; its discussion of corrected papers suggests that later citations may instead refer to corrected versions. Page numbers are not available in our copy. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
2 retracted
Retraction rates are on the rise
Cokol, M., Ozbay, F., & Rodriguez‐Esteban, R. (2008) · The EMBO Reports · doi:10.1038/sj.embor.7401143 · Cited by 149 · see citing works
RetractedRetracted publications in pharmacy systematic reviews
Brown, S. J., Bakker, C., & Theis-Mahon, N. R. (2022) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2022.1280 · Cited by 35 · see citing works
RetractedSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · see citing works
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 2× · see citing works
“What is unclear is whether the growth is due to rising misconduct or improvements in our abilities to detect such situations as argued by Fanelli [2].”
Automatic classification of citation function
Teufel, S., Siddharthan, A., & Tidhar, D. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610091 · Cited by 437 · see citing works
Why Growing Retractions Are (Mostly) a Good Sign
Fanelli, D. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001563 · Cited by 272 · see citing works
Improving the Dependability of Research in Personality and Social Psychology
Funder, D. C., Levine, J. M., Mackie, D. M., Morf, C. C., Sansone, C., Vazire, S., & West, S. G. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868313507536 · Cited by 254 · see citing works
Post retraction citations in context: a case study
Bar-Ilan, J., & Halevi, G. (2017) · Scientometrics · doi:10.1007/s11192-017-2242-0 · Cited by 191 · see citing works
The Continued Use of Retracted, Invalid Scientific Literature
Pfeifer, M. P. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100140020 · Cited by 188 · see citing works
Brainard, J. (2018) · Science · doi:10.1126/science.362.6413.390 · Cited by 149 · see citing works
What a massive database of retracted papers reveals about science publishing's ‘death penalty'
(2018) · AAAS Articles DO Group · doi:10.1126/science.aav8384 · Cited by 144 · see citing works
Erikson, M. G., & Erlandson, P. (2014) · Social Studies of Science · doi:10.1177/0306312714522871 · Cited by 112 · see citing works
Schneider, J., Ye, D., Hill, A. M., & Whitehorn, A. S. (2020) · Scientometrics · doi:10.1007/s11192-020-03631-1 · Cited by 88 · cited 2× · see citing works
“This analysis of post-retraction citation patterns was followed up by Mott et al. [13] and Schneider et al. [14] both in the field of clinical trials.”
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81 · see citing works
Citation of retracted publications: A challenging problem
Bolland, M. J., Grey, A., & Avenell, A. (2022) · Accountability in Research · doi:10.1080/08989621.2021.1886933 · Cited by 78 · see citing works
Bordignon, F. (2020) · Scientometrics · doi:10.1007/s11192-020-03536-z · Cited by 51 · see citing works
Automatic Classification of Algorithm Citation Functions in Scientific Literature
Tuarob, S., Kang, S. W., Wettayakorn, P., Pornprasit, C., Sachati, T., Hassan, S., & Haddawy, P. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2913376 · Cited by 39 · see citing works
Mott, A., Fairhurst, C., & Torgerson, D. (2019) · Journal of Health Services Research & Policy · doi:10.1177/1355819618797965 · Cited by 36 · see citing works
Heibi, I., & Peroni, S. (2021) · Scientometrics · doi:10.1007/s11192-021-04097-5 · Cited by 29 · see citing works
Avenell, A., Bolland, M. J., Gamble, G. D., & Grey, A. (2024) · Accountability in Research · doi:10.1080/08989621.2022.2082290 · Cited by 26 · see citing works
A quantitative and qualitative open citation analysis of retracted articles in the humanities
Heibi, I., & Peroni, S. (2022) · Quantitative Science Studies · doi:10.1162/qss_a_00222 · Cited by 21 · see citing works
Multiple Publications: The Main Reason for the Retraction of Papers in Computer Science
Hidabi, M. A., & Teh, P. L. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-03402-3_35 · Cited by 15 · see citing works
A protocol to gather, characterize and analyze incoming citations of retracted articles
Heibi, I., & Peroni, S. (2022) · PLoS ONE · doi:10.1371/journal.pone.0270872 · Cited by 12 · see citing works
Lesk, M., Mattern, J. B., & Moulaison, H. L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15742-5_14 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Global retractions due to randomly generated content: Characterization and trends
Lei, F., Du, L., Dong, M., & Liu, X. (2024) · Scientometrics · doi:10.1007/s11192-024-05172-3 · Cited by 35 · see citing works
What is research data “misuse”? And how can it be prevented or mitigated?
Pasquetto, I. V., Cullen, Z., Thomer, A. K., & Wofford, M. F. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24944 · Cited by 17 · see citing works
Sebo, P., & Sebo, M. (2025) · Accountability in Research · doi:10.1080/08989621.2025.2484555 · Cited by 16 · see citing works
A data mining-based study on academic publication retractions in the 21st Century
Shen, Q., Gao, X.-Y., & Xiong, X.-M. (2025) · Accountability in Research · doi:10.1080/08989621.2025.2528064 · Cited by 3 · see citing works
Tracking the Retracted in Asia Higher Education Research Field
Cheng, M., & Xinhua, Y. (2026) · Higher Education Quarterly · doi:10.1111/hequ.70127 · Cited by 3 · see citing works
Rojas, E. M., & Zahn-Muñoz, C. (2025) · Data & Metadata · doi:10.56294/dm2025655 · Cited by 4 · see citing works
Wang, P. (2023) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3576840.3578281 · Cited by 4 · see citing works
Profiling article retractions in accounting and finance research
Charitou, A., Leventis, S., & Nerantzidis, M. (2025) · Journal of Accounting Literature · doi:10.1108/jal-10-2024-0287 · Cited by 1 · see citing works
Examining Retractions in Sustainability Research: Frequency, Causes, and Key Themes
Moustafa, A. M., & Mostafa, M. M. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-025-09669-1 · Cited by 1 · see citing works
Self-plagiarism: A retrospective study of its prevalence and patterns across scientific disciplines
Sevryugina, Y. V., & Li, Y. (2025) · Accountability in Research · doi:10.1080/08989621.2025.2472016 · Cited by 2 · see citing works
FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers
Xi, S., Rao, V., Payan, J., & Shah, N. B. (2025) · arXiv · doi:10.48550/arxiv.2511.21843
“Such oversights can have lasting consequences: erroneous publications may continue to shape subsequent research even after corrections or retractions (Shepperd and Yousefi, 2023), while retractions themselves impose substantial financial and scientific costs (Stern et al., 2014).”
A Hybrid Framework for Predicting Citation Harm from Retracted Papers
Zhao, J., Li, H., & Tang, S. (2025) · International Conference on Generative Artificial Intelligence and Information Security (GAIIS) · doi:10.1145/3728725.3728790
Saikia, S., Nisha, F., & Verma, M. K. (2025) · Applied Environmental Education & Communication · doi:10.1080/1533015x.2025.2599330
Oladokun, B. D., David-West, B., Yahaya, D. O., & Onaivi, E. N. (2026) · Serials Review · doi:10.1080/00987913.2026.2663258
Wang, J. J., Wei, S., Yuan, X., & Ye, F. Y. (2026) · Scientometrics · doi:10.1007/s11192-026-05741-8
From discovery to disrepute: a study of retractions in biological research
Bhat, M., & Malik, B. A. (2026) · Global Knowledge, Memory and Communication · doi:10.1108/gkmc-06-2025-0414
Citation Selectivity in Retraction Literature: A Large-Scale Bibliometric Analysis
Scolari, C., Zahra, R., Chen, D. L., Parsa, S., & Shang, L. (2026) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3805696.3846472
- Authors
- Shepperd, M., & Yousefi, L.
- Affiliations
- Brunel University of London (GB)
- Corresponding author
- Shepperd, Martin
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-05-09
- Volume
- 18
- Issue
- 5
- Pages
- e0285383
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10168558
- OpenAlex
- W4376131671
- Cited by
- 16
- References
- 25
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 2 · 2023: 1
- FWCI
- 4.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Chalmers Research (Chalmers University of Technology) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Brunel University Research Archive (BURA) (Brunel University London) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Vetenskapsrådet
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- retracted article, retraction, computer science research, citation behavior, Retraction Watch, systematic reviews, research integrity
- MeSH terms
- Computers, Contrast Media, Databases, Factual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
