‘Wasted’ research and lost citations: A scientometric assessment of retracted documents in Scopus between 2001 and 2024
Open accessAbstract
This study presents a large-scale scientometric analysis of 35,514 retracted publications indexed in Scopus between 2001 and 2024, focusing on temporal trends, citation dynamics and global disparities in retraction patterns. The primary goal was to investigate how retractions evolve over time and how retracted articles continue to influence academic discourse through ‘lost citations’ – citations to works no longer considered valid. Using generalised additive models, we identified two significant surges in retraction volume, in 2010–2011 and 2020–2022, and revealed that these were largely driven by bulk retractions in conference proceedings. Citation analysis showed that while early retracted papers were often highly cited, most retracted documents in the last decade have negligible scholarly impact. However, severe inequality persists: a small subset of retracted papers accumulates the majority of citations. At the author level, we found that citation influence is only weakly correlated with the number of retracted articles, indicating reputational distortions. Country-level trends revealed that China leads retraction counts, followed by India and the United States, while publisher-level data showed the dominance of conference-oriented outlets such as the IEEE Computer Society. Our findings raise important concerns about the persistence of invalidated knowledge in citation metrics and the structural vulnerabilities of certain publishing formats. This study calls for enhanced retraction labelling, improved transparency in databases and retraction-aware bibliometric evaluations to preserve academic integrity.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Retractions have become more common in recent years.
“Retractions have become more common in recent years, potentially because of lax oversight of conference proceedings, the use of ‘paper mills’ to generate substantial numbers of ghost-written publications, and other factors.1,2”
From Why retracted articles can have an influential afterlife—and what to do about it (2026) · cites this paper as [2]
Cites
4 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405 · cited 1× · see citing works
“…platforms such as the aforementioned Retraction Watch, or the foundation of different indices, the most famous of which is the Retraction Index [31], systematic analyses that combine retraction volume with citation metrics remain a novel field with scarcely frequent outputs in more recent…”
RetractedRetraction policies of top scientific journals ranked by impact factor
Resnik, D. B., Wager, E., & Kissling, G. E. (2015) · Journal of the Medical Library Association JMLA · doi:10.3163/1536-5050.103.3.006 · Cited by 72 · see citing works
RetractedKoo, M., & Lin, S.-C. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e38620 · Cited by 41 · see citing works
RetractedRetracted articles in oncology in the last three decades: frequency, reasons, and themes
Gholampour, B., Gholampour, S., Noruzi, A., Arsenault, C., Haertlé, T., & Saboury, A. A. (2022) · Scientometrics · doi:10.1007/s11192-022-04305-w · Cited by 32 · see citing works
Retractedggplot2: Elegant Graphics for Data Analysis
Ginestet, C. E. (2011) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00676_9.x · Cited by 14,484 · see citing works
Baas, J., Schotten, M., Plume, A., Côté, G., & Karimi, R. (2020) · Quantitative Science Studies · doi:10.1162/qss_a_00019 · Cited by 2,093 · see citing works
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1× · see citing works
“This can be traced back to the latency in academic misconduct [62], especially given that high-profile incidents might serve as ‘chilling effects’ in admitting scholarly mistakes, but the problem also lies in national policy reforms that may invertedly ‘force’ or ‘ignite’ researchers to be less…”
Generalized Additive Models: An Introduction With R
Anderson‐Cook, C. M. (2007) · Journal of the American Statistical Association · doi:10.1198/jasa.2007.s188 · Cited by 1,628 · see citing works
Misconduct accounts for the majority of retracted scientific publications
Fang, F. C., Steen, R. G., & Casadevall, A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1212247109 · Cited by 1,232 · cited 1× · see citing works
“After all, the retraction of an article essentially means that the respective work suffers from an error so grave – whether due to a factual error or academic misconduct, each representing different points on a spectrum of severity and intent, it is considered unfit for further research use [1–5].”
Methodology and Application of the Kruskal-Wallis Test
Ostertagová, E., Ostertag, O., & Kováč, J. (2014) · Applied Mechanics and Materials · doi:10.4028/www.scientific.net/amm.611.115 · Cited by 886 · see citing works
The Oxford Handbook of the Science of Science Communication
(2017) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780190497620.001.0001 · Cited by 525 · see citing works
A Comprehensive Survey of Retracted Articles from the Scholarly Literature
Grieneisen, M. L., & Zhang, M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0044118 · Cited by 355 · see citing works
More than 10,000 research papers were retracted in 2023 — a new record
Noorden, R. V. (2023) · Nature · doi:10.1038/d41586-023-03974-8 · Cited by 352 · see citing works
The fight against fake-paper factories that churn out sham science
Else, H., & Van Noorden, R. (2021) · Nature · doi:10.1038/d41586-021-00733-5 · Cited by 251 · see citing works
Wang, D., & Barabási, A.-Ĺ. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108610834 · Cited by 224 · see citing works
Post retraction citations in context: a case study
Bar-Ilan, J., & Halevi, G. (2017) · Scientometrics · doi:10.1007/s11192-017-2242-0 · Cited by 191 · see citing works
Financial costs and personal consequences of research misconduct resulting in retracted publications
Stern, A. M., Casadevall, A., Steen, R. G., & Fang, F. C. (2014) · eLife · doi:10.7554/elife.02956 · Cited by 189 · see citing works
Academic Knowledge Production and the Global South
Demeter, M. (2020) · Springer eBooks · doi:10.1007/978-3-030-52701-3 · Cited by 168 · see citing works
Retractions in the research literature: misconduct or mistakes?
Nath, S. B., Marcus, S. C., & Druss, B. G. (2006) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2006.tb00504.x · Cited by 161 · see citing works
Vuong, Q. (2020) · Learned Publishing · doi:10.1002/leap.1282 · Cited by 126 · see citing works
Butler, L.-A., Matthias, L., Simard, M.-A., Mongeon, P., & Haustein, S. (2023) · Quantitative Science Studies · doi:10.1162/qss_a_00272 · Cited by 112 · see citing works
Why do some retracted papers continue to be cited?
Teixeira da Silva, J. A., & Bornemann-Cimenti, H. (2017) · Scientometrics · doi:10.1007/s11192-016-2178-9 · Cited by 111 · see citing works
Abalkina, A. (2023) · Learned Publishing · doi:10.1002/leap.1574 · Cited by 101 · see citing works
Hsiao, T.-K., & Schneider, J. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00155 · Cited by 98 · see citing works
Sources of error in the retracted scientific literature
Casadevall, A., Steen, R. G., & Fang, F. C. (2014) · The FASEB Journal · doi:10.1096/fj.14-256735 · Cited by 89 · see citing works
The (lack of) Impact of Retraction on Citation Networks
Madlock-Brown, C. R., & Eichmann, D. (2015) · Science and Engineering Ethics · doi:10.1007/s11948-014-9532-1 · Cited by 89 · see citing works
Misconduct versus Honest Error and Scientific Disagreement
Resnik, D. B., & Stewart, C. N. (2012) · Accountability in Research · doi:10.1080/08989621.2012.650948 · Cited by 84 · see citing works
What studies of retractions tell us
Marcus, A., & Oransky, I. (2014) · Journal of Microbiology & Biology Education · doi:10.1128/jmbe.v15i2.855 · Cited by 81 · see citing works
China’s Research Evaluation Reform: What are the Consequences for Global Science?
Shu, F., Liu, S., & Larivière, V. (2022) · Minerva · doi:10.1007/s11024-022-09468-7 · Cited by 74 · see citing works
Analysis of retractions in Indian science
Elango, B., Kozak, M., & Rajendran, P. (2019) · Scientometrics · doi:10.1007/s11192-019-03079-y · Cited by 68 · see citing works
Ribeiro, M. D., & Vasconcelos, S. M. R. (2018) · Scientometrics · doi:10.1007/s11192-017-2621-6 · Cited by 58 · see citing works
Vuong, Q., La, V., Ho, T. M., Vuong, T., & Ho, M. (2020) · Science Editing · doi:10.6087/kcse.187 · Cited by 57 · see citing works
Kokol, P. (2023) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2023.1513 · Cited by 51 · see citing works
Fanelli, D., Wong, J., & Moher, D. (2022) · Accountability in Research · doi:10.1080/08989621.2021.1947810 · Cited by 49 · see citing works
Language in Social Science Databases: English Versus Non-English Articles in JSTOR and Scopus
Albarillo, F. (2014) · Behavioral & Social Sciences Librarian · doi:10.1080/01639269.2014.904693 · Cited by 47 · see citing works
Li, G., Kamel, M., Jin, Y., Xu, M. K., Mbuagbaw, L. C. E., Samaan, Z., Levine, M., & Thabane, L. (2018) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s151745 · Cited by 47 · see citing works
Team size and retracted citations reveal the patterns of retractions from 1981 to 2020
Sharma, K. (2021) · Scientometrics · doi:10.1007/s11192-021-04125-4 · Cited by 45 · see citing works
Ortega, J. L., & Delgado‐Quirós, L. (2024) · Scientometrics · doi:10.1007/s11192-024-05034-y · Cited by 45 · see citing works
China conducts first nationwide review of retractions and research misconduct
Mallapaty, S. (2024) · Nature · doi:10.1038/d41586-024-00397-x · Cited by 44 · see citing works
An analysis of Malaysian retracted papers: Misconduct or mistakes?
Khalid, Y. I. A. M., Abdullah, N., & Abrizah, A. (2018) · Scientometrics · doi:10.1007/s11192-018-2720-z · Cited by 40 · see citing works
Science’s fake-paper problem: high-profile effort will tackle paper mills
Sanderson, K. (2024) · Nature · doi:10.1038/d41586-024-00159-9 · Cited by 39 · see citing works
Describing long-term trends in precipitation using generalized additive models
Underwood, F. M. (2008) · Journal of Hydrology · doi:10.1016/j.jhydrol.2008.11.003 · Cited by 39 · see citing works
Silent or Stealth Retractions, the Dangerous Voices of the Unknown, Deleted Literature
da Silva, J. A. T. (2015) · Publishing Research Quarterly · doi:10.1007/s12109-015-9439-y · Cited by 38 · see citing works
Mott, A., Fairhurst, C., & Torgerson, D. (2019) · Journal of Health Services Research & Policy · doi:10.1177/1355819618797965 · Cited by 36 · see citing works
De Cassai, A., Geraldini, F., De Pinto, S., Carbonari, I., Cascella, M., Boscolo, A., Sella, N., Monteleone, F., Cavaliere, F., Munari, M., et al. (2022) · Anesthesiology · doi:10.1097/aln.0000000000004302 · Cited by 34 · see citing works
Identification of retracted publications and completeness of retraction notices in public health
Bakker, C. J., Reardon, E. E., Brown, S. J., Theis-Mahon, N., Schroter, S., Bouter, L., & Zeegers, M. P. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111427 · Cited by 32 · see citing works
Characteristics of retracted publications related to pain research: a systematic review
Ferraro, M. C., Moore, A., Williams, A. C. D. C., Fisher, E., Stewart, G. B., Ferguson, M. C., Eccleston, C., & O'Connell, N. E. (2023) · Pain · doi:10.1097/j.pain.0000000000002947 · Cited by 32 · see citing works
What time-lag for a retraction search on PubMed?
Decullier, É., Huot, L., & Maisonneuve, H. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-395 · Cited by 30 · see citing works
Systematic examination of pre‐ and post‐retraction citations
Dinh, L., Sarol, J., Cheng, Y., Hsiao, T., Parulian, N., & Schneider, J. (2019) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.35 · Cited by 23 · see citing works
Analysis of Retracted Publications in Medical Literature Due to Ethical Violations
Koçyiğit, B. F., Akyol, A., Zhaksylyk, A., Seiil, B., & Yessirkepov, M. (2023) · Journal of Korean Medical Science · doi:10.3346/jkms.2023.38.e324 · Cited by 22 · see citing works
Identifying fabricated networks within authorship-for-sale enterprises
Porter, S. J., & McIntosh, L. D. (2024) · Scientific Reports · doi:10.1038/s41598-024-71230-8 · Cited by 22 · see citing works
Liang, H., Zhao, K., & Li, J. (2024) · Journal of Higher Education Policy and Management · doi:10.1080/1360080x.2024.2446372 · Cited by 22 · see citing works
Didier, E., & Guaspare, C. (2018) · Social Studies of Science · doi:10.1177/0306312718756202 · Cited by 22 · see citing works
An Analysis of Recently Retracted Articles by Authors Affiliated with Hospitals in Mainland China
Zhao, T., Dai, T., Lun, Z., & Gao, Y. (2021) · Journal of Scholarly Publishing · doi:10.3138/jsp.52.2.03 · Cited by 22 · see citing works
Retractions from altmetric and bibliometric perspectives
Shema, H., Hahn, O., Mazarakis, A., & Peters, I. (2019) · Information - Wissenschaft & Praxis · doi:10.1515/iwp-2019-2006 · Cited by 21 · see citing works
Characterizing the effect of retractions on publishing careers
Memon, S. A., Makovi, K., & AlShebli, B. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02154-0 · Cited by 21 · see citing works
Chain retraction: how to stop bad science propagating through the literature
Cabanac, G. (2024) · Nature · doi:10.1038/d41586-024-02747-1 · Cited by 21 · see citing works
Dobránszki, J., & da Silva, J. A. T. (2019) · Scientometrics · doi:10.1007/s11192-019-03190-0 · Cited by 19 · see citing works
Never mind predatory publishers… what about ‘grey’ publishers?
Nicholas, D., Herman, E., Abrizah, A., Bravo, B. R., Boukacem‐Zeghmouri, C., Watkinson, A., Świgoń, M., Xu, J., Jamali, H. R., & Tenopir, C. (2023) · El Profesional de la Informacion · doi:10.3145/epi.2023.sep.09 · Cited by 19 · see citing works
The effectiveness of peer review in identifying issues leading to retractions
Zheng, X. M., Chen, J., Tollas, A., & Ni, C. (2023) · Journal of Informetrics · doi:10.1016/j.joi.2023.101423 · Cited by 19 · see citing works
High retraction rate of Chinese articles: it is time to do something about academic misconduct
Xiao, Y., Chen, J., Xiao-hong, W., & Qiu, Q.-M. (2021) · Postgraduate Medical Journal · doi:10.1136/postgradmedj-2021-140853 · Cited by 18 · see citing works
A bibliometric study of article retractions in technology fields in developing economies countries
Eldakar, M. A. M., & Shehata, A. M. K. (2023) · Scientometrics · doi:10.1007/s11192-023-04823-1 · Cited by 18 · see citing works
The decrease in uncited articles and its effect on the concentration of citations
Kozlowski, D., Andersen, J. P., & Larivière, V. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24852 · Cited by 18 · see citing works
Why are retractions so difficult?
Wager, E. (2015) · Science Editing · doi:10.6087/kcse.34 · Cited by 17 · see citing works
Mapping retracted articles and exploring regional differences in China, 2012–2023
Shi, L., Zhang, X., Ma, X., Sun, X., Li, J., & He, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0314622 · Cited by 14 · see citing works
Abramo, G., Aguillo, I. F., Aksnes, D. W., Boyack, K. W., Burrell, Q. L., Campanario, J. M., Chinchilla‐Rodríguez, Z., Costas, R., D’Angelo, C. A., Harzing, A.-W. K., et al. (2022) · Scientometrics · doi:10.1007/s11192-022-04565-6 · Cited by 12 · see citing works
Scientific retraction: a synonym for pseudoscience?
Sheth, B. P., & Thaker, V. S. (2014) · Acta bioethica · doi:10.4067/s1726-569x2014000100010 · Cited by 11 · see citing works
Szabo, C. (2025) · Columbia University Press eBooks · doi:10.7312/szab21623 · Cited by 11 · see citing works
Thelwall, M. A., & Levitt, J. M. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23969 · Cited by 10 · see citing works
Chinese authors are overrepresented in medical articles retracted for fake peer review or paper mill
Sebo, P. (2024) · Internal and Emergency Medicine · doi:10.1007/s11739-024-03616-5 · Cited by 9 · see citing works
Tóth, J., & Demeter, M. (2021) · KOME · doi:10.17646/kome.75672.61 · Cited by 9 · see citing works
Retractions and Rewards in Science: An Open Question for Reviewers and Funders
Ribeiro, M. D., KALICHMAN, M. W., & Vasconcelos, S. M. R. (2023) · Science and Engineering Ethics · doi:10.1007/s11948-023-00446-0 · Cited by 7 · see citing works
Retractions in Scopus: An Engineering Journal Articles Investigation
Solomon, D., Heckman, C., & Hubbard, D. E. (2022) · Science & Technology Libraries · doi:10.1080/0194262x.2022.2098892 · Cited by 4 · see citing works
Conference Issues Vs. Conference Proceedings
Kampourakis, K. (2017) · Science & Education · doi:10.1007/s11191-017-9922-2 · Cited by 3 · see citing works
Köstenbach, T., & Oransky, I. (2024) · Information - Wissenschaft & Praxis · doi:10.1515/iwp-2023-2041 · Cited by 1 · see citing works
Paper Mills and Intellectual Prostitution in Pakistani Research: A Challenge to Scientific Integrity
Waheed, U. (2024) · Journal of Islamabad Medical & Dental College · doi:10.35787/jimdc.v13i(suppl.).1320 · Cited by 1 · see citing works
China has already taken steps to reduce retractions of papers from its hospitals
Zhang, G. J., & Zhou, Y. B. (2025) · Nature · doi:10.1038/d41586-025-01007-0 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Why retracted articles can have an influential afterlife—and what to do about it
Snyder, J., & Turner, L. (2026) · Stem Cells Translational Medicine · doi:10.1093/stcltm/szag080
Yaghtin, M., Sotudeh, H., Hoseini, M., & Amiri, M. (2026) · Online Information Review · doi:10.1108/oir-04-2025-0270
Modeling the impact of research data unavailability on science
Chamorro-Padial, J., Rodrigo-Ginés, F.-J., Sánchez, R. M. R., Gil, R., & Garcı́a, R. (2026) · Journal of Informetrics · doi:10.1016/j.joi.2026.101813
Peng, Z., Zhao, M., Wang, H., Yang, A. J., & Deng, S. (2026) · Scientometrics · doi:10.1007/s11192-026-05808-6
Pérez-Campdesuñer, R., Sánchez-Rodríguez, A., Martínez-Vivar, R., Miguel-Guzmán, M. D., Rodríguez-Flores, J., & García-Vidal, G. (2026) · Publications · doi:10.3390/publications14040065
Retractions in the field of pregnancy and pain: bibliometric characteristics, reasons, and trends
Yerlanova, D., Zimba, O., Khojakulova, U., Nurmashev, B., Qumar, A. B., & Koçyiğit, B. F. (2026) · Rheumatology International · doi:10.1007/s00296-026-06319-6
- Authors
- Lendvai, G. F., & Sasvári, P.
- Affiliations
- Ludovika University of Public Service (HU)
- Corresponding author
- Lendvai, Gergely Ferenc
- Venue
- Journal of Information Science
- Publisher
- SAGE Publications
- Published
- 2025-08-23
- Volume
- 52
- Issue
- 3
- Pages
- 666-683
- ISSN
- 0165-5515 (ISSN-L)
- 1741-6485
- OpenAlex
- W4413471211
- Cited by
- 6
- References
- 77
- Citations by year
- 2026: 6
- FWCI
- 3.86
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Information Science (publishedVersion, cc-by-nc)
- Közszolgálati Tudásportál (submittedVersion, other-oa)
- Funders
- Ministry of Culture and Innovation of Hungary (TKP2021-NKTA-51)
- National Research, Development and Innovation Fund (TKP2021-NKTA)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- retracted article, citation dynamics, citation inequality, author reputation, academic integrity, Scopus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
