The DOI matches its Crossref record; 67 of the 73 references it cites were found in a registry or an index; 27 of the 53 sources its claims cite were checked for support.
Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine
Open accessAbstract
We present the first database-wide study on the citation contexts of retracted papers, which covers 7,813 retracted papers indexed in PubMed. Our temporal analyses show that retracted papers continued to be cited. Furthermore, among the 13,252 postretraction citation contexts studied, only 722 (5.4%) citation contexts acknowledged the retraction, meaning 94.6% of post-retraction citations in biomedicine do not document the retraction and continue to cite retracted research as valid.
Overview
CoChat Summary
TL;DR
This database-wide study of 7,813 PubMed retracted papers, 169,434 iCite citations, and 48,134 PMC open-access citation contexts finds that retracted papers continued to be cited, but old retracted papers stopped being cited as time progressed. Among 13,252 postretraction citation contexts, only 722 (5.4%) acknowledged the retraction, and retraction did not change how the papers were cited.
Key findings
- Retracted papers continued to be cited: 6,704 (85.81%) retracted papers were cited 169,434 times, while 1,109 (14.19%) were never cited; 75% of retracted papers were cited no more than six times after retraction. The proportion of active retracted papers decreased from 80.23% to 25.36% since 2013, though up to 2018 more than 50% were still active.
- Among 13,252 postretraction citation contexts, only 722 (5.4%) acknowledged the retraction; 94.6% did not. In citing–cited pairs, 422 of 9,405 postretraction pairs (4.49%) acknowledged retractions, while 8,983 (95.51%) did not.
- Retraction did not change citation patterns: pre- and postretraction citation contexts showed similar text progression, with postretraction contexts slightly more concentrated at the beginning and end of papers and more often in introduction/background and discussion/conclusion sections.
- For the 722 intentional postretraction citation contexts, the most common purpose was related work (453, 62.74%); the second most prevalent was example of problematic science. In systematic reviews and meta-analyses, exclusion rationale was most common; in articles about retraction or scientific misconduct, subject of study was most common.
Methods
- Data sources: retracted papers from PubMed using “retracted publication”[PT] on August 20, 2020 (7,813 results); citations from iCite (169,434 citations after exclusions); citation contexts from the PMCOA Citation Context Dataset built from 2,049,871 PMC open-access articles (48,134 contexts analyzed).
- Postretraction citation was operationalized as a citation made after the calendar year of retraction. Temporal analyses categorized retracted papers as active, inactive, or uncited over time.
- Intentional postretraction citations were identified using cue words in the citation context or a five-sentence acknowledgment window, or co-citation with the retraction notice, followed by manual review; purposes were manually classified into 11 categories, with intercoder agreement Kappa = .63 on a 100-context sample.
Limitations
- Stated: findings cannot be projected into the future because journals and authors can reach back in time to retract articles; the data are right-censored because retractions typically happen after publication, and uncited or inactive retracted papers could become active in the future.
- Stated: annotation was done primarily by the first author, and intercoder agreement was fair (Kappa = .63); the cue-word approach might falsely capture some citations, so each identified context was manually reviewed.
- Evident from the text: citation contexts come only from the PMC open-access subset, and citations come from iCite/PubMed, so the study does not cover all postretraction citations; defining postretraction as after the calendar year of retraction may exclude same-year citations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
19 Valid · 6 Not valid · 12 Not checked · 96 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Although retraction is intended to remove published research from the citable literature, retraction does not stop the diffusion of the retracted paper. Empirical studies have shown that papers continue to be cited after being retracted”
§ Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine > 1. INTRODUCTION
What the sources say
- Supportedcites Bar-Ilan & Halevi, 2017 source
The Discussion and conclusions state that the retracted articles in this case study continued to be cited years after retraction despite posted notices. The quantitative findings report 238 post-retraction citations, directly supporting the claim that retraction does not stop citations. (LLM, conf=0.94)
- Not checked — no full text of the sourcecites Bolland, Grey, & Avenell, 2021 source
- Supportedcites Candal-Pedreira, Ruano-Ravina et al., 2020 source
The Abstract (Conclusions) states that retracted articles continue to be cited. The Discussion likewise reports that articles retracted due to misconduct continue to receive citations, so retraction does not eliminate their use. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Five out of nine previous studies published in the last 5 years (2016–2021) reported fewer than 7% of citations acknowledging retractions (Bolboacă et al., 2019; Budd et al., 2016; Hamilton, 2019; Schneider et al., 2020; van der Vet & Nijveen, 2016).”
§ Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine > 2. LITERATURE REVIEW > 2.2. Citation Contexts Citing Retracted Papers
What the sources say
- Supportedcites Bolboacă et al., 2019 source
The Conclusions report that fewer than 2% of cases identified the cited manuscript as retracted in the text or reference list. That supports including this 2019 study among studies finding fewer than 7% of citations acknowledging retractions; the paper does not itself establish the citing sentence’s count of five out of nine studies. (LLM, conf=0.92)
- Supportedcites Budd et al., 2016 source
In the CITATIONS results (“Total Articles: 265”), Budd et al. report that 204 citations, or 4.15%, noted the retraction. That supports including this 2016 study among those reporting fewer than 7%; the sentence’s five-of-nine count is a collective summary of the cited studies, not a count attributed to Budd et al. alone. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites Hamilton, 2019 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“However, postretraction citations can persist for more than 10 years (Kim, Yi et al., 2019; Schneider et al., 2020)”
§ Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine > 1. INTRODUCTION
What the sources say
- Supportedcites Kim, Yi et al., 2019 source
The Key results section explicitly states that the authors found post-retraction citations could persist for more than 10 years after the retraction announcement. (LLM, conf=0.94)
- Supportedcites Schneider et al., 2020 source
The section on combining citation context and network analysis says the study examines continued direct citation more than a decade after the paper’s 2008 retraction, through 2019—11 years later. This directly supports the claim that post-retraction citations can persist for more than 10 years. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Suelzer et al. (2019) studied citations to the Wakefield paper and reported that 71.7% of the citations documented the retraction after the paper was fully retracted in 2010.”
§ Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine > 5. DISCUSSION AND CONCLUSIONS
What the sources say
- Supportedcites Suelzer et al. (2019) source
The Results > Retractions passage reports that 360 of 502 citing works published between 2011 and 2018 (71.7%) documented either the partial retraction or the 2010 notice of retraction. This supports the claim for that post-2010 period. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Low proportions of postretraction citations acknowledging the retraction were also reported in some case studies (Schneider et al., 2020; van der Vet & Nijveen, 2016) and older studies (Budd et al., 1999; Pfeifer & Snodgrass, 1990).”
§ Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine > 5. DISCUSSION AND CONCLUSIONS
What the sources say
- Supportedcites Schneider et al., 2020 source
The Abstract reports that the retraction was not mentioned in 96% (107/112) of direct post-retraction citations examined, leaving a low proportion that acknowledged it. It explicitly describes the work as a single-case study (N = 1). (LLM, conf=0.94)
- Supportedcites van der Vet & Nijveen, 2016 source
In Results and discussion > Results (page unavailable), the case study reports that, with two exceptions, none of the papers directly citing the Narayan paper showed awareness of its retraction, although most papers in the 2015 network were published well after it. Only one exception explicitly noted the retraction; the other called the result controversial. (LLM, conf=0.94)
- Not checked — no full text of the sourcecites Budd et al., 1999
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
60 more claims · 3 with their cited sources checked
Create a free account to see all 133 claims and every source verdict.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 38
- Borderline: one of the tests for a substantial claim is unclear. 2
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We decided to keep retracted publications and publications with an expression of concern as they usually continue to receive citations48.”
From Scientific publications that use promotional language in the abstract receive more citations and public attention (2025) · cites this paper as [48]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Cited works that were retracted
Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Wakefield, 1998, Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children, Lancet, 351, 637, 10.1016/s0140-6736(97)11096-0”
Cited work 10.1016/s0140-6736(97)11096-0 · retracted 2010-02-06 · notice 10.1016/s0140-6736(10)60175-4 · this paper published 2021
A note, not an issue: for example, a cited work that was retracted only after this paper was published, which its authors could not have known.
In the paper
“Lancet, 2010, Retraction—Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children, Lancet, 375, 445, 10.1016/S0140-6736(10)60175-4”
Cited work 10.1016/s0140-6736(10)60175-4 · retraction date not known · this paper published 2021
Citations
Cites
2 retracted
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
RetractedLancet, T. E. O. T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 465
RetractedA Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
(2015) · Chinese Nursing Research · doi:10.1016/j.cnre.2016.01.001 · Cited by 739
Automatic classification of citation function
Teufel, S., Siddharthan, A., & Tidhar, D. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610091 · Cited by 437
Content‐based citation analysis: The next generation of citation analysis
Ding, Y., Zhang, G., Chambers, T., Song, M., Wang, X., & Zhai, C. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23256 · Cited by 252
Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence
Albrecht, S. V., Crandall, J. W., & Ramamoorthy, R. (2015) · National Conference on Artificial Intelligence · Cited by 248
Tahamtan, I., & Bornmann, L. (2019) · Scientometrics · doi:10.1007/s11192-019-03243-4 · Cited by 236
Post retraction citations in context: a case study
Bar-Ilan, J., & Halevi, G. (2017) · Scientometrics · doi:10.1007/s11192-017-2242-0 · Cited by 191
The Continued Use of Retracted, Invalid Scientific Literature
Pfeifer, M. P. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100140020 · Cited by 188 · cited 2×
“Different approaches have been used to identify post-retraction citations, such as after one month (Mott et al., 2019), after six months (Hamilton, 2019), and after the calendar year of retraction (Budd et al., 2011, 2016; Madhugiri et al., 2021; Pfeifer & Snodgrass, 1990).”
MethodResultGuidelines for retracting articles
Barbour, V., Kleinert, S., Wager, E., & Yentis, S. (2009) · doi:10.24318/cope.2019.1.4 · Cited by 185
Identifying Meaningful Citations
Valenzuela, M., Ha, V., & Etzioni, O. (2015) · National Conference on Artificial Intelligence · Cited by 153
NLP-driven citation analysis for scientometrics
Jha, R. K., JBARA, A.-A., Qazvinian, V., & Radev, D. (2016) · Natural Language Engineering · doi:10.1017/s1351324915000443 · Cited by 131
The NIH Open Citation Collection: A public access, broad coverage resource
Hutchins, B. I., Baker, K. L., Davis, M. T., Diwersy, M., Haque, E., Harriman, R. M., Hoppe, T. A., Leicht, S. A., Meyer, P., & Santangelo, G. M. (2019) · PLoS Biology · doi:10.1371/journal.pbio.3000385 · Cited by 124
Temporal characteristics of retracted articles
Bar-Ilan, J., & Halevi, G. (2018) · Scientometrics · doi:10.1007/s11192-018-2802-y · Cited by 107 · cited 2×
“As addressed in previous studies (Bertin et al., 2016; Dong & Schäfer, 2011) the location of citation implies the function of a cited work in the citing work.”
BackgroundResultEffects of article retraction on citation and practice in medicine.
(1999) · Europe PMC (PubMed Central) · Cited by 101
The invariant distribution of references in scientific articles
Bertin, M., Atanassova, I., Gingras, Y., & Larivière, V. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23367 · Cited by 91 · cited 3×
“As addressed in previous studies (Bertin et al., 2016; Dong & Schäfer, 2011) the location of citation implies the function of a cited work in the citing work.”
BackgroundResultEmpirical developments in retraction
Redman, B. K., Yarandi, H. N., & Merz, J. F. (2008) · Journal of Medical Ethics · doi:10.1136/jme.2007.023069 · Cited by 91
Schneider, J., Ye, D., Hill, A. M., & Whitehorn, A. S. (2020) · Scientometrics · doi:10.1007/s11192-020-03631-1 · Cited by 88 · cited 10×
“Five out of nine previous studies published in the last five years (2016 – 2021) reported fewer than 7% of citations acknowledging retractions (Bolboaca (cid:1) et al., 2019; Budd et al., 2016; Hamilton, 2019; Schneider et al., 2020; van der Vet & Nijveen, 2016).”
BackgroundResultInfluentialSelf-citationBornemann-Cimenti, H., Szilagyi, I. S., & Sandner-Kiesling, A. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-015-9680-y · Cited by 87 · cited 3×
“Interestingly, studies on infamous, high-profile retracted papers (Bornemann-Cimenti et al., 2016; Piller, 2021; Suelzer et al., 2019) found higher rates of citations acknowledging retractions.”
BackgroundInfluential(2011) · Cited by 84
Citation of retracted publications: A challenging problem
Bolland, M. J., Grey, A., & Avenell, A. (2022) · Accountability in Research · doi:10.1080/08989621.2021.1886933 · Cited by 78
A visual analytic study of retracted articles in scientific literature
Chen, C., Hu, Z., Milbank, J., & Schultz, T. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22755 · Cited by 75 · cited 2×
“However, post-retraction citations can persist for more than 10 years (Kim, Yi et al., 2019; Schneider et al., 2020), and retracted papers may be deeply embedded in citation networks (Chen, Hu et al., 2013).”
BackgroundInfluentialVisibility of retractions: a cross-sectional one-year study
Decullier, E., Huot, L., Samson, G., & Maisonneuve, H. (2013) · BMC Research Notes · doi:10.1186/1756-0500-6-238 · Cited by 69 · cited 1×
“%) retracted papers in one sample (Decullier, Huot et al., 2013) and 15/123 (12.2”
BackgroundInfluentialSuelzer, E. M., Deal, J., Hanus, K. L., Ruggeri, B., Sieracki, R., & Witkowski, E. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.15552 · Cited by 69 · cited 8×
“Studies that more deeply analyzed citation contexts (e.g., beyond the tone) focused only on a single retracted paper and its citations (Fulton, Coates et al., 2015; Schneider et al., 2020; Suelzer et al., 2019; van der Vet & Nijveen, 2016).”
BackgroundMethodInfluentialDoes retraction after misconduct have an impact on citations? A pre–post study
Candal-Pedreira, C., Ruano-Ravina, A., Fernández, E., Ramos, J., Campos-Varela, I., & Pérez-Ríos, M. (2020) · BMJ Global Health · doi:10.1136/bmjgh-2020-003719 · Cited by 67
Analysis of Citations to Biomedical Articles Affected by Scientific Misconduct
Neale, A. V., Dailey, R. K., & Abrams, J. (2010) · Science and Engineering Ethics · doi:10.1007/s11948-009-9151-4 · Cited by 66 · cited 1×
“…that explored full-text articles mainly focused on the acknowl-edgment of retraction shown in citation contexts (Budd, Coble, & Abritis, 2016; Neale, Dailey, & Abrams, 2010; Piller, 2021; Schneider et al., 2020; Suelzer et al., 2019) and the tone (positive, negative, and neutral) of each…”
BackgroundCitation function, polarity and influence classification
Hernández-Álvarez, M., Gomez, J. M., & Martínez-Barco, P. (2017) · Natural Language Engineering · doi:10.1017/s1351324916000346 · Cited by 65
AMA Manual of Style Committee (2020) · doi:10.1093/jama/9780190246556.001.0001 · Cited by 65
Automatically classifying the role of citations in biomedical articles.
Agarwal, S., Choubey, L., & Yu, H. Q. (2010) · PubMed · Cited by 51
Bordignon, F. (2020) · Scientometrics · doi:10.1007/s11192-020-03536-z · Cited by 51 · cited 2×
“Bordignon ’ s (2020) finding echoes ours in that retracted papers are not cited differently before and after the retraction.”
Backgroundvan der Vet, P. E., & Nijveen, H. (2016) · Research Integrity and Peer Review · doi:10.1186/s41073-016-0008-5 · Cited by 50
A Multidimensional Investigation of the Effects of Publication Retraction on Scholarly Impact
Shuai, X., Rollins, J., Moulinier, I., Custis, T., Edmunds, M., & Schilder, F. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23826 · Cited by 48
Publication lag in biomedical journals varies due to the periodical's publishing model
Dong, P., Loh, M. C. S., & Mondry, A. (2006) · Scientometrics · doi:10.1007/s11192-006-0148-3 · Cited by 46
Journal Retractions in Oncology: A Bibliometric Study
Pantziarka, P., & Meheus, L. (2019) · Future Oncology · doi:10.2217/fon-2019-0233 · Cited by 45
Towards Fine-grained Citation Function Classification
Li, X., He, Y., Meyers, A., & Grishman, R. (2013) · International conference Recent advances in natural language processing · Cited by 44
The continued citation of retracted publications in dentistry
Theis-Mahon, N. R., & Bakker, C. (2020) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2020.824 · Cited by 43
Post retraction citations among manuscripts reporting a radiology-imaging diagnostic method
Bolboacă, S. D., Buhai, D.-V., Aluaș, M., & Bulboacă, A. E. (2019) · PLOS ONE · doi:10.1371/journal.pone.0217918 · Cited by 42
Continued citation of retracted radiation oncology literature—Do we have a problem?
Hamilton, D. G. (2019) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2018.11.014 · Cited by 42 · cited 4×
“Different approaches have been used to identify post-retraction citations, such as after one month (Mott et al., 2019), after six months (Hamilton, 2019), and after the calendar year of retraction (Budd et al., 2011, 2016; Madhugiri et al., 2021; Pfeifer & Snodgrass, 1990).”
MethodResultInfluentialSebo, P., Fournier, J., Ragot, C., Gorioux, P.-H., Herrmann, F. R., & Maisonneuve, H. (2019) · Scientometrics · doi:10.1007/s11192-019-03061-8 · Cited by 41
Citation of Retracted Articles in Engineering: A Study of the Web of Science Database
Rubbo, P., Pìlatti, L. A., & Picinin, C. T. (2018) · Ethics & Behavior · doi:10.1080/10508422.2018.1559064 · Cited by 39
An investigation of retracted articles in the biomedical literature
Budd, J. M., Coble, Z., & Abritis, A. (2016) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.2016.14505301055 · Cited by 36
Mott, A., Fairhurst, C., & Torgerson, D. (2019) · Journal of Health Services Research & Policy · doi:10.1177/1355819618797965 · Cited by 36 · cited 5×
“Mott et al. (2019) studied citations to 387 retracted randomized controlled trials (RCTs).”
BackgroundMethodResultInfluentialOn the citation lifecycle of papers with delayed recognition
Lachance, C., & Larivière, V. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.08.002 · Cited by 36
Challenges in identifying the retracted status of an article
Suelzer, E. M., Deal, J., Hanus, K., Ruggeri, B. E., & Witkowski, E. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.15648 · Cited by 35
Disgraced COVID-19 studies are still routinely cited
Piller, C. (2021) · Science · doi:10.1126/science.371.6527.331 · Cited by 30
Icmje (2018) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 29
Correcting the literature: Improvement trends seen in contents of retraction notices
Deculllier, E., & Maisonneuve, H. (2018) · BMC Research Notes · doi:10.1186/s13104-018-3576-2 · Cited by 27
The unfortunately long life of some retracted biomedical research publications
Hagberg, J. M. (2020) · Journal of Applied Physiology · doi:10.1152/japplphysiol.00003.2020 · Cited by 25
Systematic examination of pre‐ and post‐retraction citations
Dinh, L., Sarol, J., Cheng, Y., Hsiao, T., Parulian, N., & Schneider, J. (2019) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.35 · Cited by 23
Self-citationDal-Ré, R., & Ayuso, C. (2021) · Accountability in Research · doi:10.1080/08989621.2020.1835479 · Cited by 21 · cited 2×
“However, many citation studies focused on a particular Q1 field (e.g., re-tracted articles in genetics (Dal-Ré & Ayuso, 2020), in dentistry (Theis-Mahon & Bakker, 2020), and in engineering (Rubbo, Pilatti, & Picinin, 2019)).”
BackgroundResultRetractions and post-retraction citations in the COVID-19 infodemic
Santos-d’Amorim, K., de Melo, R. R., & dos Santos, R. N. M. (2021) · Liinc em Revista · doi:10.18617/liinc.v17i1.5593 · Cited by 20
An analysis of retractions in neurosurgery and allied clinical and basic science specialties
Madhugiri, V. S., Nagella, A. B., & Uppar, A. M. (2021) · Acta Neurochirurgica · doi:10.1007/s00701-020-04615-z · Cited by 19
Citing Medicine: The NLM Style Guide for Authors, Editors, and Publishers
Patrias, K., & Wendling, D. (2018) · Morphologia · doi:10.26641/1997-9665.2018.4.122-129 · Cited by 18
OpCitance: Citation contexts identified from the PubMed Central open access articles
Hsiao, T., & Torvik, V. I. (2023) · Scientific Data · doi:10.1038/s41597-023-02134-x · Cited by 16
Self-citationWhitehorn, A., Coates, A. M., Williams, M., Howe, P. R. C., & Hill, A. M. (2015) · Publications · doi:10.3390/publications3010017 · Cited by 13
How many retracted articles indexed in KoreaMed were cited 1 year after retraction notification
Kim, S. Y., Yi, H. J., Cho, H.-M., & Huh, S. (2019) · Science Editing · doi:10.6087/kcse.172 · Cited by 11
A citation context analysis of retracted scientific articles
Wright, N. D. (1991) · Medical Entomology and Zoology · Cited by 9
Exploring the Influence of Publication Retraction on Citations in Psychology Science
Yang, S., Qi, F., & Diao, H. (2020) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3383583.3398583 · Cited by 6
Hsiao, T., Fu, Y., & Schneider, J. (2020) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.315 · Cited by 5
Self-citationTechnology footprints in scientific discovery: Citation contexts of paper‐to‐patent citations
Hsiao, T., & Torvik, V. I. (2020) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.337 · Cited by 2
Self-citation(2021) · doi:10.13012/b2idb-8255619_v2 · Cited by 1
Self-citationiCite Database Snapshot 2020-08
ICite, Hutchins, B. I., & Santangelo, G. M. (2020) · Figshare · doi:10.35092/yhjc.12957656 · Cited by 1
Kochan, C. (1992) · doi:10.1002/(sici)1097-4571(199208)43:7<488::aid-asi3>3.0.co;2-7
iCite Database Snapshot 2020-08
Hutchins, B. I., & Santangelo, G. (2020) · The NIH Figshare Archive · doi:10.35092/yhjc.12957656.v1
Research works printed but not linked: 8
- American Psychological Association, 2020, Publication manual of the American Psychological Association: The official guide to APA style
- Budd, 2011, Retracted publications in biomedicine: Cause for concern, 390
- Garfield, 1965, Can citation indexing be automated?, Statistical Association Methods for Mechanized Documentation, Symposium Proceedings, 269, 189
- Nane, 2015, Time to first citation estimation in the presence of additional information, 249
- PubMed Central, .., Open Access Subset
- Retraction Watch database user guide, 2018, Retraction Watch
- scite, 2020, Reference Check: An easy way to check the reliability of your references
- Zotero, 2019, Retracted item notifications with Retraction Watch integration
Cited by
Ioannidis, J. P. A., Pezzullo, A. M., Cristiano, A., Boccia, S., & Baas, J. (2025) · PLoS Biology · doi:10.1371/journal.pbio.3002999 · Cited by 52
Ortega, J. L., & Delgado‐Quirós, L. (2024) · Scientometrics · doi:10.1007/s11192-024-05034-y · Cited by 55
Reducing the Inadvertent Spread of Retracted Science: recommendations from the RISRS report
Schneider, J., Woods, N. D., Proescholdt, R., the RISRS Team, Burns, H., Howell, K., Campbell, M. T., Hsiao, T., Yip, Y. Y. ‘., Fu, Y., et al. (2022) · Research Integrity and Peer Review · doi:10.1186/s41073-022-00125-x · Cited by 54
Self-citationDe Cassai, A., Geraldini, F., De Pinto, S., Carbonari, I., Cascella, M., Boscolo, A., Sella, N., Monteleone, F., Cavaliere, F., Munari, M., et al. (2022) · Anesthesiology · doi:10.1097/aln.0000000000004302 · Cited by 34
Updated science-wide author databases of standardized citation indicators including retraction dataPreprint
Ioannidis, J. P. A., Pezzullo, A. M., Cristiano, A., Boccia, S., & Baas, J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.09.16.613258 · Cited by 31
Metadata
- Authors
- Hsiao, T.-K., & Schneider, J.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Schneider, Jodi
- Venue
- Quantitative Science Studies
- Publisher
- MIT Press
- Published
- 2021
- Volume
- 2
- Issue
- 4
- Pages
- 1144-1169
- ISSN
- 2641-3337 (ISSN-L)
- PMC
- PMC9520488
- OpenAlex
- W4200394820
- Cited by
- 98
- References
- 73
- Citations by year
- 2026: 27 · 2025: 22 · 2024: 17 · 2023: 19 · 2022: 9 · 2021: 4
- FWCI
- 7.10
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Quantitative Science Studies (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- U.S. National Library of Medicine (R01LM010817)
- Alfred P. Sloan Foundation (G-2020-12623)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- retracted article, citation practices, citation context, biomedical literature, research integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.