Assessing the impact of retraction on the citation of randomized controlled trial reports: an interrupted time-series analysis
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Objectives: To assess the impact of retraction on the citation of randomized controlled trials. Methods: We used an interrupted time-series with matched controls. PubMed, CINHAL, Google and the Retraction Watch Database were searched. We identified retracted publications reporting the results of randomized controlled trials involving human participants with two years of available data before and after retraction. We obtained monthly citation counts across all articles for the 24 months before and after retraction, from Web of Science. We used a Poisson segmented regression to detect changes in the level and trend of citation following retraction. We also undertook a matched control analysis of unretracted randomized controlled trials and a sensitivity analysis to account for cases of large-scale, well-advertised fraud. Results: We identified 387 retracted randomized controlled trial reports, of which 218 (56.3%) were included in the interrupted time-series analysis. A reduction of 22.9% (95% CI 4.0% to 38.2%, p = 0.02) was observed in the number of citations in the month after retraction, and a further reduction of 1.9% (95% CI 0.4% to 3.5%, p = 0.02) per month in the following 24 months, relative to the expected trend. There was no evidence of a statistically significant reduction among the matched controls. Authors with a large number of retractions saw a 48.2% reduction at the time of retraction (95% CI 17.7% to 67.3%, p = 0.01). Other cases had a more gradual reduction with no change at the time of retraction and a 1.8% reduction per month in the following 24 months (95% CI 0.2% to 3.4%, p = 0.03). Conclusions: Retractions of randomized controlled trial reports can be effective in reducing citations. Other factors, such as the scale of the retractions and media attention, may play a role in the effectiveness of the reduction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our findings are consistent with those of others who have investigated the impact of publications affected by research misconduct on subsequent publications and systematic reviews.74 76 80–83”
From An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature (2019) · cites this paper as [83]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“citations to the 64 RCTs decreased 1.8% per month, compared to the trend in matched controls”
From Continued post-retraction citation of a fraudulent clinical trial report, 11 years after it was retracted for falsifying data (2020) · cites this paper as Mott et al. 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They reported that retracted papers that were part of large-scale retractions that received broad media attention had a larger reduction in postretraction citations than other retracted papers (Mott et al., 2019).”
From Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine (2021) · cites this paper as Mott et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Dinh et al. (2019) and Mott et al. (2019) reported that retracted papers’ citation counts dropped after the retraction.”
From Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine (2021) · cites this paper as Mott et al. (2019)
The cited source says what the claim attributes to it.
In the citing paper
“Mott and colleagues found a 46% reduction in citations of randomised clinical trial reports in the first year after retraction,83”
From An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature (2019) · cites this paper as [83]
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Retraction rates are on the rise
Cokol, M., Ozbay, F., & Rodriguez‐Esteban, R. (2008) · The EMBO Reports · doi:10.1038/sj.embor.7401143 · Cited by 149
RetractedSegmented regression analysis of interrupted time series studies in medication use research
Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00430.x · Cited by 3,279
Interrupted time series regression for the evaluation of public health interventions: a tutorial
Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw098 · Cited by 3,049
Misconduct accounts for the majority of retracted scientific publications
Fang, F. C., Steen, R. G., & Casadevall, A. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1212247109 · Cited by 1,232
Time series regression studies in environmental epidemiology
Bhaskaran, K., Gasparrini, A., Hajat, S., Smeeth, L., & Armstrong, B. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dyt092 · Cited by 1,150
Zarychanski, R., Abou-Setta, A. M., Turgeon, A. F., Houston, B. L., McIntyre, L., Marshall, J. C., & Fergusson, D. A. (2013) · JAMA · doi:10.1001/jama.2013.430 · Cited by 649
Relative Citation Impact of Various Study Designs in the Health Sciences
Patsopoulos, N. A. (2005) · JAMA · doi:10.1001/jama.293.19.2362 · Cited by 498
A Comprehensive Survey of Retracted Articles from the Scholarly Literature
Grieneisen, M. L., & Zhang, M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0044118 · Cited by 355
Zhang, F., Wagner, A. K., & Ross-Degnan, D. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.02.007 · Cited by 291
The incidence and role of negative citations in science
Catalini, C., Lacetera, N., & Oettl, A. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1502280112 · Cited by 189
Research works printed but not linked: 4
- Bernal JL, 2017, Int J Epidemiol, 46, 348
- Tobias A, 1998, Stata Tech Bull, 46, 33
- Budd JM, Coble ZC, Anderson KM. Retracted publications in biomedicine: cause for concern. In: Declaration of interdependence: the proceedings of the ACRL 2011 conference , 30 March–2 April 2011, pp.390–395. Philadelphia, PA: Association of College and Research Libraries.
- Guthrie S, Krapels J, Lichten CA, et al. 100 Metrics to assess and communicate the value of biomedical research . Washington DC, USA: Research Reports, Association of American Medical Colleges, 2016, p.71. · doi:10.7249/rr1606
Cited by
Hsiao, T.-K., & Schneider, J. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00155 · Cited by 98
“Mott et al. (2019) studied citations to 387 retracted randomized controlled trials (RCTs).”
BackgroundMethodResultInfluentialSchneider, J., Ye, D., Hill, A. M., & Whitehorn, A. S. (2020) · Scientometrics · doi:10.1007/s11192-020-03631-1 · Cited by 88
Citation of retracted publications: A challenging problem
Bolland, M. J., Grey, A., & Avenell, A. (2022) · Accountability in Research · doi:10.1080/08989621.2021.1886933 · Cited by 78
“…retracted articles (Pfeifer and Snodgrass 1990; Budd, Sievert, and Schultz 1998; Sox and Rennie 2006; Neale, Dailey, and Abrams 2010; BornemannCimenti, Szilagyi, and Sandner-Kiesling 2016; Bar-Ilan and Halevi 2017; Bolboaca et al. 2019; Mott, Fairhurst, and Torgerson 2019; Dal-Re and Ayuso 2020).”
BackgroundDoes retraction after misconduct have an impact on citations? A pre–post study
Candal-Pedreira, C., Ruano-Ravina, A., Fernández, E., Ramos, J., Campos-Varela, I., & Pérez-Ríos, M. (2020) · BMJ Global Health · doi:10.1136/bmjgh-2020-003719 · Cited by 67
Avenell, A., Stewart, F., Grey, A., Gamble, G., & Bolland, M. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-031909 · Cited by 59
Fanelli, D., Wong, J., & Moher, D. (2022) · Accountability in Research · doi:10.1080/08989621.2021.1947810 · Cited by 49
“Multiple independent studies have documented how the authors of a retracted study suffer a significant cost in terms of citations, productivity and funding (Lu et al. 2013,7; Mott, Fairhurst, and Torgerson 2018).”
BackgroundBordignon, F. (2020) · Scientometrics · doi:10.1007/s11192-020-03536-z · Cited by 51
Kataoka, Y., Banno, M., Tsujimoto, Y., Ariie, T., Taito, S., Suzuki, T., Oide, S., & Furukawa, T. A. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.06.015 · Cited by 44
RetractedXu, C., Fan, S., Tian, Y., Liu, F., Furuya-Kanamori, L., Clark, J., Zhang, C., Li, S., Lin, L., Chu, H., et al. (2025) · BMJ · doi:10.1136/bmj-2024-082068 · Cited by 39
Reducing Health Misinformation in Science: A Call to Arms
Swire‐Thompson, B., & Lazer, D. M. J. (2022) · The Annals of the American Academy of Political and Social Science · doi:10.1177/00027162221087686 · Cited by 49
Avenell, A., Bolland, M. J., Gamble, G. D., & Grey, A. (2024) · Accountability in Research · doi:10.1080/08989621.2022.2082290 · Cited by 26
“Even after the retraction of publications, citations usually continue (Candal-Pedreira et al. 2020; Mott, Fairhurst, and Torgerson 2019), although these later citations may constitute a minority of all citations for the retracted work (Bolland, Grey, and Avenell 2022).”
BackgroundHeibi, I., & Peroni, S. (2021) · Scientometrics · doi:10.1007/s11192-021-04097-5 · Cited by 34
Szilagyi, I.-S., Schittek, G. A., Klivinyi, C., Simonis, H., Ullrich, T., & Bornemann‐Cimenti, H. (2022) · Scientometrics · doi:10.1007/s11192-022-04321-w · Cited by 28
“, an editorial or case report instead of an original article), the next article of the same category as Reuben’s publication was used (Mott et al., 2019).”
BackgroundA quantitative and qualitative open citation analysis of retracted articles in the humanities
Heibi, I., & Peroni, S. (2022) · Quantitative Science Studies · doi:10.1162/qss_a_00222 · Cited by 23
“Other factors might play a crucial role, such as the popularity of and media attention to the retraction case, as has been discussed in the studies by Mott, Fairhurst, and Torgerson (2019) and Bar-Ilan and Halevi (2017).”
BackgroundAn analysis of retracted papers in Computer Science
Shepperd, M., & Yousefi, L. (2023) · PLOS ONE · doi:10.1371/journal.pone.0285383 · Cited by 16
“This analysis of post-retraction citation patterns was followed up by Mott et al. [13] and Schneider et al. [14] both in the field of clinical trials.”
The Citation of Retracted Papers and Impact on the Integrity of the Scientific Biomedical Literature
da Silva, J. A. T., Santos-d’Amorim, K., & Bornemann‐Cimenti, H. (2025) · Learned Publishing · doi:10.1002/leap.1667 · Cited by 21
“The expectation is that citations to retracted literature, and attention to papers retracted due to misconduct, should decrease over time (Mistry, Grey, and Bolland 2019; Mott, Fairhurst, and Torgerson 2019).”
Schneider, J., Woods, N. D., Proescholdt, R., Fu, Y., & The RISRS Team (2021) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/ms579 · Cited by 18
Retracted publications in BRICS countries: an analytical study
Siva, N., & Rajendran, P. (2023) · Scientometrics · doi:10.1007/s11192-023-04856-6 · Cited by 18
“It is reported that citation counts dropped after the retraction (Dinh et al., 2019; Mott et al., 2019).”
BackgroundWhy do some retracted articles continue to get cited?
Schmidt, M. A. (2024) · Scientometrics · doi:10.1007/s11192-024-05147-4 · Cited by 23
“In a study focusing on retracted publications that are based on randomized controlled trials, Mott et al. (2019) estimate that citations decrease by 23% in relation to matched control papers in the first months after retraction.”
Fanelli, D., & Moher, D. (2019) · doi:10.1101/734137 · Cited by 10
“Several studies have documented the fact that retracted articles keep being cited after their retraction, although typically at much lower rates [5,6].”
BackgroundWhy does retraction take so much longer than publication?
Loadsman, J. A. (2018) · Anaesthesia · doi:10.1111/anae.14484 · Cited by 9
Retraction of Scientific Papers: Types of Retraction, Consequences, and Impacts
Candal‐Pedreira, C., Pérez‐Ríos, M., & Ruano‐Raviña, A. (2022) · Springer eBooks · doi:10.1007/978-3-030-99680-2_40 · Cited by 10
Wang, P., & Su, J. (2022) · Journal of Information Science · doi:10.1177/01655515221074329 · Cited by 9
“For RCT papers, retractions were effective in reducing citations [36].”
BackgroundArticle retraction rates in selected MeSH term categories in PubMed published between 2010 and 2020
Alexander, R. L., Peterson, C., Yang, S., & NUGENT, K. M. (2023) · Accountability in Research · doi:10.1080/08989621.2023.2272246 · Cited by 11
“…2017; Cassão et al. 2018; Dal-Ré and Ayuso 2019; Elizabeth and Maria 2016; Furman, Jensen, and Murray 2012; Gaudino et al. 2021; King et al. 2018; Mott, Fairhurst, and Torgerson 2019); one study suggested that the time to retraction has decreased more than 50% since 2002 (49.82 vs 23.82 months,…”
InfluentialLendvai, G. F., & Sasvári, P. (2026) · Journal of Information Science · doi:10.1177/01655515251362383 · Cited by 6
Bhattacharyya, S., Chatterjee, N., & Ramanathan, A. (2025) · International Journal for Educational Integrity · doi:10.1007/s40979-025-00193-8 · Cited by 4
“Though retraction has been introduced as a measure to weed out unreliable and erroneous publications, several studies have reported continued citation of retracted articles as a preliminary source of scientific data even long after their retraction (Shuai, 2017; Mott et al. 2019).”
Hansson, S. O. (2023) · Theoria · doi:10.1111/theo.12510 · Cited by 5
“Research exceptionalism” in the COVID-19 pandemic: an analysis of scientific retractions in Scopus
Rubbo, P., Lievore, C., dos Santos, C. B., Picinin, C. T., Pìlatti, L. A., & Pedroso, B. (2022) · Ethics & Behavior · doi:10.1080/10508422.2022.2080067 · Cited by 6
“…retraction on the academic life and integrity of researchers (Dobránszki & Teixeira da Silva, 2019; Fanelli et al., 2015; Mongeon & Larivière, 2016; Mott et al., 2019), research at World Class Universities (WCU; Lievore et al., 2021), and the impact and cost of withdrawals using meta-analyses…”
BackgroundAnalyzing the Drivers Behind Retractions in Tuberculosis Research
García-Solórzano, F. O., Anticona, S. M. D. L. C., Pezua-Espinoza, M., Jesus, F. A. C., Sanabria-Pinilla, K. D., Veliz, C. C., Huayta-Alarcón, V. A., Mayta‐Tristán, P., & Lecca, L. W. (2025) · Publications · doi:10.3390/publications13010004 · Cited by 3
“One study of randomized trials showed a significant reduction in the number of citations after retraction (Mott et al., 2019).”
Lawson, S., Doulaveris, G., Anderson, K., Bennett, C., & Berghella, V. (2025) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2025.03.047 · Cited by 3
Xu, Q., Lin, Z., Fan, Q., & Zhu, M. (2026) · Scientometrics · doi:10.1007/s11192-026-05551-y · Cited by 2
“…(Bar-Ilan & Halevi, 2017; Hsiao & Schneider, 2021), and this approach aligns with established retraction impact studies (Azoulay et al., 2017; Mott et al., 2019), future adaptations may incorporate citation sentiment or stance analysis to provide finer-grained assessment of how retracted…”
Retracted articles on stem cells are continuously cited in publications and used by ChatGPT
Zhang, X., Gu, T., Loganathan, S., Xie, Y., & Chen, J. (2025) · Scientometrics · doi:10.1007/s11192-025-05484-y · Cited by 1
“Instead, the disparity appears specific to retracted content, suggesting that retraction status for Asian-authored articles may be less consistently recognized or integrated into the model’s knowledge filtering mechanisms (Dal-Ré & Ayuso, 2020; Mott et al., 2019; Shuai et al., 2017).”
RetractedGarcía-Romero, A., lez-Mancebo, S. G. A., Espallardo, C., & Concepción, M. A. (2026) · Scientometrics · doi:10.1007/s11192-026-05699-7 · Cited by 4
“Similarly, studies by Mott et al. (2019) and Neale et al. (2010) confirm that journal prestige influences citation persistence, even for retracted papers.”
Understanding and predicting retractions of published work
Modukuri, S. A., Rajtmajer, S. M., Squicciarini, A., Wu, J. F., & Giles, C. L. (2021) · ODU Digital Commons (Old Dominion University) · Cited by 3
Fabrication, Falsification and Spin
Crombie, I. (2021) · Wiley eBooks · doi:10.1002/9781394322183.ch6
Retracted Research Publications of Assisted Reproduction: A Review
Turki, H. A. A., & Alshamlan, D. Y. (2026) · Journal of Clinical Medical Research · doi:10.46889/jcmr.2026.7108
Studd, H., Avenell, A., Grey, A. B., & Bolland, M. J. (2026) · medRxiv · doi:10.64898/2026.02.25.26346683
Metadata
- Authors
- Mott, A., Fairhurst, C., & Torgerson, D.
- Affiliations
- University of York (GB)
- Corresponding author
- Mott, Andrew
- Venue
- Journal of Health Services Research & Policy
- Publisher
- SAGE Publications
- Published
- 2018-09-24
- Volume
- 24
- Issue
- 1
- Pages
- 44-51
- ISSN
- 1355-8196 (ISSN-L)
- 1758-1060
- OpenAlex
- W2893976406
- Cited by
- 36
- References
- 26
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 1 · 2023: 4 · 2022: 8 · 2021: 7 · 2020: 3 · 2019: 2 · 2018: 1
- FWCI
- 6.48
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Journal of Health Services Research & Policy (publishedVersion)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- retraction, citation impact, randomized controlled trials, citation counts, research integrity, scientific fraud, media attention, Retraction Watch
- MeSH terms
- Humans, Retraction of Publication as Topic, Serial Publications, Poisson Distribution, Randomized Controlled Trials as Topic, Interrupted Time Series Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.