CoChat

Media and social media attention to retracted articles according to Altmetric

Open access

The DOI matches its Crossref record; 31 of the 34 references it cites were found in a registry or an index; 8 of the 37 sources its claims cite were checked for support.

Abstract

The number of retracted articles has grown fast. However, the extent to which researchers and the public are made adequately aware of these retractions and how the media and social media respond to them remains unknown. Here, we aimed to evaluate the media and social media attention received by retracted articles and assess also the attention they receive post-retraction versus pre-retraction. We downloaded all records of retracted literature maintained by the Retraction Watch Database and originally published between January 1, 2010 to December 31, 2015. For all 3,008 retracted articles with a separate DOI for the original and its retraction, we downloaded the respective Altmetric Attention Score (AAS) (from Altmetric) and citation count (from Crossref), for the original article and its retraction notice on June 6, 2018. We also compared the AAS of a random sample of 572 retracted full journal articles available on PubMed to that of unretracted full articles matched from the same issue and journal. 1,687 (56.1%) of retracted research articles received some amount of Altmetric attention, and 165 (5.5%) were even considered popular (AAS>20). 31 (1.0%) of 2,953 with a record on Crossref received >100 citations by June 6, 2018. Popular articles received substantially more attention than their retraction, even after adjusting for attention received post-retraction (Median difference, 29; 95% CI, 17–61). Unreliable results were the most frequent reason for retraction of popular articles (32; 19%), while fake peer review was the most common reason (421; 15%) for the retraction of other articles. In comparison to matched articles, retracted articles tended to receive more Altmetric attention (23/31 matched groups; P-value, 0.01), even after adjusting for attention received post-retraction. Our findings reveal that retracted articles may receive high attention from media and social media and that for popular articles, pre-retraction attention far outweighs post-retraction attention.

Overview

CoChat Summary

TL;DR

Retracted articles often receive media and social media attention, and popular retracted articles receive substantially more attention than their retraction notices even after accounting for post-retraction attention. Retracted articles also tended to receive more Altmetric attention than matched unretracted articles from the same journal issues.

Key findings

  • Among 3,097 retracted research articles with Altmetric data, 1,733 (56%) received any media/social media attention (AAS>0) and 168 (5.4%) were “popular” (AAS>20); 28/3,008 (0.9%) original articles with Crossref data had ≥100 citations, while no retraction notices did.
  • Popular original articles received substantially more attention than their retraction notices (median difference 30; IQR 14–91; P<10−16); after restricting to 279 articles retracted within a year and adjusting for post-retraction AAS, the median difference for popular articles was 31 (IQR 21–82).
  • Retracted articles tended to receive more Altmetric attention than matched unretracted articles: in 47 matched groups of articles retracted within a year, the median retracted article had higher AAS on 23 occasions versus 8 for unretracted articles after excluding the last year (relative risk 2.88; 95% CI 1.24–7.40; P=0.01).
  • Common retraction reasons differed: unreliable results was most frequent for popular articles (33/168; 20%), while fake peer review was most common for other articles (487/2,923; 17%); for highly cited articles (>100 citations), manipulation/duplication of images was most common (8/28; 29%).

Methods

  • Retrospective cohort using Retraction Watch Database records originally published 2010–2015; 4,603 eligible research articles were identified, with Altmetric Attention Score and Crossref citation counts retrieved on June 6, 2018 for original articles and retraction notices.
  • Case-control comparison: a random sample of 572 retracted full journal articles was matched to up to 5 unretracted articles from the same issue and journal, creating 450 distinct groups initially and 47 groups for articles retracted within a year of Altmetric data retrieval.
  • Analyses used descriptive statistics; paired comparisons with Wilcoxon signed-rank and binomial sign tests; matched comparisons with nonparametric bootstrap percentile confidence intervals; all processing in R.

Limitations

  • Stated: the retrospective design lacked longitudinal data to control for possible effects of retraction over time; the analysis was descriptive and exploratory; popular-article analyses included relatively small samples, especially when adjusting for retraction effects over the last year.
  • Evident: Altmetric and citation data were from June 6, 2018 and based on a beta Retraction Watch version, while some descriptive results use an updated database, producing count mismatches; AAS captures media/social media attention only, and missing AAS/citation data were excluded without imputation.
  • Evident: tweet negativity/non-negativity was judged by the authors’ impression, and only a subset of tweets was openly available for the sensitivity analysis.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0248625

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig 1 Proportion of retracted research articles by country. Proportion of retracted research in relation to all peer-reviewed documents published in 2010–2015 for countries with >500 peer-reviewed documents within those 6 years, as indicated by the National Science Foundation (see Materials and Methods). This varies substantially by country and by continent. The continent with most retractions is Asia and the continent with least retractions is Europe. The country with the highest proportion of retractions was the Republic of Congo (3/922; 0.3%) and the country with the least proportion was Hungary (3/55,609, 0.005%). Note that NSF counts of total peer-reviewed documents do not include letters, which represent 21/4,603 (0.5%) of our research articles as per our eligibility criteria. Grey signifies no data for those countries.
Fig 1 Proportion of retracted research articles by country. Proportion of retracted research in relation to all peer-reviewed documents published in 2010–2015 for countries with >500 peer-reviewed documents within those 6 years, as indicated by the National Science Foundation (see Materials and Methods). This varies substantially by country and by continent. The continent with most retractions is Asia and the continent with least retractions is Europe. The country with the highest proportion of retractions was the Republic of Congo (3/922; 0.3%) and the country with the least proportion was Hungary (3/55,609, 0.005%). Note that NSF counts of total peer-reviewed documents do not include letters, which represent 21/4,603 (0.5%) of our research articles as per our eligibility criteria. Grey signifies no data for those countries.CC BY · PubMed Central
Fig 2 AAS and citation count across original articles and retraction notices. The distribution of AAS between 3,097 original articles and their retraction notices is fairly similar. However, a small number of original articles tend to receive more extreme AAS scores (5 articles with very high original AAS (Range, 1019–3166) are not shown for clarity). Unlike AAS, the citation count in 3,008 original articles is far greater than that in their retraction notices.
Fig 2 AAS and citation count across original articles and retraction notices. The distribution of AAS between 3,097 original articles and their retraction notices is fairly similar. However, a small number of original articles tend to receive more extreme AAS scores (5 articles with very high original AAS (Range, 1019–3166) are not shown for clarity). Unlike AAS, the citation count in 3,008 original articles is far greater than that in their retraction notices.CC BY · PubMed Central
Fig 3 Features across popularity and impact. (A) The distribution of Country, Journal, Publisher and Reason for retraction is different across levels of popularity in 3,097 original articles. Popular retracted articles often came from the USA, were published in journals such as Nature and Science and were mostly retracted because of unreliable results. On the contrary, other retracted articles often came from China, were published in journals such as Tumor Biol and J Biol Chem and were primarily retracted because of fake peer review. (B) The distribution of Country, Journal, Publisher and Reason for retraction across levels of impact for 3,008 original articles had a similar pattern as the pattern seen across levels of Altmetric attention. However, the commonest reason for retraction in highly cited research (>100 citations) was duplication or manipulation of images.
Fig 3 Features across popularity and impact. (A) The distribution of Country, Journal, Publisher and Reason for retraction is different across levels of popularity in 3,097 original articles. Popular retracted articles often came from the USA, were published in journals such as Nature and Science and were mostly retracted because of unreliable results. On the contrary, other retracted articles often came from China, were published in journals such as Tumor Biol and J Biol Chem and were primarily retracted because of fake peer review. (B) The distribution of Country, Journal, Publisher and Reason for retraction across levels of impact for 3,008 original articles had a similar pattern as the pattern seen across levels of Altmetric attention. However, the commonest reason for retraction in highly cited research (>100 citations) was duplication or manipulation of images.CC BY · PubMed Central
Fig 4 Change in attention by popularity. All points in green represent an original article that received more media and social media attention than its retraction notice and all points in red represent the opposite; points in grey represent no difference between the two. The large point and solid line in grey represent the median and its interquartile range. The difference is rather balanced for 2,923 articles that are not popular—the extreme negative values at -395 came from a single retraction in Tumor Biol, which retracted 103 unique original articles. In 168 popular articles, the difference is skewed to the right such that most popular articles did not have an equally popular retraction.
Fig 4 Change in attention by popularity. All points in green represent an original article that received more media and social media attention than its retraction notice and all points in red represent the opposite; points in grey represent no difference between the two. The large point and solid line in grey represent the median and its interquartile range. The difference is rather balanced for 2,923 articles that are not popular—the extreme negative values at -395 came from a single retraction in Tumor Biol, which retracted 103 unique original articles. In 168 popular articles, the difference is skewed to the right such that most popular articles did not have an equally popular retraction.CC BY · PubMed Central
Fig 5 Altmetric Attention Score (AAS) over time. (A) Cumulative AAS for 279 articles retracted within a year of Altmetric data retrieval (from June 6, 2017 to June 6, 2018). The horizontal axis denotes how many months it has been since the article was retracted from the day of Altmetric data retrieval and the vertical axis the amount of AAS gained before and after retraction. It illustrates that, over the past year, most gains in AAS occurred after retraction. (B) Even though the median gain is zero in both 2,812 recently and 279 not recently retracted articles, proportionally many more recent articles experienced a gain, as denoted by the large interquartile range and the mean (grey dot denotes mean and range denotes the bootstrapped 95% CI).
Fig 5 Altmetric Attention Score (AAS) over time. (A) Cumulative AAS for 279 articles retracted within a year of Altmetric data retrieval (from June 6, 2017 to June 6, 2018). The horizontal axis denotes how many months it has been since the article was retracted from the day of Altmetric data retrieval and the vertical axis the amount of AAS gained before and after retraction. It illustrates that, over the past year, most gains in AAS occurred after retraction. (B) Even though the median gain is zero in both 2,812 recently and 279 not recently retracted articles, proportionally many more recent articles experienced a gain, as denoted by the large interquartile range and the mean (grey dot denotes mean and range denotes the bootstrapped 95% CI).CC BY · PubMed Central
Fig 6 Difference in attention within 47 matched groups of articles retracted within a year of Altmetric data extraction. Each point represents the difference in AAS between the median retracted vs. the median unretracted article within matched groups. Green points represent groups in which the original article received more media and social media attention than matched unretracted articles and red points represent the opposite; points in grey represent no difference between the two. The “With last year” column represents all of the Altmetric attention received, whereas the “Without last year” column represents all Altmetric attention minus the last year. The large point and solid line in grey represent the median and its interquartile range. The median difference between retracted and matched unretracted articles is small (Median, 1) when including attention received over the last year and 0 otherwise. However, substantially more retracted articles received higher attention than matched unretracted articles, regardless of including attention from last year or not.
Fig 6 Difference in attention within 47 matched groups of articles retracted within a year of Altmetric data extraction. Each point represents the difference in AAS between the median retracted vs. the median unretracted article within matched groups. Green points represent groups in which the original article received more media and social media attention than matched unretracted articles and red points represent the opposite; points in grey represent no difference between the two. The “With last year” column represents all of the Altmetric attention received, whereas the “Without last year” column represents all Altmetric attention minus the last year. The large point and solid line in grey represent the median and its interquartile range. The median difference between retracted and matched unretracted articles is small (Median, 1) when including attention received over the last year and 0 otherwise. However, substantially more retracted articles received higher attention than matched unretracted articles, regardless of including attention from last year or not.CC BY · PubMed Central

Verification

Claims the paper makes

6 Valid · 0 Not valid · 22 Not checked · 70 Own results and interpretations

45 more claims

Create a free account to see all 98 claims and to verify the 29 references still open.

See all claims — free account
  • It may state two separate claims as one. 36
  • Probably not a substantial claim. 26

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 5

Citations

Cites

4 retracted

Research works printed but not linked: 8

  1. A Gawrylewski, 2007, Glossary of retractions., The Scientist
  2. I Oransky, 2020, The Retraction Watch Database., Retraction Watch
  3. I Oransky, 2010, Why write a blog about retractions?, Retraction Watch
  4. I Oransky, 2019, Top 10 most highly cited retracted papers., Retraction Watch
  5. I Oransky, 2016, How to better flag retractions? Here’s what PubMed is trying., Retraction Watch
  6. D Stillman, 2019, Retracted item notifications with Retraction Watch integration., Zotero
  7. A Marcus, 2014, Retraction Watch is growing, thanks to a $400,000 grant from the MacArthur Foundation., Retraction Watch
  8. A Marcus, 2015, New $300,000 grant marks the fifth anniversary of Retraction Watch., Retraction Watch

Cited by

See all 55

Metadata

Authors
Serghiou, S., Marton, R. M., & Ioannidis, J. P. A.
Affiliations
Stanford Medicine (US)
Corresponding author
Ioannidis, John P. A.
Venue
PLOS ONE
Published
2021-05-12
Volume
16
Issue
5
Pages
e0248625
ISSN
1932-6203 (ISSN-L)
OpenAlex
W3163888857
Cited by
55
References
34
Citations by year
2026: 10 · 2025: 9 · 2024: 9 · 2023: 6 · 2022: 17 · 2021: 3
FWCI
7.48
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • OSF Preprints (OSF Preprints) (submittedVersion)
Funders
  • Laura and John Arnold Foundation
  • Department of Epidemiology and Population Health, Stanford University
  • Stanford Data Science Initiative
Topics
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Social Sciences › Education › Academic integrity and plagiarism
Keywords
retracted article, media attention, social media attention, Altmetric Attention Score, fake peer review, Retraction Watch Database
MeSH terms
Humans, Peer Review, Publications, Bibliometrics, Social Media

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.