CoChat

Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers

Open access
Haunschild, Robin, Bornmann, Lutz · 2021 · Scientometrics

The DOI matches its Crossref record; 46 of the 47 references it cites were found in a registry or an index.

Abstract

Methodological mistakes, data errors, and scientific misconduct are considered prevalent problems in science that are often difficult to detect. In this study, we explore the potential of using data from Twitter for discovering problems with publications. In this case study, we analyzed tweet texts of three retracted publications about COVID-19 (Coronavirus disease 2019)/SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) and their retraction notices. We did not find early warning signs in tweet texts regarding one publication, but we did find tweets that casted doubt on the validity of the two other publications shortly after their publication date. An extension of our current work might lead to an early warning system that makes the scientific community aware of problems with certain publications. Other sources, such as blogs or post-publication peer-review sites, could be included in such an early warning system. The methodology proposed in this case study should be validated using larger publication sets that also include a control group, i.e., publications that were not retracted.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11192-021-03962-7

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Number (left panel) and percentage (right panel) of tweets mentioning any of the three publications or their retraction notices
Fig. 1 Number (left panel) and percentage (right panel) of tweets mentioning any of the three publications or their retraction noticesCC BY · PubMed Central
Fig. 2 Twitter classification of users mentioning any of the three publications or their retraction notices in tweets
Fig. 2 Twitter classification of users mentioning any of the three publications or their retraction notices in tweetsCC BY · PubMed Central
Fig. 3 Tweets per day that mentioned either the paper Bae et al., (2020a) or its retraction Bae et al., (2020b). The publication dates of the paper and the retraction notice are marked as gray vertical lines
Fig. 3 Tweets per day that mentioned either the paper Bae et al., (2020a) or its retraction Bae et al., (2020b). The publication dates of the paper and the retraction notice are marked as gray vertical linesCC BY · PubMed Central
Fig. 4 Word cloud from tweet texts based on the tweets mentioning Bae et al., (2020a) before publication date of the retraction
Fig. 4 Word cloud from tweet texts based on the tweets mentioning Bae et al., (2020a) before publication date of the retractionCC BY · PubMed Central
Fig. 5 Word cloud from tweet texts based on the tweets mentioning Bae et al., (2020a) or Bae et al., (2020b) since publication date of the retraction
Fig. 5 Word cloud from tweet texts based on the tweets mentioning Bae et al., (2020a) or Bae et al., (2020b) since publication date of the retractionCC BY · PubMed Central
Fig. 6 Tweets per day that mentioned either the paper Wang et al., (2020a) or its retraction Wang et al., (2020b). The publication dates of the paper and the retraction notice are marked as gray vertical lines
Fig. 6 Tweets per day that mentioned either the paper Wang et al., (2020a) or its retraction Wang et al., (2020b). The publication dates of the paper and the retraction notice are marked as gray vertical linesCC BY · PubMed Central
Fig. 7 Word cloud from tweet texts based on the tweets mentioning Wang et al., (2020a) before publication date of the retraction
Fig. 7 Word cloud from tweet texts based on the tweets mentioning Wang et al., (2020a) before publication date of the retractionCC BY · PubMed Central
Fig. 8 Word cloud from tweet texts based on the tweets mentioning Wang et al., (2020a) or Wang et al., (2020b) since publication date of the retraction
Fig. 8 Word cloud from tweet texts based on the tweets mentioning Wang et al., (2020a) or Wang et al., (2020b) since publication date of the retractionCC BY · PubMed Central
Fig. 9 Tweets per day that mention either the paper Mehra et al., (2020a) or its retraction Mehra et al., (2020b) . The publication dates of the paper and the retraction notice are marked as gray vertical lines
Fig. 9 Tweets per day that mention either the paper Mehra et al., (2020a) or its retraction Mehra et al., (2020b) . The publication dates of the paper and the retraction notice are marked as gray vertical linesCC BY · PubMed Central
Fig. 10 Word cloud from tweet texts based on the tweets mentioning Mehra et al., (2020a), before the day before the publication date of the retraction
Fig. 10 Word cloud from tweet texts based on the tweets mentioning Mehra et al., (2020a), before the day before the publication date of the retractionCC BY · PubMed Central
Fig. 11 Word cloud from tweet texts based on the tweets mentioning Mehra et al., (2020a), or Mehra et al., (2020b) since the day before the publication date of the retraction
Fig. 11 Word cloud from tweet texts based on the tweets mentioning Mehra et al., (2020a), or Mehra et al., (2020b) since the day before the publication date of the retractionCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

6 retracted

Show all 46 references

Research works printed but not linked: 2

  1. Roe, P., & Lewison, G. (2019). The anatomy of retracted papers in the Web of Science, 1998–2017. Paper presented at the ISSI 2019—17th International Conference of the International Society for Scientometrics and Informetrics, Rome, Italy.
  2. Toupin, R., Millerand, F., & Larivière, V. (2019). Scholarly communication or public communication of science? Assessing who engage with climate change research on Twitter. Paper presented at the 17th International Conference on Scientometrics and Informetrics (ISSI 2019) with a special STI conference track, Rome, Italy.

Cited by

See all 29

Metadata

Authors
Haunschild, R., & Bornmann, L.
Affiliations
  • Max Planck Institute for Solid State Research (DE)
  • Max Planck Society (DE)
Corresponding author
Haunschild, Robin
Venue
Scientometrics
Published
2021-04-26
Volume
126
Issue
6
Pages
5181-5199
ISSN
  • 0138-9130 (ISSN-L)
  • 1588-2861
OpenAlex
W3158277584
Cited by
29
References
47
Citations by year
2026: 4 · 2025: 7 · 2024: 2 · 2023: 8 · 2022: 4 · 2021: 4
FWCI
3.84
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientometrics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
Max Planck Institute for Solid State Research
Topics
  • Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Social Sciences › Education › Academic integrity and plagiarism
Keywords
Twitter, retracted article, COVID-19 research, SARS-CoV-2, research misconduct, early warning systems, tweet analysis, post-publication peer review, research integrity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers. https://app.cochat.ai/papers/10.1007/s11192-021-03962-7
The paper
Haunschild, R., & Bornmann, L. (2021). Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers. Scientometrics. https://doi.org/10.1007/s11192-021-03962-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.