Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers
Open accessThe DOI matches its Crossref record; 46 of the 47 references it cites were found in a registry or an index.
Abstract
Methodological mistakes, data errors, and scientific misconduct are considered prevalent problems in science that are often difficult to detect. In this study, we explore the potential of using data from Twitter for discovering problems with publications. In this case study, we analyzed tweet texts of three retracted publications about COVID-19 (Coronavirus disease 2019)/SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) and their retraction notices. We did not find early warning signs in tweet texts regarding one publication, but we did find tweets that casted doubt on the validity of the two other publications shortly after their publication date. An extension of our current work might lead to an early warning system that makes the scientific community aware of problems with certain publications. Other sources, such as blogs or post-publication peer-review sites, could be included in such an early warning system. The methodology proposed in this case study should be validated using larger publication sets that also include a control group, i.e., publications that were not retracted.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our finding validates at population-wide scale a recent case study of three retracted COVID-19 papers (40) in suggesting that collective attention on Twitter includes meaningful discussions about science and may contain useful early signals related to problematic papers that could eventually contribute to their investigation.”
From Dynamics of cross-platform attention to retracted papers (2022) · cites this paper as [40]
In the citing paper
“A recent case study suggested that Twitter attention received after publication could be indicative of factual information that should be investigated (40).”
From Dynamics of cross-platform attention to retracted papers (2022) · cites this paper as [40]
In the citing paper
“As suggested in a recent case study (40), knowing how retracted papers are mentioned may uncover users who are improving science-related discussions on Twitter by identifying papers that require a closer examination.”
From Dynamics of cross-platform attention to retracted papers (2022) · cites this paper as [40]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
6 retracted
Mehra, M. R., Desai, S. S., Ruschitzka, F., & Patel, A. N. (2020) · The Lancet · doi:10.1016/s0140-6736(20)31180-6 · Cited by 1,200 · cited 14×
“Figure 9 shows the tweets per day that mention either the paper Mehra et al., (2020a) or its retraction Mehra et al. (2020b). The publication dates of the paper and the retraction notice are marked as gray vertical lines. The retraction notice has been tweeted a day before its official publication date. On these 2 day…”
BackgroundInfluentialRetractedMehra, M. R., Ruschitzka, F., & Patel, A. N. (2020) · The Lancet · doi:10.1016/s0140-6736(20)31324-6 · Cited by 553 · cited 4×
“Figure 9 shows the tweets per day that mention either the paper Mehra et al., (2020a) or its retraction Mehra et al. (2020b).”
BackgroundInfluentialRetractedBae, S., Kim, M.-C., Kim, J. Y., Cha, H.-H., Lim, J. S., Jung, J., Kim, M.-J., Oh, D. K., Lee, M.-K., Choi, S.-H., et al. (2020) · Annals of Internal Medicine · doi:10.7326/m20-1342 · Cited by 193 · cited 1×
“The study was published on 6 April 2020 and retracted on 2 June 2020 because they “had not fully recognized the concept of limit of detection (LOD) of the in-house reverse transcriptase polymerase chain reaction used in the study” (Bae et al., 2020b).”
RetractedWang, X., Xu, W., Hu, G., Xia, S., Sun, Z., Liu, Z., Xie, Y., Zhang, R., Jiang, S., & Lu, L. (2020) · Cellular & Molecular Immunology · doi:10.1038/s41423-020-0498-4 · Cited by 175 · cited 5×
“8 Word cloud from tweet texts based on the tweets mentioning Wang et al., (2020a) or Wang et al., (2020b) since publication date of the retraction”
BackgroundMethodResultInfluentialRetractedWang, X., Xu, W., Hu, G., Xia, S., Sun, Z., Liu, Z., Xie, Y., Zhang, R., Jiang, S., & Lu, L. (2020) · Cellular & Molecular Immunology · doi:10.1038/s41423-020-0424-9 · Cited by 161 · cited 1×
“This paper has been published on 7 April 2020, and retracted on 10 July 2020 (Wang et al., 2020b) because “[a]fter the publication of this article, it came to the authors attention that in order to support the conclusions of the study, the authors should have used primary T cells instead of T-cell…”
ResultRetractedRetracted publications and their citation in dental literature: A systematic review
Rapani, A., Lombardi, T., Berton, F., Lupo, V. D., Lenarda, R. D., & Stacchi, C. (2020) · Clinical and Experimental Dental Research · doi:10.1002/cre2.292 · Cited by 70 · cited 1×
“About half of the retracted publications contained at least one co-author from Asia (Rapani et al., 2020).”
BackgroundRetractedWickham, H. (2016) · Use R! · doi:10.1007/978-3-319-24277-4 · Cited by 33,115
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
tidyverse: Easily Install and Load the 'Tidyverse'
Wickham, H. (2016) · CRAN: Contributed Packages · doi:10.32614/cran.package.tidyverse · Cited by 1,381
Web of Science as a Data Source for Research on Scientific and Scholarly Activity
Birkle, C., Pendlebury, D. A., Schnell, J., & Adams, J. (2020) · Quantitative Science Studies · doi:10.1162/qss_a_00018 · Cited by 1,283 · cited 3×
“The Web of Science (WoS; Birkle et al., 2020) has a document type named “retraction” that has indexed documents with publication years since 1974.”
BackgroundMethod
Research works printed but not linked: 2
- Roe, P., & Lewison, G. (2019). The anatomy of retracted papers in the Web of Science, 1998–2017. Paper presented at the ISSI 2019—17th International Conference of the International Society for Scientometrics and Informetrics, Rome, Italy.
- Toupin, R., Millerand, F., & Larivière, V. (2019). Scholarly communication or public communication of science? Assessing who engage with climate change research on Twitter. Paper presented at the 17th International Conference on Scientometrics and Informetrics (ISSI 2019) with a special STI conference track, Rome, Italy.
Cited by
Dynamics of cross-platform attention to retracted papers
Peng, H., Romero, D. M., & Horvát, E.-Á. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2119086119 · Cited by 57
Schneider, J., Woods, N. D., Proescholdt, R., Fu, Y., & The RISRS Team (2021) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/ms579 · Cited by 18
Amiri, M., Yaghtin, M., & Sotudeh, H. (2023) · Scientometrics · doi:10.1007/s11192-023-04871-7 · Cited by 11
“Moreover, tweets are found to contain retraction-related keywords (Abhari et al., 2022), and some hints that raise doubts about retracted articles in their pre-retraction phase (Haunschild & Bornmann, 2021).”
A comparative study on characteristics of retracted publications across different open access levels
Zheng, E., & Fu, H. (2024) · Journal of Data and Information Science · doi:10.2478/jdis-2024-0010 · Cited by 11
“Readers could use PubPeer or other online platforms to help build an early warning system that made the scientific community aware of the problems in publications (Haunschild & Bornmann, 2021).”
Unraveling retraction dynamics in COVID-19 research: Patterns, reasons, and implications
Khurana, P., Sharma, K., & Uddin, Z. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2379906 · Cited by 11
“The scholarly literature can be corrected through retraction, which is gaining more attention as a means of ensuring that publications adhere to ethical standards and research rules (Haunschild and Bornmann, 2021; Eldakar and Shehata, 2023).”
Metadata
- Authors
- Haunschild, R., & Bornmann, L.
- Affiliations
- Max Planck Institute for Solid State Research (DE)
- Max Planck Society (DE)
- Corresponding author
- Haunschild, Robin
- Venue
- Scientometrics
- Published
- 2021-04-26
- Volume
- 126
- Issue
- 6
- Pages
- 5181-5199
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- PMC
- PMC8072087
- OpenAlex
- W3158277584
- Cited by
- 29
- References
- 47
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 2 · 2023: 8 · 2022: 4 · 2021: 4
- FWCI
- 3.84
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientometrics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Max Planck Institute for Solid State Research
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- Twitter, retracted article, COVID-19 research, SARS-CoV-2, research misconduct, early warning systems, tweet analysis, post-publication peer review, research integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers. https://app.cochat.ai/papers/10.1007/s11192-021-03962-7
- The paper
- Haunschild, R., & Bornmann, L. (2021). Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers. Scientometrics. https://doi.org/10.1007/s11192-021-03962-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.