The DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.
TweepFake: About detecting deepfake tweets
Open accessAbstract
The recent advances in language modeling significantly improved the generative capabilities of deep neural models: in 2019 OpenAI released GPT-2, a pre-trained language model that can autonomously generate coherent, non-trivial and human-like text samples. Since then, ever more powerful text generative models have been developed. Adversaries can exploit these tremendous generative capabilities to enhance social bots that will have the ability to write plausible deepfake messages, hoping to contaminate public debate. To prevent this, it is crucial to develop deepfake social media messages detection systems. However, to the best of our knowledge no one has ever addressed the detection of machine-generated texts on social networks like Twitter or Facebook. With the aim of helping the research in this detection field, we collected the first dataset of real deepfake tweets, TweepFake . It is real in the sense that each deepfake tweet was actually posted on Twitter. We collected tweets from a total of 23 bots, imitating 17 human accounts. The bots are based on various generation techniques, i.e., Markov Chains, RNN, RNN+Markov, LSTM, GPT-2. We also randomly selected tweets from the humans imitated by the bots to have an overall balanced dataset of 25,572 tweets (half human and half bots generated). The dataset is publicly available on Kaggle. Lastly, we evaluated 13 deepfake text detection methods (based on various state-of-the-art approaches) to both demonstrate the challenges that Tweepfake poses and create a solid baseline of detection techniques. We hope that TweepFake can offer the opportunity to tackle the deepfake detection on social media messages as well.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly, a text deepfake can be used to spread disinformation alongside manipulated images, amplifying its impact on social media platforms (Fagni et al., 2021).”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Fagni et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, Fagni et al. (2021) highlighted the potential misuse of text-based deepfakes on Twitter but found existing detection methods to be inadequate.”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Fagni et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The results demonstrate TweepFake’s challenges and provide a baseline for future deepfake detection techniques.”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Fagni et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“According to their findings, a wide range of detectors based on ML or DL approaches and transformer-based utilizing transfer learning) had more trouble successfully identifying a deepfake tweet than a tweet that was actually produced by a human.”
From Deepfakes: current and future trends (2024) · cites this paper as Fagni et al. 2021
The cited source says what the claim attributes to it.
In the citing paper
“A noteworthy exception is the work by Fagni et al., who identify 23 self-disclosed bot accounts on Twitter and share the dataset (Fagni et al., 2021).”
From Anatomy of an AI-powered malicious social botnet (2024) · cites this paper as Fagni et al., 2021
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1×
“ed. Table 3. Description of the methods used in the experimentation. Encoding Method name Algorithm BoW+TF IDF log reg bow Logistic regression classifier [56] rand forest bow Random forest classifier [57] svc bow Support vector machine classifier [58] BERT log reg bert Logistic regression classifier rand forest bert R…”
MethodVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“In 2017, the development of the selfattentionmechanism and the [25]’s transformer led to the improvement of the language models.”
BackgroundMethodInfluentialExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 4×
“and perform training on a specific dataset (typically small) for very few epochs [27].”
BackgroundMethodInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 4×
“and perform training on a specific dataset (typically small) for very few epochs [27].”
BackgroundMethodInfluentialLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“Characters char cnn Single CNN network [59] using internal char embeddings representation char gru Single GRU network [60] using internal char embeddings representation char cnngru Combined CNN and GRU networks using internal char embeddings representation”
MethodRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 5×
“However, OpenAI [31] proved it wrong on GPT-2 generated texts: they showed that fine-tuning a RoBERTa-based detector achieved consistently higher accuracy than fine-tuning a GPT-2 based detector with equivalent capacity.”
MethodInfluentialAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Research works printed but not linked: 3
- R Gorwa, 2020, Special Issue:Cleaning Up Social Media, 225
- R Zellers, 2019, Advances in Neural Information Processing Systems, 9054
- Z Yang, 2019, Advances in Neural Information Processing Systems, 5754
Cited by
Combining EfficientNet and Vision Transformers for Video Deepfake Detection
Coccomini, D. A., Messina, N., Gennaro, C., & Falchi, F. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-06433-3_19 · Cited by 296
“For example, the recent work in [12] analyzes deepfakes in tweets for finding and defeating false content in social networks.”
BackgroundSelf-citationA Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Wong, D. F., & Chao, L. S. (2023) · arXiv · doi:10.48550/arxiv.2310.14724 · Cited by 212
“…a) In-domain Fine-tuning is All You Need: Fine-tuning Roberta provides a robust baseline for detecting text generated by LLMs. Fagni et al. [68] observed that fine-tuning Roberta led to optimal classification outcomes in various encoding configurations [132], with the subsequent OpenAI…”
BackgroundMachine-Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Crothers, E. N., Japkowicz, N., & Viktor, H. L. (2023) · IEEE Access · doi:10.1109/access.2023.3294090 · Cited by 196
“Application-specific work has applied feature-based [57] and neural [168, 172, 174] language model based detection methods to social media.”
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Crothers, E., Japkowicz, N., & Viktor, H. (2022) · arXiv · doi:10.48550/arxiv.2210.07321 · Cited by 196
“Application-specific work has applied feature-based [57] and neural [168, 172, 174] language model based detection methods to social media.”
Metadata
- Authors
- Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M.
- Affiliations
- Institute of Informatics and Telematics (IT)
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- University of Pisa (IT)
- University of Trento (IT)
- Corresponding author
- Falchi, Fabrizio
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2021-05-13
- Volume
- 16
- Issue
- 5
- Pages
- e0251415
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC8118345
- OpenAlex
- W3046357466
- Cited by
- 185
- References
- 50
- Citations by year
- 2026: 24 · 2025: 45 · 2024: 50 · 2023: 40 · 2022: 18 · 2021: 7
- FWCI
- 39.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- ISTI Open Portal (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Horizon 2020 Framework Programme (871042, 95191)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- deepfake detection, AI-generated text detection, social bots, Twitter, text generation models, GPT-2, Markov chains, recurrent neural networks, LSTM, bot detection
- MeSH terms
- Artificial Intelligence, Humans, Language, Markov Chains, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: TweepFake: About detecting deepfake tweets. https://app.cochat.ai/papers/10.1371/journal.pone.0251415
- The paper
- Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M. (2021). TweepFake: About detecting deepfake tweets. PLOS ONE. https://doi.org/10.1371/journal.pone.0251415
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.