TweepFake: About detecting deepfake tweets
Open accessThe DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.
Abstract
The recent advances in language modeling significantly improved the generative capabilities of deep neural models: in 2019 OpenAI released GPT-2, a pre-trained language model that can autonomously generate coherent, non-trivial and human-like text samples. Since then, ever more powerful text generative models have been developed. Adversaries can exploit these tremendous generative capabilities to enhance social bots that will have the ability to write plausible deepfake messages, hoping to contaminate public debate. To prevent this, it is crucial to develop deepfake social media messages detection systems. However, to the best of our knowledge no one has ever addressed the detection of machine-generated texts on social networks like Twitter or Facebook. With the aim of helping the research in this detection field, we collected the first dataset of real deepfake tweets, TweepFake . It is real in the sense that each deepfake tweet was actually posted on Twitter. We collected tweets from a total of 23 bots, imitating 17 human accounts. The bots are based on various generation techniques, i.e., Markov Chains, RNN, RNN+Markov, LSTM, GPT-2. We also randomly selected tweets from the humans imitated by the bots to have an overall balanced dataset of 25,572 tweets (half human and half bots generated). The dataset is publicly available on Kaggle. Lastly, we evaluated 13 deepfake text detection methods (based on various state-of-the-art approaches) to both demonstrate the challenges that Tweepfake poses and create a solid baseline of detection techniques. We hope that TweepFake can offer the opportunity to tackle the deepfake detection on social media messages as well.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly, a text deepfake can be used to spread disinformation alongside manipulated images, amplifying its impact on social media platforms (Fagni et al., 2021).”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Fagni et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, Fagni et al. (2021) highlighted the potential misuse of text-based deepfakes on Twitter but found existing detection methods to be inadequate.”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Fagni et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The results demonstrate TweepFake’s challenges and provide a baseline for future deepfake detection techniques.”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Fagni et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“According to their findings, a wide range of detectors based on ML or DL approaches and transformer-based utilizing transfer learning) had more trouble successfully identifying a deepfake tweet than a tweet that was actually produced by a human.”
From Deepfakes: current and future trends (2024) · cites this paper as Fagni et al. 2021
The cited source says what the claim attributes to it.
In the citing paper
“A noteworthy exception is the work by Fagni et al., who identify 23 self-disclosed bot accounts on Twitter and share the dataset (Fagni et al., 2021).”
From Anatomy of an AI-powered malicious social botnet (2024) · cites this paper as Fagni et al., 2021
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1×
“ed. Table 3. Description of the methods used in the experimentation. Encoding Method name Algorithm BoW+TF IDF log reg bow Logistic regression classifier [56] rand forest bow Random forest classifier [57] svc bow Support vector machine classifier [58] BERT log reg bert Logistic regression classifier rand forest bert R…”
MethodVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“In 2017, the development of the selfattentionmechanism and the [25]’s transformer led to the improvement of the language models.”
BackgroundMethodInfluentialExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 4×
“and perform training on a specific dataset (typically small) for very few epochs [27].”
BackgroundMethodInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 4×
“and perform training on a specific dataset (typically small) for very few epochs [27].”
BackgroundMethodInfluentialLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“Characters char cnn Single CNN network [59] using internal char embeddings representation char gru Single GRU network [60] using internal char embeddings representation char cnngru Combined CNN and GRU networks using internal char embeddings representation”
MethodRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 5×
“However, OpenAI [31] proved it wrong on GPT-2 generated texts: they showed that fine-tuning a RoBERTa-based detector achieved consistently higher accuracy than fine-tuning a GPT-2 based detector with equivalent capacity.”
MethodInfluentialAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Research works printed but not linked: 1
- Fagni T. Scripts used in “TweepFake: about Detecting Deepfake Tweets”; 2021. Repository: GitHub [Internet]. Available from: https://github.com/tizfa/tweepfake_deepfake_text_detection [cited 2021 March 16]. PMC8118345 33984021
Cited by
Combining EfficientNet and Vision Transformers for Video Deepfake Detection
Coccomini, D. A., Messina, N., Gennaro, C., & Falchi, F. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-06433-3_19 · Cited by 296
“For example, the recent work in [12] analyzes deepfakes in tweets for finding and defeating false content in social networks.”
BackgroundSelf-citationA Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Wong, D. F., & Chao, L. S. (2023) · arXiv · doi:10.48550/arxiv.2310.14724 · Cited by 212
“…a) In-domain Fine-tuning is All You Need: Fine-tuning Roberta provides a robust baseline for detecting text generated by LLMs. Fagni et al. [68] observed that fine-tuning Roberta led to optimal classification outcomes in various encoding configurations [132], with the subsequent OpenAI…”
BackgroundMachine-Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Crothers, E. N., Japkowicz, N., & Viktor, H. L. (2023) · IEEE Access · doi:10.1109/access.2023.3294090 · Cited by 196
“Application-specific work has applied feature-based [57] and neural [168, 172, 174] language model based detection methods to social media.”
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Crothers, E., Japkowicz, N., & Viktor, H. (2022) · arXiv · doi:10.48550/arxiv.2210.07321 · Cited by 196
“Application-specific work has applied feature-based [57] and neural [168, 172, 174] language model based detection methods to social media.”
Can AI-Generated Text be Reliably Detected?
Sadasivan, V. S., Kumar, A., Balasubramanian, S., Wang, W., & Feizi, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11156 · Cited by 810
“Several detection works study this problem as a binary classification problem [3, 14, 2, 15, 16].”
BackgroundAutomatic Detection of Machine Generated Text: A Critical Survey
Jawahar, G., Abdul-Mageed, M., & Lakshmanan, V. S. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.208 · Cited by 156
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., & Finn, C. (2023) · arXiv · doi:10.48550/arxiv.2301.11305 · Cited by 151
“Other works have trained supervised models for machinegenerated text detection on top of neural representations (Bakhtin et al., 2019; Solaiman et al., 2019; Uchendu et al., 2020; Ippolito et al., 2020; Fagni et al., 2021), bag-of-words features (Solaiman et al.”
BackgroundDeepfakes: current and future trends
Gambín, Á. F., Yazidi, A., Vasilakos, A., Haugerud, H., & Djenouri, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10679-x · Cited by 145
“…et al. (2018) Disinformation spreading DL-based fake news detection mechanism Kaliyar et al. (2020) Disinformation spreading DL-based fake news detection mechanism Maddocks (2020) Misinformation spreading Political and pornographic deepfakes analysis Mjaaland (2020 reference (Fagni et al. 2021).”
Deepfake: definitions, performance metrics and standards, datasets, and a meta-review
Altuncu, E., Franqueira, V. N. L., & Li, S. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1400024 · Cited by 134
A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132
“A study conducted by Fagni et al. [246] in 2021 addressed the issue of identifying deepfake tweets, which they dubbed the ‘‘TweepFake’’ problem.”
MethodThe Science of Detecting LLM-Generated Text
Tang, R., Chuang, Y.-N., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3624725 · Cited by 125
“LLMs Domain HC3 [18] ChatGPT Question Answering Neural Fake News [39] Grover News TweepFake [12] GPT2 Tweets GPT2-Output [26] GPT2 WebText TURINGBENCH [37] GPT1,2,3 News Table 1.”
BackgroundDesaire, H., Chua, A. E., Isom, M., Jarosova, R., & Hua, D. (2023) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2023.101426 · Cited by 110
Testing human ability to detect ‘deepfake’ images of human faces
Bray, S. D., Johnson, S. D., & Kleinberg, B. (2023) · Journal of Cybersecurity · doi:10.1093/cybsec/tyad011 · Cited by 96
Security and Privacy on Generative Data in AIGC: A Survey
Wang, T., Zhang, Y., Qi, S., Zhao, R., Xia, Z., & Weng, J. (2024) · ACM Computing Surveys · doi:10.1145/3703626 · Cited by 156
“For text data, most datasets are domain-specific, e.g., Student Essays [128] for essays, TuringBench [125] for news, GPABenchmark [89] for academic writing, SynSciPass [111] for scientific text, TweepFake and [42] for tweets.”
BackgroundA Review of Image Processing Techniques for Deepfakes
Shahzad, H. F., Rustam, F., Flores, E. S., Luís Vidal Mazón, J., de la Torre Diez, I., & Ashraf, I. (2022) · Sensors · doi:10.3390/s22124556 · Cited by 79
“Although for the same purpose of deepfake detection, another study [114] created a unique dataset for deepfake tweets where different bots are used to generate tweets about specific topics.”
BackgroundDetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text
Su, J., Zhuo, T., Wang, D., & Nakov, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.827 · Cited by 75
Detecting computer-generated disinformation
Stiff, H., & Johansson, F. (2022) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00299-5 · Cited by 79
“…reason, a number of additional datasets have been experimented with to test the detectors’ in-the-wild detection capabilities: – TweepFake dataset [16]: TweepFake (Twitter Deep Fake Dataset) is a dataset consisting of a mix of tweets written by 23 genuine Twitter accounts and equally many bot…”
BackgroundMAGE: Machine-generated Text Detection in the Wild
Li, Y., Li, Q., Cui, L., Bi, W., Wang, Z., Wang, L., Yang, L., Shi, S., & Zhang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.3 · Cited by 79
“Another alternative paradigm is training neural-based detectors (Bakhtin et al., 2019; Fagni et al., 2021; Uchendu et al., 2020; OpenAI, 2023a).”
Quantifying large language model usage in scientific papers
Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., Cao, H., Liu, S., He, S., Huang, Z., et al. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02273-8 · Cited by 80
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71
“A noteworthy exception is the work by Fagni et al., who identify 23 self-disclosed bot accounts on Twitter and share the dataset (Fagni et al., 2021).”
BackgroundLiang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07183 · Cited by 73
“…the risks associated with AI-generated content has sparked a debate about their effectiveness and reliability (OpenAI, 2019; Jawahar et al., 2020; Fagni et al., 2021; Ippolito et al., 2019; Mitchell et al., 2023b; Gehrmann et al., 2019; Heikkil ¨ a, 2022; Crothers et al., 2022; Solaiman et al.,…”
Shaaban, O. A., Yıldırım, R., & Alguttar, A. A. (2023) · IEEE Access · doi:10.1109/access.2023.3333866 · Cited by 107
“Some studies employ transformer-based algorithms to identify fraudulent text on social media [24], [25].”
BackgroundShaikh, S., Daudpota, S. M., Imran, A. S., & Kastrati, Z. (2021) · Applied Sciences · doi:10.3390/app11020869 · Cited by 79
“The possible reason for improved performance in GPT-2 is due to the transformer language model which relies on the attention mechanism, trained on millions of sentences, words and characters which learns each and every possible context resulting in generating coherent and human-like text [32].”
BackgroundJiang, Y., Hao, J., Fauss, M., & Li, C. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105070 · Cited by 68
Qureshi, S. M., Saeed, A., Almotiri, S. H., Ahmad, F., & Al Ghamdi, M. A. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2037 · Cited by 71
“Fagni et al. (2021) presents a novel dataset for detecting deepfake tweets called TweepFake.”
BackgroundInfluentialIbrahim, K. (2023) · Language Testing in Asia · doi:10.1186/s40468-023-00260-2 · Cited by 66
“…classifiers based on RoBERTA model (Liu et al., 2019), a fine-tuned AI detector trained on the output of existing large language models (e.g., GPT-2), has demonstrated higher detection accuracy compared to other classifiers (Solaiman et al., 2019; Jawahar et al., 2020; Fagni et al., 2021).”
MethodDeep Learning in Diverse Intelligent Sensor Based Systems
Zhu, Y., Wang, M., Yin, X., Zhang, J., Meijering, E. H. W., & Hu, J. (2022) · Sensors · doi:10.3390/s23010062 · Cited by 84
Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?
Bhattacharjee, A., & Liu, H. (2024) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3655103.3655106 · Cited by 65
“TweepFake [13]: This dataset comprises Tweets written by humans and also deepfake Tweets.”
BackgroundThe social anatomy of climate change denial in the United States
Yang, J., Gounaridis, D., Miaomiao, L., & Newell, J. (2024) · Scientific Reports · doi:10.1038/s41598-023-50591-6 · Cited by 111
“Previous studies found that the GPT-2 model performs well in classifying short text from social media 44 .”
BackgroundHuman performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers
Diel, A., Lalgi, T., Schröter, I. C., MacDorman, K. F., Teufel, M., & Bäuerle, A. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100538 · Cited by 64
An Investigation of the Effectiveness of Deepfake Models and Tools
Mukta, M. S. H., Ahmad, J., Raiaan, M. A. K., Islam, S., Azam, S., Ali, M. E., & Jonkman, M. (2023) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan12040061 · Cited by 85
“Decision trees [40,87], random forests [41,43,88], and other tree-based machine learning techniques use a tree to symbolize the decision-making process.”
MethodDetecting AI-Generated Text: Factors Influencing Detectability with Current Methods
Fraser, K. C., Dawkins, H., & Kiritchenko, S. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.16665 · Cited by 104
“However, we expect the situation to continue to (Liu et al., 2024) English Academic publications 2800K CHEAT (Yu et al., 2025) English Academic publications 50K OUTFOX (Koike et al., 2024) English Essays 30K ArguGPT (Liu et al., 2023b) English Essays 8K LLMFake (Chen & Shu, 2023) English Misinformation 5K COVID-19 Twe…”
Mark My Words: Analyzing and Evaluating Language Model Watermarks
Piet, J., Sitawarin, C., Fang, V., Mu, N., & Wagner, D. (2023) · arXiv · doi:10.48550/arxiv.2312.00273 · Cited by 61
“Many schemes can be found in the literature, including OpenAI’s classifiers [24, 25], GLTR [26], DetectGPT [27], and others [28, 29, 30, 31, 32].”
Deepfake Text Detection: Limitations and Opportunities
Pu, J., Sarwar, Z., Abdullah, S. M., Rehman, A., Kim, Y., Bhattacharya, P., Javed, M., & Viswanath, B. (2023) · 2023 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46215.2023.10179387 · Cited by 53
Identifying and Mitigating the Security Risks of Generative AI
Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000041 · Cited by 57
“Neural network-based detectors: These detectors are trained as binary classifiers to distinguish between AI and human-generated content [69, 41, 65, 11, 24].”
BackgroundWang, J., Zhang, Z., Feng, L., Lin, K.-Y., & Liu, P. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122481 · Cited by 55
Sadiq, S., Aljrees, T., & Ullah, S. (2023) · IEEE Access · doi:10.1109/access.2023.3308515 · Cited by 78
“There is also a lack of properly labelled datasets containing human and machine-generated short text in the research community [19].”
BackgroundMethodInfluentialUsing deepfakes for experiments in the social sciences - A pilot study
Eberl, A., Kühn, J., & Wolbring, T. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.907199 · Cited by 48
“Due to the threat potential attributed to deepfakes, several studies deal with the computer-assisted detection of deepfakes , i.e., automated detection through machine learning (e.g., Zhang et al., 2017; Matern et al., 2019; Fagni et al., 2021; Mehta et al., 2021; Trinh and Liu, 2021).”
BackgroundOn the Generalization of Deep Learning Models in Video Deepfake Detection
Coccomini, D. A., Caldelli, R., Falchi, F., & Gennaro, C. (2023) · Preprint (MDPI) · doi:10.20944/preprints202303.0161.v1 · Cited by 46
“This is a problem not just in the field of images, but also in text, where recent work such as [7] has analysed deepfakes in tweets to identify fake content on social networks.”
BackgroundSelf-citationCross-Forgery Analysis of Vision Transformers and CNNs for Deepfake Image Detection
Coccomini, D. A., Caldelli, R., Falchi, F., Gennaro, C., & Amato, G. (2022) · Proceedings of the 1st International Workshop on Multimedia AI against Disinformation · doi:10.1145/3512732.3533582 · Cited by 47
“This is a problem that also affects other areas such as text, with recent work such as [11] analyzing deepfakes in tweets to identify fake content in social networks.”
BackgroundSelf-citationVasist, P. N., & Krishnan, S. (2022) · Internet Research · doi:10.1108/intr-06-2022-0465 · Cited by 49
Human vs. Machine: A Comparative Study on the Detection of AI-Generated Content
Boutadjine, A., Harrag, F., & Shaalan, K. F. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3708889 · Cited by 69
“In response, researchers have embarked on an exploration of a spectrum of techniques, from classical machine learning ( ML ) algorithms [ 1 , 28 , 53 ] to state-of-the-art DL models [ 4 , 21 , 26 ] with the overarching goal of achieving precise and accurate classification results.”
Cross-domain detection of GPT-2-generated technical text
Rodriguez, J., Hay, T., Gros, D., Shamsi, Z., & Srinivasan, R. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.88 · Cited by 40
“This is in line with results in (So-laiman et al., 2019; Uchendu et al., 2020; Fagni et al., 2021), which showed the effectiveness of RoBERTa for detecting GPT-2-generated text.”
BackgroundResultGPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2304.02819 · Cited by 40
Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges
Huang, B., Chen, C., & Shu, K. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3715073.3715076 · Cited by 42
“Typically, this task is approached as a binary classification problem (Fagni et al., 2021, Jawahar et al., 2020, Mitchell et al., 2023, Solaiman et al., 2019, Zellers et al., 2019).”
Adversarial attacks and defenses for large language models (LLMs): methods, frameworks & challenges
Rai, P. K. (2024) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-024-00334-8 · Cited by 57
“In their examination, Fagni et al. [225] observed that fine-tuning Roberta yielded optimal classification outcomes across various encoding configura-tions [226].”
Deepfakes: evolution and trends
Gil, R., Virgili‐Gomà, J., López, J. M., & García, R. (2023) · Soft Computing · doi:10.1007/s00500-023-08605-y · Cited by 50
“…and Safro (2021), Diakopoulos and Johnson (2021), José and García-Ull (2021), Huber et al. (2021), Tahir and Batool (2021), Nygren et al. (2021), Fagni et al. (2021), Medoff and B.K. (2021), Pu et al. (2021b), Castillo Camacho and Wang (2021b), Mcglynn and Johnson (2021), D’Alessandra and…”
BackgroundDeep learning detection method for large language models-generated scientific content
Alhijawi, B., Jarrar, R., AbuAlRub, A., & Bader, A. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10538-y · Cited by 52
“Therefore, substantial research efforts have been made to develop plagiarism detection [2, 22, 21, 14, 30, 3, 6, 29, 4, 19, 33], machine-generated text detection [25, 13, 26, 1, 7, 17, 5, 24, 9, 11], and authorship identification methods [2, 30, 28, 27].”
InfluentialDisinformation Capabilities of Large Language Models
Vykopal, I., Pikuliak, M., Srba, I., Moro, R., Macko, D., & Bielikova, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.793 · Cited by 34
“Such disinformation is being spread across online platforms, including newspaper articles (Zellers et al., 2020), X posts (Fagni et al., 2021), Reddit posts (Cutler et al., 2021), Amazon reviews (Adelani et al., 2020), or online comments (Yanagi et al., 2020).”
Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?
Uchendu, A., Lee, J., Shen, H., Le, T., Huang, T.-H. '., & Lee, D. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27557 · Cited by 30
“In recent years, significant advancements in AI technologies have revolutionized the generation of high-quality artifacts across various modalities, including texts, images, and videos (Fagni et al. 2021; Zhang 2022; Pu et al. 2023; Shen and Wu 2023).”
StyloAI: Distinguishing AI-Generated Content with Stylometric Analysis
Opara, C. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64312-5_13 · Cited by 89
Visual User-Generated Content Verification in Journalism: An Overview
Khan, S. A., Sheikhi, G., Opdahl, A. L., Rabbi, F., Stoppel, S., Trattner, C., & Dang‐Nguyen, D. (2023) · IEEE Access · doi:10.1109/access.2023.3236993 · Cited by 53
“Besides this, it has been shown that the generative models are capable of generating synthetic news articles and tweets [114], [115].”
BackgroundLLMDet: A Third Party Large Language Models Generated Text Detection Tool
Wu, K., Pang, L., Shen, H.-W., Cheng, X.-Q., & Chua, T.-S. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.findings-emnlp.139 · Cited by 93
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective
Uchendu, A., Le, T., & Lee, D. (2023) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3606274.3606276 · Cited by 31
“TweepFake [32] Collected tweets generated by Twitter bots and grouped them into tweets generated by GPT-2, RNN, and other bots Binary Tweets Human & Machine”
MethodComparative analysis of text-based plagiarism detection techniques
Sajid, M., Sanaullah, M., Fuzail, M., Malik, T. S., & Shuhidan, S. M. (2025) · PLoS ONE · doi:10.1371/journal.pone.0319551 · Cited by 48
“As part of the “Tweep Fake” project [41,42], a detector was created by the authors Fagni et al. and Jawahar et al. in 2021 to detect deepfake tweets.”
Artificial Text Detection via Examining the Topology of Attention Maps
Kushnareva, L., Cherniavskii, D., Mikhailov, V., Artemova, E., Barannikov, S., Bernstein, A., Piontkovskaya, I., Piontkovski, D., & Burnaev, E. V. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.50 · Cited by 81
“, 2019) has demonstrated an outstanding performance with respect to many TGMs and domains (Adelani et al., 2020; Fagni et al., 2021).”
AI-Generated Text Detector for Arabic Language Using Encoder-Based Transformer Architecture
Alshammari, H., El-Sayed, A., & Elleithy, K. M. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8030032 · Cited by 48
“In 2021, a detector was developed to identify deepfake tweets as part of the ‘Tweep-Fake’ project [42,43].”
MethodA Review of Spam Detection in Social Media
Yurtseven, I., Bağrıyanık, S., & Ayvaz, S. (2021) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk52708.2021.9558993 · Cited by 33
Enhancing Machine-Generated Text Detection: Adversarial Fine-Tuning of Pre-Trained Language Models
Lee, D., & Jang, B. (2024) · IEEE Access · doi:10.1109/access.2024.3396820 · Cited by 30
Fine-Tuned Understanding: Enhancing Social Bot Detection With Transformer-Based Classification
Sallah, A., Alaoui, E. A. A., Agoujil, S., Wani, M. A., Hammad, M. A., Maleh, Y., & El‐Latif, A. A. A. (2024) · IEEE Access · doi:10.1109/access.2024.3440657 · Cited by 43
Detection of AI-Generated Texts: A Bi-LSTM and Attention-Based Approach
Blake, J., Miah, A. S. M., Kredens, K., & Shin, J. (2025) · IEEE Access · doi:10.1109/access.2025.3562750 · Cited by 30
Deep Fake Recognition in Tweets Using Text Augmentation, Word Embeddings and Deep Learning
Tesfagergish, S. G., Damaševičius, R., & Kapočiūtė-Dzikienė, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86979-3_37 · Cited by 25
Tactics, Threats & Targets: Modeling Disinformation and its Mitigation
Mirza, S., Begum, L., Niu, L., Pardo, S., Abouzied, A., Papotti, P., & Pöpper, C. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2023.23657 · Cited by 45
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2023) · arXiv · doi:10.48550/arxiv.2307.12267 · Cited by 25
“Driven by the above risks and concerns, many studies focused on differentiating human-written content from AI-generated content have been conducted (Mitchell et al. 2023; Ma, Liu, and Yi 2023; Zellers et al. 2019; Uchendu et al. 2020; Fagni et al. 2021; Ippolito et al. 2020).”
BackgroundMethodInfluentialStylometric Detection of AI-Generated Text in Twitter Timelines
Kumarage, T., Garland, J., Bhattacharjee, A., Trapeznikov, K., Ruston, S., & Liu, H. (2023) · arXiv · doi:10.48550/arxiv.2303.03697 · Cited by 25
“Since fine-tuned RoBERTa models are known to perform well for generated text detection [5,24], we chose to use RoBERTa as the language model for our stylom-etry fusion architecture in the task of Human-vs.”
InfluentialAdversarial Robustness of Neural-Statistical Features in Detection of Generative Transformers
Crothers, E., Japkowicz, N., Viktor, H., & Branco, P. (2022) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892269 · Cited by 22
“Discriminators from generative models such as Grover do, however, demonstrate improved performance when trained against the output of a smaller architecture (e.g., the GPT-2 355M parameter variant) and tested against a larger architecture (e.g., the GPT-2 1.5B parameter variant) [13] [23].”
MethodDetection of AI-Generated Text Using Large Language Model
Prajapati, M., Baliarsingh, S. K., Dora, C., Bhoi, A., Hota, J., & Mohanty, J. P. (2024) · International Conference on Emerging Systems and Intelligent Computing (ESIC) · doi:10.1109/esic60604.2024.10481602 · Cited by 33
The New Paradigm of Deepfake Detection at the Text Level
Roșca, C.-M., Stancu, A., & Iovanovici, E. M. (2025) · Applied Sciences · doi:10.3390/app15052560 · Cited by 34
DeSI: Deepfake Source Identifier for Social Media
Narayan, K., Agarwal, H., Mittal, S., Thakral, K., Kundu, S., Vatsa, M., & Singh, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00323 · Cited by 21
“Textual media: Fagni et al. [8] presented the first dataset of deepfake texts extracted from Twitter consisting of tweets from 23 bot accounts each mapped to one of the 17 human accounts they imitated.”
BackgroundShould we agree to disagree about Twitter’s bot problem?
Varol, O. (2023) · Online Social Networks and Media · doi:10.1016/j.osnem.2023.100263 · Cited by 28
A Benchmark Dataset to Distinguish Human-Written and Machine-Generated Scientific Papers
Abdalla, M. H. I., Malberg, S., Dementieva, D., Mosca, E., & Groh, G. (2023) · Information · doi:10.3390/info14100522 · Cited by 28
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30258 · Cited by 20
“…texts that may be maliciously used to sway public opinion in a variety of ways, e.g., fake news (Zellers et al. 2019), fake app reviews (Martens and Maalej 2019), fake social media posts (Fagni et al. 2021), and others (Weidinger et al. 2021; Abid, Fa-rooqi, and Zou 2021; Gehman et al. 2020).”
BackgroundMethodInfluentialStylometry recognizes human and LLM-generated texts in short samples
Przystalski, K., Argasiński, J. K., Grabska-Gradzińska, I., & Ochab, J. K. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129001 · Cited by 54
On pushing DeepFake Tweet Detection capabilities to the limits
Gambini, M., Fagni, T., Falchi, F., & Tesconi, M. (2022) · 14th ACM Web Science Conference 2022 · doi:10.1145/3501247.3531560 · Cited by 19
“Starting from the state-of-the-art research about deepfake social media text detection [1, 11, 28, 31, 34], we study whether the detection of GPT-2 social media texts can be improved by optimizing the hyper-parameters of the most promising detection techniques, i.”
BackgroundMethodInfluentialSelf-citationMulBot: Unsupervised Bot Detection Based on Multivariate Time Series
Mannocci, L., Cresci, S., Monreale, A., Vakali, A. I., & Tesconi, M. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020363 · Cited by 26
“On the other hand, most unsupervised approaches rely on textual content analysis, which is now unreliable due to recent improvements in natural language generation [27], [28].”
BackgroundSelf-citationKhalid, M., Raza, A., Younas, F., Rustam, F., Villar, M. G., Ashraf, I., & Akhtar, A. (2024) · IEEE Access · doi:10.1109/access.2024.3398582 · Cited by 35
Countering Social Media Cybercrime Using Deep Learning: Instagram Fake Accounts Detection
Alharbi, N. H., Alkalifah, B., Alqarawi, G., & Rassam, M. A. (2024) · Future Internet · doi:10.3390/fi16100367 · Cited by 38
New Automation for Social Bots: From Trivial Behavior to AI-Powered Communication
Grimme, C., Pohl, J., Cresci, S., Lüling, R., & Preuss, M. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-18253-2_6 · Cited by 19
Automated Identification of Social Media Bots Using Deepfake Text Detection
Saravani, S. M., Ray, I., & Ray, I. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-92571-0_7 · Cited by 18
“Finally, the authors of [11], who have published the dataset that we use in this work, have drawn attention to detecting deep fake text in social media platforms.”
BackgroundMethodInfluentialRobinson, J. R., Stey, A. M., Schneider, D. F., Kothari, A. N., Lindeman, B. M., Kaafarani, H. M. A., & Haines, K. (2025) · Journal of Surgical Research · doi:10.1016/j.jss.2024.12.059 · Cited by 21
An Overview on Language Models: Recent Developments and OutlookPreprint
Wei, C., Wang, Y.-C., Wang, B., & Kuo, C. ‐. J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.05759 · Cited by 15
“, [123, 124, 125], are collected for the first type.”
BackgroundAI-generated text detection: A comprehensive review of methods, datasets, and applications
Kehkashan, T., Riaz, R. A., Al‐Shamayleh, A. S., Akhunzada, A., Ali, N., Hamza, M., & Akbar, F. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100793 · Cited by 39
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future
Yang, L., Song, Y., Ren, X., Lyu, C., Wang, Y., Zhuo, J., Liu, L., Wang, J., Foster, J., & Zhang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.276 · Cited by 54
“An alternative paradigm involves training neural-based detectors (Bakhtin et al., 2019; Fagni et al., 2021), including the official implementation of OpenAI (OpenAI, 2023a).”
MethodLee, S.-C., Lee, D.-G., & Seo, Y. (2023) · Applied Intelligence · doi:10.1007/s10489-023-05201-3 · Cited by 21
“Some of these studies focused on the detection of deepfake tweets on Twitter [31].”
Manifestations of xenophobia in AI systems
Tomašev, N., Maynard, J. L., & Gabriel, I. (2024) · AI & Society · doi:10.1007/s00146-024-01893-4 · Cited by 25
Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
Shamardina, T., Mikhailov, V., Chernianskii, D., Fenogenova, A., Saidov, M., Valeeva, A., Shavrina, T., Smurov, I., Tutubalina, E., & Artemova, E. (2022) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2022-21-497-511 · Cited by 48
“Despite this, the ATD task is generally explored w.r.t. only one particular domain, e.g., product reviews (Adelani et al., 2020), social media posts (Fagni et al., 2021), or news (Uchendu et al., 2021).”
BackgroundMosca, E., Abdalla, M. H. I., Basso, P., Musumeci, M., & Groh, G. (2023) · Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023) · doi:10.18653/v1/2023.trustnlp-1.17 · Cited by 14
“…have focused on utilizing hand-crafted features (Gehrmann et al., 2019), bag-of-words features (Fagni et al., 2021), or neural features in combination with supervised models to distinguish between human and machine-generated content (Bakhtin et al., 2019; Ippolito et al., 2019; Fagni et al., 2021).”
BackgroundInfluentialBot or Human? Detection of DeepFake Text with Semantic, Emoji, Sentiment and Linguistic Features
Chong, A. T. Y., Chua, H. N., Jasser, M. B., & Wong, R. T. K. (2023) · IEEE International Conference on System Engineering and Technology (ICSET) · doi:10.1109/icset59111.2023.10295100 · Cited by 16
“With the continuous evolution of natural language processing (NLP) techniques, Deepfakes have expanded into text generation [5], enabling the production of MGT that closely mimics HWT.”
BackgroundMethodInfluentialAPT-Pipe: A Prompt-Tuning Tool for Social Data Annotation using ChatGPT
Zhu, Y., Yin, Z., Tyson, G., Haq, E.-U., Lee, L., & Hui, P. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645642 · Cited by 25
“• AI-writing detection: Tweepfake [15], FakeHate [12, 22], GPT-Article [39], and GPT-Wiki [23].”
Detecting AI-Generated Texts in Cross-Domains
Zhou, Y., & Wang, J. (2024) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/3685650.3685673 · Cited by 24
“TweepFake [3] is a collection of tweets from both genuine and fake accounts, comprising a total of 25,836 tweets by 23 bots and 17 human accounts, with equal numbers of samples written by humans and generated by machines.”
BackgroundImproved Optical Flow Estimation Method for Deepfake Videos
Nassif, A. B., Nasir, Q., Talib, M. W. A., & Gouda, O. M. (2022) · Sensors · doi:10.3390/s22072500 · Cited by 14
“These fake multimedia are easily spread all over the world thanks to social media platforms that connect people with a click of a button [3].”
BackgroundBhat, S., & Varma, V. (2023) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2023.eval4nlp-1.8 · Cited by 31
“Traditionally, compiling annotations using human experts has been the primary step in formulating a supervised solution 1 for various tasks such as sentiment analysis (Rosenthal et al., 2017), bot detection (Fagni et al., 2021), and inference (Bowman et al., 2015; Wang et al., 2018).”
AI-generated or AI touch-up? Identifying AI contribution in text data
Hashemi, A., Shi, W., & Corriveau, J. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00693-9 · Cited by 17
“Existing datasets in the literature are mainly designed to differentiate machine-generated text from human-written text [10, 13].”
Emerging need to regulate deepfakes in international law: the Russo–Ukrainian war as an example
Kuźnicka-Błaszkowska, D., & Kostyuk, N. (2025) · Journal of Cybersecurity · doi:10.1093/cybsec/tyaf008 · Cited by 27
Detecting multi-modal GAI-manipulated tourism review
Li, J., Zheng, W., & Guo, X. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105220 · Cited by 13
Li, Q., Zeng, Z., Li, T., & Sun, S. (2024) · Journal of Information Science · doi:10.1177/01655515241281491 · Cited by 14
Folk Models of Misinformation on Social Media
Sharevski, F., Devine, A., Pieroni, E., & Jachim, P. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2023.24293 · Cited by 17
A study on deep fake identification techniques using deep learning
Lal, K., & Saini, M. L. (2023) · AIP conference proceedings · doi:10.1063/5.0154828 · Cited by 12
Enhancing the Robustness of AI-Generated Text Detectors: A Survey
Liu, X., Li, Y., & Li, K. (2025) · Mathematics · doi:10.3390/math13132145 · Cited by 10
Metadata
- Authors
- Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M.
- Affiliations
- Institute of Informatics and Telematics (IT)
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- University of Pisa (IT)
- University of Trento (IT)
- Corresponding author
- Falchi, Fabrizio
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2021-05-13
- Volume
- 16
- Issue
- 5
- Pages
- e0251415
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC8118345
- OpenAlex
- W3046357466
- Cited by
- 185
- References
- 50
- Citations by year
- 2026: 24 · 2025: 45 · 2024: 50 · 2023: 40 · 2022: 18 · 2021: 7
- FWCI
- 39.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- ISTI Open Portal (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Horizon 2020 Framework Programme (871042, 95191)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- deepfake detection, AI-generated text detection, social bots, Twitter, text generation models, GPT-2, Markov chains, recurrent neural networks, LSTM, bot detection
- MeSH terms
- Artificial Intelligence, Humans, Language, Markov Chains, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.