Toward Mitigating Misinformation and Social Media Manipulation in LLM Era
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
The pervasive abuse of misinformation to influence public opinion on social media has become increasingly evident in various domains, encompassing politics, as seen in presidential elections, and healthcare, most notably during the recent COVID-19 pandemic. This threat has grown in severity as the development of Large Language Models (LLMs) empowers manipulators to generate highly convincing deceptive content with greater efficiency. Furthermore, the recent strides in chatbots integrated with LLMs, such as ChatGPT, have enabled the creation of human-like interactive social bots, posing a significant challenge to both human users and the social-bot-detection systems of social media platforms.These challenges motivate researchers to develop algorithms to mitigate misinformation and social media manipulations. This tutorial introduces the advanced machine learning researches that are helpful for this goal, including (1) detection of social manipulators, (2) learning causal models of misinformation and social manipulation, and (3) LLM-generated misinformation detection. In addition, we also present possible future directions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous works have shown that helpfulness can be misused to generate unfaithful information and a lack of scientific grounding about health16–18.”
From When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior (2025) · cites this paper as [18]
Citations
Cites
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973 · cited 1×
“Researchers have been persistently working on combating mis-information and its manipulation toward social media [2, 13, 18, 32, 33, 38].”
BackgroundThe Effects of Anti-Vaccine Conspiracy Theories on Vaccination Intentions
Jolley, D., & Douglas, K. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089177 · Cited by 1,232 · cited 1×
“Researchers have been persistently working on combating mis-information and its manipulation toward social media [2, 13, 18, 32, 33, 38].”
BackgroundThe spread of low-credibility content by social bots
Shao, C., Ciampaglia, G. L., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2018) · Nature Communications · doi:10.1038/s41467-018-06930-7 · Cited by 1,125
Online Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 981
Li, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908 · cited 1×
“On the other side, LLM’s advanced capacity in tackling text as well as inputs in other modalities (e.g., images, graph and tabular data [16, 23, 37]) benefits automatic fact verification.”
BackgroundChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1166120 · Cited by 708
Combating Fake News: A Survey on Identification and Mitigation Techniques
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3305260 · Cited by 514
Self-citationDeep neural networks for bot detection
Kudugunta, S., & Ferrara, E. (2018) · Information Sciences · doi:10.1016/j.ins.2018.08.019 · Cited by 506
van der Linden, S. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2015.07.045 · Cited by 391
(2020) · doi:10.1609/aaai.v34i01.5460 · Cited by 361
All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation
Kreps, S., McCain, R. M., & Brundage, M. (2022) · Journal of Experimental Political Science · doi:10.1017/xps.2020.37 · Cited by 360
Guo, J., Du, L., Liu, H., Zhou, M., He, X., & Han, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15066 · Cited by 335
Self-citationFake News: Fundamental Theories, Detection Strategies and Challenges
Zhou, X., Zafarani, R., Shu, K., & Liu, H. (2019) · Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining · doi:10.1145/3289600.3291382 · Cited by 256
(2020) · Sabanci University · Cited by 197
Characterizing social media manipulation in the 2020 U.S. presidential election
Ferrara, E., Chang, H., Chen, E., Muric, G., & Patel, J. (2020) · First Monday · doi:10.5210/fm.v25i11.11431 · Cited by 171
Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources
Abdelnabi, S., Hasan, R., & Fritz, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01452 · Cited by 166
Detection of Novel Social Bots by Ensembles of Specialized Classifiers
Sayyadiharikandeh, M., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412698 · Cited by 159
Online Detection of AI-Generated Images
Epstein, D. C., Jain, I., Wang, O., & Zhang, R. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00045 · Cited by 149
Luceri, L., Giordano, S., & Ferrara, E. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7311 · Cited by 114
Linguistic Cues to Deception: Identifying Political Trolls on Social Media
Addawood, A., Badawy, A., Lerman, K., & Ferrara, E. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3205 · Cited by 91 · cited 1×
“Researchers have been persistently working on combating mis-information and its manipulation toward social media [2, 13, 18, 32, 33, 38].”
BackgroundIdentifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours
Sharma, K., Zhang, Y., Ferrara, E., & Liu, Y. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467391 · Cited by 81 · cited 1×
“Moreover, misinformation has frequently been harnessed to manipulate social outcomes and public opinion, severely undermining the credibility of content on social media platforms [12, 34].”
BackgroundSelf-citationDetecting computer-generated disinformation
Stiff, H., & Johansson, F. (2022) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00299-5 · Cited by 79
Detecting and Grounding Multi-Modal Media Manipulation
Shao, R., Wu, T., & Liu, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00667 · Cited by 76
Combating Fake News: A Survey on Identification and Mitigation TechniquesPreprint
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.06437 · Cited by 75
Self-citationNewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media
Luo, G., Darrell, T., & Rohrbach, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.545 · Cited by 71
Causal Understanding of Fake News Dissemination on Social Media
Cheng, L., Guo, R., Shu, K., & Liu, H. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467321 · Cited by 69
Fake News Research: Theories, Detection Strategies, and Open Problems
Zafarani, R., Zhou, X., Shu, K., & Liu, H. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3332287 · Cited by 68
Zhang, Y., Cao, D., & Liu, Y. (2022) · neural information processing systems · doi:10.52202/068431-0773 · Cited by 35
Self-citationLess is More: Semi-Supervised Causal Inference for Detecting Pathogenic Users in Social Media
Alvari, H., Shaabani, E., Sarkar, S., Beigi, G., & Shakarian, P. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3316500 · Cited by 32
The Battle Against Online Harmful Information: The Cases of Fake News and Hate Speech
Giachanou, A., & Rosso, P. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3412169 · Cited by 31
Combating fake news: a data management and mining perspective
Lakshmanan, L. V. S., Simpson, M., & Thirumuruganathan, S. (2019) · Proceedings of the VLDB Endowment · doi:10.14778/3352063.3352117 · Cited by 26
Network Inference from a Mixture of Diffusion Models for Fake News Mitigation
Sharma, K., He, X., Seo, S., & Liu, Y. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18093 · Cited by 25
Self-citationZhang, Y., Sharma, K., & Liu, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28238-6_61 · Cited by 15
Self-citationThe Battlefront of Combating Misinformation and Coping with Media Bias
Fung, Y. R., Huang, K.-H., Nakov, P., & Ji, H. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3542615 · Cited by 13 · cited 1×
“Fung et al host a tutorial on SIGKDD 2022 to discuss the natural language processing tools for combating misinformation [14].”
BackgroundFact-Checking, Fake News, Propaganda, and Media Bias: Truth Seeking in the Post-Truth Era
Nakov, P., & Martino, G. D. S. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2020.emnlp-tutorials.2 · Cited by 13
Text-Image De-Contextualization Detection Using Vision-Language Models
Huang, M., Jia, S., Chang, M., & Lyu, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746193 · Cited by 10
Social spam, campaigns, misinformation and crowdturfing
Lee, K., Caverlee, J., & Pu, C. (2014) · International World Wide Web Conference (WWW) · doi:10.1145/2567948.2577270 · Cited by 8
Detection and Resolution of Rumors and Misinformation with NLP
Derczynski, L., & Zubiaga, A. (2020) · doi:10.18653/v1/2020.coling-tutorials.4 · Cited by 7 · cited 1×
“Derczynski introduces the techniques in the whole pipelines involved in developing fake news detector and data collection [8].”
MethodCOVID-19 Vaccines: Characterizing Misinformation Campaigns and Vaccine Hesitancy on TwitterPreprint
Sharma, K., Zhang, Y., & Liu, Y. (2021) · arXiv (Cornell University) · Cited by 7
Self-citationFact-Checking, Fake News, Propaganda, Media Bias, and the COVID-19 Infodemic
Nakov, P., Martino, G. D. S., & Alam, F. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3501395 · Cited by 7 · cited 1×
“Nakov et al host tutorials on EMNLP 2020 and WSDM 2023 to disucss the techniques of fact-checking and stance detection [26, 27].”
Pescetelli, N., Barkoczi, D., & Cebrián, M. (2022) · Studies in computational intelligence · doi:10.1007/978-3-030-93413-2_3 · Cited by 3
False Information, Bots and Malicious Campaigns: Demystifying Elements of Social Media Manipulations
Akhtar, M. M., Masood, R., Ikram, M., & Kanhere, S. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.12497
Research works printed but not linked: 7
- Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788, 2023. · arXiv
- Sarah Evanega, Mark Lynas, Jordan Adams, Karinne Smolenyak, and Cision Global Insights. Coronavirus misinformation: quantifying sources and themes in the covid-19 'infodemic'. JMIR Preprints, 19(10):2020, 2020.
- Emilio Ferrara. Social bot detection in the age of chatgpt: Challenges and opportunities. First Monday, 2023.
- Yuan Sui, Jiaru Zou, Mengyu Zhou, Xinyi He, Lun Du, Shi Han, and Dongmei Zhang. Tap4llm: Table provider on sampling, augmenting, and packing semi-structured data for large language model reasoning. arXiv preprint arXiv:2312.09039, 2023. · arXiv
- Kai-Cheng Yang and Filippo Menczer. Anatomy of an ai-powered malicious social botnet. arXiv preprint arXiv:2307.16336, 2023. · arXiv
- Yizhou Zhang, Karishma Sharma, and Yan Liu. Vigdet: Knowledge informed neural temporal point process for coordination detection on social media. Advances in Neural Information Processing Systems, 34, 2021.
- Yizhou Zhang, Loc Trinh, Defu Cao, Zijun Cui, and Yan Liu. Detecting out-ofcontext multimodal misinformation with interpretable neural-symbolic model. arXiv preprint arXiv:2304.07633, 2023. · arXiv
Cited by
Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02008-z · Cited by 112
Foundation models and intelligent decision-making: Progress, challenges, and perspectives
Huang, J., Xu, Y., Wang, Q., Wang, Q., Liang, X., Wang, F., Zhang, Z., Wei, W., Zhang, B., Huang, L., et al. (2025) · The Innovation · doi:10.1016/j.xinn.2025.100948 · Cited by 186
Silver Lining in the Fake News Cloud: Can Large Language Models Help Detect Misinformation?
Kumar, R., Goddu, B., Saha, S., & Jatowt, A. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3440248 · Cited by 32
“…shows an average accuracy of 63.41% in reasoning tasks, un-derscoring its limitations in logical, nontextual, and common-sense reasoning, alongside issues of hallucination shared with other LLMs. Additionally, Zhang et al. [16] investigated the widespread misuse of misinformation on social media.”
InfluentialTorkamaan, H., Steinert, S., Pera, M. S., Kudina, O., Freire, S. K., Verma, H., Kelly, S., Sekwenz, M.-T., Yang, J., van Nunen, K., et al. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2431068 · Cited by 43
On the (In)Security of LLM App Stores
Hou, X., Zhao, Y., & Wang, H. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00117 · Cited by 44
Metadata
- Authors
- Zhang, Y., Sharma, K., Du, L., & Liu, Y.
- Affiliations
- University of Southern California (US)
- Amazon (United States) (US)
- Coupang, Inc, Beijing, China
- Venue
- Companion Proceedings of the ACM Web Conference 2024
- Publisher
- ACM
- Published
- 2024-05-13
- Pages
- 1302-1305
- OpenAlex
- W4396843637
- Cited by
- 25
- References
- 39
- Citations by year
- 2026: 10 · 2025: 12 · 2024: 3
- FWCI
- 32.80
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- Funders
- National Science Foundation (IIS-2226087)
- University of Southern California (Annenberg Fellowship)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- misinformation, social media manipulation, large language models, social bot detection, public opinion influence, ChatGPT, COVID-19 misinformation, political misinformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.