Toward Mitigating Misinformation and Social Media Manipulation in LLM Era
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
The pervasive abuse of misinformation to influence public opinion on social media has become increasingly evident in various domains, encompassing politics, as seen in presidential elections, and healthcare, most notably during the recent COVID-19 pandemic. This threat has grown in severity as the development of Large Language Models (LLMs) empowers manipulators to generate highly convincing deceptive content with greater efficiency. Furthermore, the recent strides in chatbots integrated with LLMs, such as ChatGPT, have enabled the creation of human-like interactive social bots, posing a significant challenge to both human users and the social-bot-detection systems of social media platforms.These challenges motivate researchers to develop algorithms to mitigate misinformation and social media manipulations. This tutorial introduces the advanced machine learning researches that are helpful for this goal, including (1) detection of social manipulators, (2) learning causal models of misinformation and social manipulation, and (3) LLM-generated misinformation detection. In addition, we also present possible future directions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous works have shown that helpfulness can be misused to generate unfaithful information and a lack of scientific grounding about health16–18.”
From When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior (2025) · cites this paper as [18]
Citations
Cites
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973 · cited 1×
“Researchers have been persistently working on combating mis-information and its manipulation toward social media [2, 13, 18, 32, 33, 38].”
BackgroundThe Effects of Anti-Vaccine Conspiracy Theories on Vaccination Intentions
Jolley, D., & Douglas, K. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089177 · Cited by 1,232 · cited 1×
“Researchers have been persistently working on combating mis-information and its manipulation toward social media [2, 13, 18, 32, 33, 38].”
BackgroundThe spread of low-credibility content by social bots
Shao, C., Ciampaglia, G. L., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2018) · Nature Communications · doi:10.1038/s41467-018-06930-7 · Cited by 1,125
Online Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 981
Li, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908 · cited 1×
“On the other side, LLM’s advanced capacity in tackling text as well as inputs in other modalities (e.g., images, graph and tabular data [16, 23, 37]) benefits automatic fact verification.”
BackgroundChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1166120 · Cited by 708
Combating Fake News: A Survey on Identification and Mitigation Techniques
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3305260 · Cited by 514
Self-citationDeep neural networks for bot detection
Kudugunta, S., & Ferrara, E. (2018) · Information Sciences · doi:10.1016/j.ins.2018.08.019 · Cited by 506
van der Linden, S. (2015) · Personality and Individual Differences · doi:10.1016/j.paid.2015.07.045 · Cited by 391
(2020) · doi:10.1609/aaai.v34i01.5460 · Cited by 361
Research works printed but not linked: 9
- Mohammad Majid Akhtar, Rahat Masood, Muhammad Ikram, and Salil S Kanhere. False information, bots and malicious campaigns: Demystifying elements of social media manipulations. arXiv preprint arXiv:2308.12497, 2023. · arXiv
- Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788, 2023. · arXiv
- Sarah Evanega, Mark Lynas, Jordan Adams, Karinne Smolenyak, and Cision Global Insights. Coronavirus misinformation: quantifying sources and themes in the covid-19 'infodemic'. JMIR Preprints, 19(10):2020, 2020.
- Emilio Ferrara. Social bot detection in the age of chatgpt: Challenges and opportunities. First Monday, 2023.
- Jiayan Guo, Lun Du, and Hengyu Liu. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. arXiv preprint arXiv:2305.15066, 2023. · arXiv
- Yuan Sui, Jiaru Zou, Mengyu Zhou, Xinyi He, Lun Du, Shi Han, and Dongmei Zhang. Tap4llm: Table provider on sampling, augmenting, and packing semi-structured data for large language model reasoning. arXiv preprint arXiv:2312.09039, 2023. · arXiv
- Kai-Cheng Yang and Filippo Menczer. Anatomy of an ai-powered malicious social botnet. arXiv preprint arXiv:2307.16336, 2023. · arXiv
- Yizhou Zhang, Karishma Sharma, and Yan Liu. Vigdet: Knowledge informed neural temporal point process for coordination detection on social media. Advances in Neural Information Processing Systems, 34, 2021.
- Yizhou Zhang, Loc Trinh, Defu Cao, Zijun Cui, and Yan Liu. Detecting out-ofcontext multimodal misinformation with interpretable neural-symbolic model. arXiv preprint arXiv:2304.07633, 2023. · arXiv
Cited by
Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02008-z · Cited by 112
Foundation models and intelligent decision-making: Progress, challenges, and perspectives
Huang, J., Xu, Y., Wang, Q., Wang, Q., Liang, X., Wang, F., Zhang, Z., Wei, W., Zhang, B., Huang, L., et al. (2025) · The Innovation · doi:10.1016/j.xinn.2025.100948 · Cited by 186
Silver Lining in the Fake News Cloud: Can Large Language Models Help Detect Misinformation?
Kumar, R., Goddu, B., Saha, S., & Jatowt, A. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3440248 · Cited by 32
“…shows an average accuracy of 63.41% in reasoning tasks, un-derscoring its limitations in logical, nontextual, and common-sense reasoning, alongside issues of hallucination shared with other LLMs. Additionally, Zhang et al. [16] investigated the widespread misuse of misinformation on social media.”
InfluentialTorkamaan, H., Steinert, S., Pera, M. S., Kudina, O., Freire, S. K., Verma, H., Kelly, S., Sekwenz, M.-T., Yang, J., van Nunen, K., et al. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2431068 · Cited by 42
On the (In)Security of LLM App Stores
Hou, X., Zhao, Y., & Wang, H. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00117 · Cited by 44
Metadata
- Authors
- Zhang, Y., Sharma, K., Du, L., & Liu, Y.
- Affiliations
- University of Southern California (US)
- Amazon (United States) (US)
- Coupang, Inc, Beijing, China
- Venue
- Companion Proceedings of the ACM Web Conference 2024
- Publisher
- ACM
- Published
- 2024-05-13
- Pages
- 1302-1305
- OpenAlex
- W4396843637
- Cited by
- 25
- References
- 39
- Citations by year
- 2026: 10 · 2025: 12 · 2024: 3
- FWCI
- 32.80
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- Funders
- National Science Foundation (IIS-2226087)
- University of Southern California (Annenberg Fellowship)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- misinformation, social media manipulation, large language models, social bot detection, public opinion influence, ChatGPT, COVID-19 misinformation, political misinformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Toward Mitigating Misinformation and Social Media Manipulation in LLM Era. https://app.cochat.ai/papers/10.1145/3589335.3641256
- The paper
- Zhang, Y., Sharma, K., Du, L., & Liu, Y. (2024). Toward Mitigating Misinformation and Social Media Manipulation in LLM Era. Companion Proceedings of the ACM Web Conference 2024. https://doi.org/10.1145/3589335.3641256
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.