A Survey on Automatic Credibility Assessment Using Textual Credibility Signals in the Era of Large Language Models
Open accessAbstract
In the age of social media and generative AI, the ability to automatically assess the credibility of online content has become increasingly critical, complementing traditional approaches to false information detection. Credibility assessment relies on aggregating diverse credibility signals—small units of information, such as content subjectivity, bias or a presence of persuasion techniques—into a final credibility label/score. However, current research in automatic credibility assessment and credibility signals detection remains highly fragmented, with many signals studied in isolation and lacking integration. Notably, there is a scarcity of approaches that detect and aggregate multiple credibility signals simultaneously. These challenges are further exacerbated by the absence of a comprehensive and up-to-date overview of research works that connects these research efforts under a common framework and identifies shared trends, challenges and open problems. In this survey, we address this gap by presenting a systematic and comprehensive literature review of 175 research papers, focusing on textual credibility signals within the field of Natural Language Processing (NLP), which undergoes a rapid transformation due to advancements in Large Language Models (LLMs). While positioning the NLP research into the broader multidisciplinary landscape, we examine both automatic credibility assessment methods as well as the detection of nine categories of credibility signals. We provide an in-depth analysis of three key categories: (1) factuality, subjectivity and bias, (2) persuasion techniques and logical fallacies and (3) check-worthy and fact-checked claims. In addition to summarising existing methods, datasets and tools, we outline future research direction and emerging opportunities, with particular attention to evolving challenges posed by generative AI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
“Fake News” Is Not Simply False Information: A Concept Explication and Taxonomy of Online Content
Molina, M. D., Sundar, S. S., Le, T., & Lee, D. (2021) · American Behavioral Scientist · doi:10.1177/0002764219878224 · Cited by 485 · see citing works
Smith, J., McCarthy, J. D., McPhail, C., & Augustyn, B. (2001) · Social Forces · doi:10.1353/sof.2001.0053 · Cited by 396 · see citing works
Credibility: A multidisciplinary framework
Rieh, S. Y., & Danielson, D. R. (2007) · Annual Review of Information Science and Technology · doi:10.1002/aris.2007.1440410114 · Cited by 336 · see citing works
The potential of generative AI for personalized persuasion at scale
Matz, S. C., Teeny, J. D., Vaid, S. S., Peters, H., Harari, G. M., & Cerf, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-53755-0 · Cited by 315 · see citing works
The Potential of Generative AI for Personalized Persuasion at Scale
Matz, S., Teeny, J., Vaid, S. S., Peters, H., Harari, G. M., & Cerf, M. (2023) · doi:10.31234/osf.io/rn97c · Cited by 312 · see citing works
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212 · see citing works
Factuality challenges in the era of large language models and opportunities for fact-checking
Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G. L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00881-z · Cited by 187 · see citing works
Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown
Simon, F. M., Altay, S., & Mercier, H. (2023) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-127 · Cited by 85 · see citing works
MAGE: Machine-generated Text Detection in the Wild
Li, Y., Li, Q., Cui, L., Bi, W., Wang, Z., Wang, L., Yang, L., Shi, S., & Zhang, Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.3 · Cited by 79 · see citing works
Truth, Lies, and Automation: How Language Models Could Change Disinformation
Center for Security and Emerging Technology, Buchanan, B., Lohn, A., Musser, M., & Sedova, K. (2021) · doi:10.51593/2021ca003 · Cited by 74 · see citing works
Disinformation Capabilities of Large Language Models
Vykopal, I., Pikuliak, M., Srba, I., Moro, R., Macko, D., & Bielikova, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.793 · Cited by 34 · see citing works
Self-citationM4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Mohammed Afzal, O., Mahmoud, T., Puccetti, G., Arnold, T., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.218 · Cited by 25 · see citing works
Xu, R., Lin, B., Yang, S., Zhang, T., Shi, W., Zhang, T., Fang, Z., Xu, W., & Qiu, H. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.858 · Cited by 17 · see citing works
Flagged first, then by citations
- Authors
- Srba, I., Razuvayevskaya, O., Leite, J. A., Moro, R., Schlicht, I. B., Tonelli, S., García, F. M., Lottmann, S. B., Teyssou, D., Porcellini, V., Scarton, C., Bontcheva, K., & Bielikova, M.
- Affiliations
- Kempelen Institute of Intelligent Technologies (SK)
- University of Sheffield (GB)
- Kempelen Institute of Intelligent Technologies, Bratislava, Slovakia
- Deutsche Welle (DE)
- Fondazione Bruno Kessler (IT)
- Universidad Politécnica de Madrid, Madrid, Spain
- Universidad Politécnica de Madrid (ES)
- Agence France-Presse (FR)
- ORCID
- Venue
- ACM Transactions on Intelligent Systems and Technology
- Published
- 2026-01-21
- Volume
- 17
- Issue
- 2
- Pages
- 1-80
- ISSN
- 2157-6904 (ISSN-L)
- 2157-6912
- OpenAlex
- W4414654425
- Cited by
- 19
- References
- 15
- Citations by year
- 2026: 12 · 2025: 6 · 2024: 1
- FWCI
- 20.64
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- ACM Transactions on Intelligent Systems and Technology (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- RiuNet (Universitat Politècnica de València) (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- European Union under the Horizon Europe projects: vera.ai (101070093)
- AI-CODE (101135437)
- AI4TRUST (101070190)
- UK’s innovation agency (10039055)
- EU NextGenerationEU through the Recovery and Resilience Plan for Slovakia (09I03-03-V03-00020)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, subjectivity detection, bias detection, persuasion techniques, logical fallacies, misinformation detection, generative AI, social media content
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
