CoChat
Sign Up
Cited by 6Misinformation and Its ImpactsTopic ModelingSpam and Phishing Detection

Automated Justification Production for Claim Veracity in Fact Checking: A Survey on Architectures and Approaches

Open access
Eldifrawi, Islam, Wang, Shengrui, Trabelsi, Amine·2024·Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
DOI 10.18653/v1/2024.acl-long.361
Open this paper in CoChat — free accountOpen PDF

Abstract

Automated Fact-Checking (AFC) is the automated verification of claim accuracy.AFC is crucial in discerning truth from misinformation, especially given the huge amounts of content are generated online daily.Current research focuses on predicting claim veracity through metadata analysis and language scrutiny, with an emphasis on justifying verdicts.This paper surveys recent methodologies, proposing a comprehensive taxonomy and presenting the evolution of research in that landscape.A comparative analysis of methodologies and future directions for improving fact-checking explainability are also discussed.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2024.acl-long.361

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Automated fact checking has evolved into a well-established multistage pipeline, which has become the dominant research paradigm in the field.

    “Driven by efforts from both academia and industry, automated fact checking has evolved into a well-established multistage pipeline, which has become the dominant research paradigm in the field Eldifrawi et al. (2024).”

    From ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks (2025) · cites this paper as Eldifrawi et al. (2024)

    Section 3 describes automated fact-checking as a pipeline with multiple stages, and Section 6.3 classifies existing pipeline architectures. The paper does not establish that this pipeline is the dominant research paradigm or attribute its development to efforts from both academia and industry. The indexed passages provide no page numbers. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 53

  1. Naser Ahmadi, Joohyung Lee, Paolo Papotti, and Mo- hammed Saeed. 2019. Explainable fact checking with probabilistic answer set programming. arXiv preprint arXiv:1906.09198. · arXiv
  2. Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fab- rizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, and Preslav Nakov. 2022. A survey on multimodal disinforma- tion detection. In Proceedings of the 29th Inter- national Conference on Computational Linguistics, pages 6625-6643, Gyeongju, Republic of Korea. In- ternational Committee on Computational Linguistics.
  3. Pepa Atanasova, Jakob Grue Simonsen, Christina Li- oma, and Isabelle Augenstein. 2022. Diagnostics- guided explanation generation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 36, pages 10445-10453.
  4. Nicolo Bonettini, Edoardo Daniele Cannas, Sara Man- delli, Luca Bondi, Paolo Bestagini, and Stefano Tubaro. 2021. Video face manipulation detection through ensemble of cnns. In 2020 25th international conference on pattern recognition (ICPR), pages 5012-5019. IEEE.
  5. Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. 2019. Low-bit quantization of neural net- works for efficient inference. In 2019 IEEE/CVF International Conference on Computer Vision Work- shop (ICCVW), pages 3009-3018. IEEE.
  6. Giovanni Luca Ciampaglia, Prashant Shiralkar, Luis M Rocha, Johan Bollen, Filippo Menczer, and Alessandro Flammini. 2015. Computational fact checking from knowledge networks. PloS one, 10(6):e0128193.
  7. Shih-Chieh Dai, Yi-Li Hsu, Aiping Xiong, and Lun-Wei Ku. 2022. Ask to know more: Generating counter- factual explanations for fake claims. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2800-2810.
  8. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Ja- son Weston. 2023. Chain-of-verification reduces hal- lucination in large language models. arXiv preprint arXiv:2309.11495. · arXiv
  9. Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenen- baum, and Igor Mordatch. 2023. Improving factual- ity and reasoning in language models through multia- gent debate. arXiv preprint arXiv:2305.14325. · arXiv
  10. Mohamed H Gad-Elrab, Daria Stepanova, Jacopo Ur- bani, and Gerhard Weikum. 2019. Exfakt: A frame- work for explaining facts over knowledge graphs and text. In Proceedings of the twelfth ACM international conference on web search and data mining, pages 87-95.
  11. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738. · arXiv
  12. Ashim Gupta and Vivek Srikumar. 2021. X-fact: A new benchmark dataset for multilingual fact checking. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 675-682, Online. Association for Computational Linguistics. · DOI
  13. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xiny- ing Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798. · arXiv
  14. Shailza Jolly, Pepa Atanasova, and Isabelle Augen- stein. 2022. Generating fluent fact checking expla- nations with unsupervised post-editing. Information, 13(10):500.
  15. Thomas N Kipf and Max Welling. 2016. Semi- supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907. · arXiv
  16. Neema Kotonya and Francesca Toni. 2020c. Explain- able automated fact-checking for public health claims. arXiv preprint arXiv:2010.09926. · arXiv
  17. Ziyi Kou, Daniel Yue Zhang, Lanyu Shang, and Dong Wang. 2020. Exfaux: A weakly supervised approach to explainable fauxtography detection. In 2020 IEEE International Conference on Big Data (Big Data), pages 631-636. IEEE.
  18. Amrith Krishna, Sebastian Riedel, and Andreas Vlachos. 2022. Proofver: Natural logic theorem proving for fact verification. Transactions of the Association for Computational Linguistics, 10:1013-1030.
  19. Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explana- tory debugging to personalize interactive machine learning. In Proceedings of the 20th international conference on intelligent user interfaces, pages 126- 137.
  20. Stephan Lewandowsky, Ullrich KH Ecker, Colleen M Seifert, Norbert Schwarz, and John Cook. 2012. Mis- information and its correction: Continued influence and successful debiasing. Psychological science in the public interest, 13(3):106-131.
  21. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for nat- ural language generation, translation, and comprehen- sion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880.
  22. Hongzhan Lin, Ziyang Luo, Wei Gao, Jing Ma, Bo Wang, and Ruichao Yang. 2024. Towards explain- able harmful meme detection through multimodal de- bate between large language models. arXiv preprint arXiv:2401.13298. · arXiv
  23. Vítor Lourenço and Aline Paes. 2022. A modality-level explainable framework for misinformation checking in social networks. arXiv preprint arXiv:2212.04272. · arXiv
  24. P Nakov, D Corney, M Hasanain, F Alam, T Elsayed, A Barron-Cedeno, P Papotti, S Shaar, G Da San Mar- tino, et al. 2021a. Automated fact-checking for as- sisting human fact-checkers. In IJCAI, pages 4551- 4558. International Joint Conferences on Artificial Intelligence.
  25. Liangming Pan, Xinyuan Lu, Min-Yen Kan, and Preslav Nakov. 2023a. Qacheck: A demonstration system for question-guided multi-hop fact-checking. arXiv preprint arXiv:2310.07609. · arXiv
  26. Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov. 2023b. Fact-checking complex claims with program-guided reasoning. arXiv preprint arXiv:2305.12744. · arXiv
  27. Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813. · arXiv
  28. Kashyap Popat, Subhabrata Mukherjee, Jannik Ströt- gen, and Gerhard Weikum. 2017. Where the truth lies: Explaining the credibility of emerging claims on the web and social media. In Proceedings of the 26th International Conference on World Wide Web Companion, pages 1003-1012.
  29. Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2018. Declare: Debunking fake news and false claims using evidence-aware deep learning. arXiv preprint arXiv:1809.06416. · arXiv
  30. Christian Nathaniel Purwanto, Joan Santoso, Po-Ruey Lei, Hui-Kuo Yang, and Wen-Chih Peng. 2021. Fake- clip: Multimodal fake caption detection with mixed languages for explainable visualization. In 2021 In- ternational Conference on Technologies and Appli- cations of Artificial Intelligence (TAAI), pages 1-6. IEEE.
  31. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International confer- ence on machine learning, pages 8748-8763.
  32. PMLR.
  33. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):5485-5551.
  34. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP). Association for Computational Linguistics.
  35. Daniel Russo, Serra Sinem Tekiroglu, and Marco Guerini. 2023. Benchmarking the generation of fact checking explanations. Transactions of the Associa- tion for Computational Linguistics, 11:1250-1264.
  36. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. · arXiv
  37. Aalok Sathe, Salar Ather, Tuan Manh Le, Nathan Perry, and Joonsuk Park. 2020. Automated fact-checking of claims from wikipedia. In Proceedings of the Twelfth Language Resources and Evaluation Confer- ence, pages 6874-6882.
  38. Lanyu Shang, Ziyi Kou, Yang Zhang, and Dong Wang. 2022. A duo-generative approach to explainable mul- timodal covid-19 misinformation detection. In Pro- ceedings of the ACM Web Conference 2022, pages 3623-3631.
  39. Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. 2019. defend: Explainable fake news detection. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 395-405.
  40. Jiasheng Si, Yingjie Zhu, and Deyu Zhou. 2023. Explor- ing faithful rationale for multi-hop fact verification via salience-aware graph learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13573-13581.
  41. Kacper Sokol and Peter Flach. 2019. Desiderata for interpretability: explaining decision tree predictions with counterfactuals. In Proceedings of the AAAI con- ference on artificial intelligence, volume 33, pages 10035-10036.
  42. Dominik Stammbach and Elliott Ash. 2020. e-fever: Ex- planations and summaries for automated fact check- ing. Proceedings of the 2020 Truth and Trust Online (TTO 2020), pages 32-43.
  43. James Thorne and Andreas Vlachos. 2018. Automated fact checking: Task formulations, methods and fu- ture directions. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 3346-3359, Santa Fe, New Mexico, USA. As- sociation for Computational Linguistics.
  44. Manju Vallayil, Parma Nand, Wei Qi Yan, and Héc- tor Allende-Cid. 2023. Explainability of automated fact verification systems: A comprehensive review. Applied Sciences, 13(23):12608.
  45. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  46. Haoran Wang and Kai Shu. 2023. Explainable claim verification via knowledge-grounded reason- ing with large language models. arXiv preprint arXiv:2310.05253. · arXiv
  47. Yuxia Wang, Revanth Gangi Reddy, Zain Muhammad Mujahid, Arnav Arora, Aleksandr Rubashevskii, Ji- ahui Geng, Osama Mohammed Afzal, Liangming Pan, Nadav Borenstein, Aditya Pillai, et al. 2023. Factcheck-gpt: End-to-end fine-grained document- level fact-checking and correction of llm output. arXiv preprint arXiv:2311.09000. · arXiv
  48. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824-24837.
  49. Yue Wu, Wael AbdAlmageed, and Premkumar Natara- jan. 2019. Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 9543-9552.
  50. Fan Yang, Shiva K Pentyala, Sina Mohseni, Mengnan Du, Hao Yuan, Rhema Linder, Eric D Ragan, Shui- wang Ji, and Xia Hu. 2019. Xfake: Explainable fake news detector with visualizations. In The world wide web conference, pages 3600-3604.
  51. Barry Menglong Yao, Aditya Shah, Lichao Sun, Jin-Hee Cho, and Lifu Huang. 2023. End-to-end multimodal fact-checking and explanation generation: A chal- lenging dataset and models. In Proceedings of the 46th International ACM SIGIR Conference on Re- search and Development in Information Retrieval, pages 2733-2743.
  52. Fanrui Zhang, Jiawei Liu, Qiang Zhang, Esther Sun, Jingyi Xie, and Zheng-Jun Zha. 2023a. Ecenet: Ex- plainable and context-enhanced network for muti- modal fact verification. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1231-1240.
  53. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023b. Siren's song in the ai ocean: A survey on hallucination in large language models. arXiv preprint arXiv:2309.01219. · arXiv

Cited by

Sorted by citations

Related

Authors
Eldifrawi, I., Wang, S., & Trabelsi, A.
Affiliations
Université de Sherbrooke (CA)
ORCID
Venue
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Publisher
Association for Computational Linguistics
Published
2024
Pages
6679-6692
ISSN
0736-587X (ISSN-L)
DOI
10.18653/v1/2024.acl-long.361
OpenAlex
W4402672110
Cited by
5
References
0
Citations by year
2026: 2 · 2025: 2 · 2024: 1
FWCI
6.55
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, public-domain)
  • arXiv (Cornell University) (public-domain)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
Keywords
automated fact-checking, misinformation, explainability, metadata analysis, claim verification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review