CoChat

FacTeR-Check: Semi-automated fact-checking through semantic similarity and natural language inference

Open access
Martín, Alejandro, Huertas-Tato, Javier, Huertas-García, Álvaro, Villar-Rodríguez, Guillermo, Camacho, David · 2022 · Knowledge-Based Systems
Cited by 61

The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.

Abstract

Our society produces and shares overwhelming amounts of information through Online Social Networks (OSNs). Within this environment, misinformation and disinformation have proliferated, becoming a public safety concern in most countries. Allowing the public and professionals to efficiently find reliable evidence about the factual veracity of a claim is a crucial step to mitigate this harmful spread. To this end, we propose FacTeR-Check, a multilingual architecture for semi-automated fact-checking and hoaxes propagation analysis that can be used to implement applications designed for both the general public and for fact-checking organisations. FacTeR-Check implements three different modules relying on the XLM-RoBERTa Transformer architecture to evaluate semantic similarity, to calculate natural language inference and to build search queries through automatic keywords extraction and Named-Entity Recognition. The three modules have been validated using state-of-the-art benchmark datasets, exhibiting good performance in all of them. Besides, FacTeR-Check is employed to collect and label a dataset, called NLI19-SP, composed of more than 40,000 tweets supporting or denying 60 hoaxes related to COVID-19, released publicly. Finally, an analysis of the data collected in this dataset is provided, which allows to obtain a deep insight of how disinformation operated during the COVID-19 pandemic in Spanish-speaking countries.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.knosys.2022.109265

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 25 references

Research works printed but not linked: 33

  1. N. Kotonya, F. Toni, Explainable automated fact-checking: a survey, in: Proceedings of the 28th International Conference on Computational Linguistics, 2020, pp. 5430–5443. · DOI · Paper page
  2. N. Naderi, G. Hirst, Automated fact-checking of claims in argumentative parliamentary debates, in: Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), 2018, pp. 60–65. · DOI
  3. M. Trokhymovych, D. Saez-Trumper, Wikicheck: An end-to-end open source automatic fact-checking api based on wikipedia, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 4155–4164. · DOI
  4. A. Alonso-Reina, R. Sepúlveda-Torres, E. Saquete, M. Palomar, Team GPLSI. Approach for automated fact checking, in: Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER), 2019, pp. 110–114. · DOI
  5. Naseem, 2020, Transformer based deep intelligent contextual embedding for twitter sentiment analysis, Future Gener. Comput. Syst., 113, 58, 10.1016/j.future.2020.06.050 · DOI · Paper page
  6. W. Yang, Y. Xie, A. Lin, X. Li, L. Tan, K. Xiong, M. Li, J. Lin, End-to-end open-domain question answering with bertserini, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 72–77. · DOI · Paper page
  7. Zhang, 2020, Learning adversarial transformer for symbolic music generation, IEEE Trans. Neural Netw. Learn. Syst.
  8. Mihalcea, 2006, Corpus-based and knowledge-based measures of text semantic similarity, Vol. 6, 775
  9. Gomaa, 2013, A survey of text similarity approaches, Int. J. Comput. Appl., 68, 13
  10. Millar, 2000, Performance and scalability of a large-scale n-gram based information retrieval system, J. Digit. Inf., 1
  11. Singthongchai, 2013, A method for measuring keywords similarity by applying jaccard’s, n-gram and vector space, Lecture Notes Inf. Theory, 1
  12. P. Shrestha, Corpus-based methods for short text similarity, in: Actes de la 18e Conférence sur Le Traitement Automatique Des Langues Naturelles. Rencontres Jeunes Chercheurs En Informatique Pour Le Traitement Automatique Des Langues (Articles Courts), 2011, pp. 1–6.
  13. M. Schuhmacher, S.P. Ponzetto, Knowledge-based graph document modeling, in: Proceedings of the 7th ACM International Conference on Web Search and Data Mining, 2014, pp. 543–552. · DOI
  14. N. Peinelt, D. Nguyen, M. Liakata, tBERT: Topic models and BERT joining forces for semantic similarity detection, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 7047–7055. · DOI
  15. Kasnesis, 2021, Transformer-based identification of stochastic information cascades in social networks using text and image similarity, Appl. Soft Comput., 108, 10.1016/j.asoc.2021.107413 · DOI
  16. Q. Chen, X. Zhu, Z.-H. Ling, S. Wei, H. Jiang, D. Inkpen, Enhanced LSTM for natural language inference, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1657–1668. · DOI · Paper page
  17. J. Thorne, A. Vlachos, Automated fact checking: task formulations, methods and future directions, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 3346–3359.
  18. S. Miranda, D. Nogueira, A. Mendes, A. Vlachos, A. Secker, R. Garrett, J. Mitchel, Z. Marinho, Automated fact checking in the news room, in: The World Wide Web Conference, 2019, pp. 3579–3583. · DOI
  19. A. Hanselowski, C. Stab, C. Schulz, Z. Li, I. Gurevych, A richly annotated corpus for different tasks in automated fact-checking, in: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), 2019, pp. 493–503. · DOI · Paper page
  20. A. Sathe, S. Ather, T.M. Le, N. Perry, J. Park, Automated fact-checking of claims from Wikipedia, in: Proceedings of the 12th Language Resources and Evaluation Conference, 2020, pp. 6874–6882.
  21. Zeng, 2021, Automated fact-checking: A survey, Lang. Linguist. Compass, 15, 10.1111/lnc3.12438 · DOI · Paper page
  22. A. Chernyavskiy, D. Ilvovsky, P. Nakov, WhatTheWikiFact: Fact-checking claims against wikipedia, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 4690–4695. · DOI
  23. M. Nadeem, W. Fang, B. Xu, M. Mohtarami, J. Glass, FAKTA: An automatic end-to-end fact checking system, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 78–83. · DOI
  24. H. Karimi, P. Roy, S. Saba-Sadiya, J. Tang, Multi-source multi-class fake news detection, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 1546–1557.
  25. N. Hassan, F. Arslan, C. Li, M. Tremayne, Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 1803–1812. · DOI · Paper page
  26. K. Shu, X. Zhou, S. Wang, R. Zafarani, H. Liu, The role of user profiles for fake news detection, in: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2019, pp. 436–439. · DOI · Paper page
  27. Alami, 2019, Enhancing unsupervised neural networks based text summarization with word embedding and ensemble learning, Expert Syst. Appl., 123, 195, 10.1016/j.eswa.2019.01.037 · DOI
  28. Devi, 2021, A hybrid ensemble word embedding based classification model for multi-document summarization process on large multi-domain document sets, Int. J. Adv. Comput. Sci. Appl., 12
  29. Sidorov, 2014, Soft similarity and soft cosine measure: similarity of features in vector space model, Comput. Y Sistemas, 18
  30. A. Akbik, T. Bergmann, D. Blythe, K. Rasul, S. Schweter, R. Vollgraf, FLAIR: An easy-to-use framework for state-of-the-art NLP, in: NAACL 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 54–59.
  31. Artetxe, 2019, Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond, Trans. Assoc. Comput. Linguist., 7, 597, 10.1162/tacl_a_00288 · DOI · Paper page
  32. Aquino, 2015, Keyword identification in spanish documents using neural networks, J. Comput. Sci. Technol., 15
  33. Campos, 2020, Yake! keyword extraction from single documents using multiple local features, Inform. Sci., 509, 257, 10.1016/j.ins.2019.09.013 · DOI · Paper page

Cited by

See all 61

Metadata

Authors
Martín, A., Huertas-Tato, J., Huertas-García, Á., Villar-Rodríguez, G., & Camacho, D.
Venue
Knowledge-Based Systems
Publisher
Elsevier BV
Published
2022-09
Volume
251
Pages
109265
ISSN
0950-7051 (ISSN-L)
OpenAlex
W3209749143
Cited by
61
References
25
FWCI
19.72
Type
journal article
Language
en
Open access
hybrid
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: FacTeR-Check: Semi-automated fact-checking through semantic similarity and natural language inference. https://app.cochat.ai/papers/10.1016/j.knosys.2022.109265
The paper
Martín, A., Huertas-Tato, J., Huertas-García, Á., Villar-Rodríguez, G., & Camacho, D. (2022). FacTeR-Check: Semi-automated fact-checking through semantic similarity and natural language inference. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2022.109265

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.