FacTeR-Check: Semi-automated fact-checking through semantic similarity and natural language inference
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Our society produces and shares overwhelming amounts of information through Online Social Networks (OSNs). Within this environment, misinformation and disinformation have proliferated, becoming a public safety concern in most countries. Allowing the public and professionals to efficiently find reliable evidence about the factual veracity of a claim is a crucial step to mitigate this harmful spread. To this end, we propose FacTeR-Check, a multilingual architecture for semi-automated fact-checking and hoaxes propagation analysis that can be used to implement applications designed for both the general public and for fact-checking organisations. FacTeR-Check implements three different modules relying on the XLM-RoBERTa Transformer architecture to evaluate semantic similarity, to calculate natural language inference and to build search queries through automatic keywords extraction and Named-Entity Recognition. The three modules have been validated using state-of-the-art benchmark datasets, exhibiting good performance in all of them. Besides, FacTeR-Check is employed to collect and label a dataset, called NLI19-SP, composed of more than 40,000 tweets supporting or denying 60 hoaxes related to COVID-19, released publicly. Finally, an analysis of the data collected in this dataset is provided, which allows to obtain a deep insight of how disinformation operated during the COVID-19 pandemic in Spanish-speaking countries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
Research works printed but not linked: 33
- N. Kotonya, F. Toni, Explainable automated fact-checking: a survey, in: Proceedings of the 28th International Conference on Computational Linguistics, 2020, pp. 5430–5443. · DOI · Paper page
- N. Naderi, G. Hirst, Automated fact-checking of claims in argumentative parliamentary debates, in: Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), 2018, pp. 60–65. · DOI
- M. Trokhymovych, D. Saez-Trumper, Wikicheck: An end-to-end open source automatic fact-checking api based on wikipedia, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 4155–4164. · DOI
- A. Alonso-Reina, R. Sepúlveda-Torres, E. Saquete, M. Palomar, Team GPLSI. Approach for automated fact checking, in: Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER), 2019, pp. 110–114. · DOI
- Naseem, 2020, Transformer based deep intelligent contextual embedding for twitter sentiment analysis, Future Gener. Comput. Syst., 113, 58, 10.1016/j.future.2020.06.050 · DOI · Paper page
- W. Yang, Y. Xie, A. Lin, X. Li, L. Tan, K. Xiong, M. Li, J. Lin, End-to-end open-domain question answering with bertserini, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 72–77. · DOI · Paper page
- Zhang, 2020, Learning adversarial transformer for symbolic music generation, IEEE Trans. Neural Netw. Learn. Syst.
- Mihalcea, 2006, Corpus-based and knowledge-based measures of text semantic similarity, Vol. 6, 775
- Gomaa, 2013, A survey of text similarity approaches, Int. J. Comput. Appl., 68, 13
- Millar, 2000, Performance and scalability of a large-scale n-gram based information retrieval system, J. Digit. Inf., 1
- Singthongchai, 2013, A method for measuring keywords similarity by applying jaccard’s, n-gram and vector space, Lecture Notes Inf. Theory, 1
- P. Shrestha, Corpus-based methods for short text similarity, in: Actes de la 18e Conférence sur Le Traitement Automatique Des Langues Naturelles. Rencontres Jeunes Chercheurs En Informatique Pour Le Traitement Automatique Des Langues (Articles Courts), 2011, pp. 1–6.
- M. Schuhmacher, S.P. Ponzetto, Knowledge-based graph document modeling, in: Proceedings of the 7th ACM International Conference on Web Search and Data Mining, 2014, pp. 543–552. · DOI
- N. Peinelt, D. Nguyen, M. Liakata, tBERT: Topic models and BERT joining forces for semantic similarity detection, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 7047–7055. · DOI
- Kasnesis, 2021, Transformer-based identification of stochastic information cascades in social networks using text and image similarity, Appl. Soft Comput., 108, 10.1016/j.asoc.2021.107413 · DOI
- Q. Chen, X. Zhu, Z.-H. Ling, S. Wei, H. Jiang, D. Inkpen, Enhanced LSTM for natural language inference, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1657–1668. · DOI · Paper page
- J. Thorne, A. Vlachos, Automated fact checking: task formulations, methods and future directions, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 3346–3359.
- S. Miranda, D. Nogueira, A. Mendes, A. Vlachos, A. Secker, R. Garrett, J. Mitchel, Z. Marinho, Automated fact checking in the news room, in: The World Wide Web Conference, 2019, pp. 3579–3583. · DOI
- A. Hanselowski, C. Stab, C. Schulz, Z. Li, I. Gurevych, A richly annotated corpus for different tasks in automated fact-checking, in: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), 2019, pp. 493–503. · DOI · Paper page
- A. Sathe, S. Ather, T.M. Le, N. Perry, J. Park, Automated fact-checking of claims from Wikipedia, in: Proceedings of the 12th Language Resources and Evaluation Conference, 2020, pp. 6874–6882.
- Zeng, 2021, Automated fact-checking: A survey, Lang. Linguist. Compass, 15, 10.1111/lnc3.12438 · DOI · Paper page
- A. Chernyavskiy, D. Ilvovsky, P. Nakov, WhatTheWikiFact: Fact-checking claims against wikipedia, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 4690–4695. · DOI
- M. Nadeem, W. Fang, B. Xu, M. Mohtarami, J. Glass, FAKTA: An automatic end-to-end fact checking system, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 78–83. · DOI
- H. Karimi, P. Roy, S. Saba-Sadiya, J. Tang, Multi-source multi-class fake news detection, in: Proceedings of the 27th International Conference on Computational Linguistics, 2018, pp. 1546–1557.
- N. Hassan, F. Arslan, C. Li, M. Tremayne, Toward automated fact-checking: Detecting check-worthy factual claims by claimbuster, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 1803–1812. · DOI · Paper page
- K. Shu, X. Zhou, S. Wang, R. Zafarani, H. Liu, The role of user profiles for fake news detection, in: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2019, pp. 436–439. · DOI · Paper page
- Alami, 2019, Enhancing unsupervised neural networks based text summarization with word embedding and ensemble learning, Expert Syst. Appl., 123, 195, 10.1016/j.eswa.2019.01.037 · DOI
- Devi, 2021, A hybrid ensemble word embedding based classification model for multi-document summarization process on large multi-domain document sets, Int. J. Adv. Comput. Sci. Appl., 12
- Sidorov, 2014, Soft similarity and soft cosine measure: similarity of features in vector space model, Comput. Y Sistemas, 18
- A. Akbik, T. Bergmann, D. Blythe, K. Rasul, S. Schweter, R. Vollgraf, FLAIR: An easy-to-use framework for state-of-the-art NLP, in: NAACL 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), 2019, pp. 54–59.
- Artetxe, 2019, Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond, Trans. Assoc. Comput. Linguist., 7, 597, 10.1162/tacl_a_00288 · DOI · Paper page
- Aquino, 2015, Keyword identification in spanish documents using neural networks, J. Comput. Sci. Technol., 15
- Campos, 2020, Yake! keyword extraction from single documents using multiple local features, Inform. Sci., 509, 257, 10.1016/j.ins.2019.09.013 · DOI · Paper page
Cited by
A review of deep learning‐based approaches for deepfake content detection
(2024) · doi:10.1111/exsy.13570 · Cited by 116
Logiqa 2.0—An improved dataset for logical reasoning in natural language understanding
Liu, H., Liu, J., Cui, L., Teng, Z., Duan, N., Zhou, M., & Zhang, Y. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3293046 · Cited by 26
Srba, I., Razuvayevskaya, O., Leite, J. A., Moro, R., Schlicht, I. B., Tonelli, S., García, F. M., Lottmann, S. B., Teyssou, D., Porcellini, V., et al. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3770077 · Cited by 19
Metadata
- Authors
- Martín, A., Huertas-Tato, J., Huertas-García, Á., Villar-Rodríguez, G., & Camacho, D.
- Venue
- Knowledge-Based Systems
- Publisher
- Elsevier BV
- Published
- 2022-09
- Volume
- 251
- Pages
- 109265
- ISSN
- 0950-7051 (ISSN-L)
- OpenAlex
- W3209749143
- Cited by
- 61
- References
- 25
- FWCI
- 19.72
- Type
- journal article
- Language
- en
- Open access
- hybrid
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: FacTeR-Check: Semi-automated fact-checking through semantic similarity and natural language inference. https://app.cochat.ai/papers/10.1016/j.knosys.2022.109265
- The paper
- Martín, A., Huertas-Tato, J., Huertas-García, Á., Villar-Rodríguez, G., & Camacho, D. (2022). FacTeR-Check: Semi-automated fact-checking through semantic similarity and natural language inference. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2022.109265
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.