The Fact Extraction and VERification (FEVER) Shared Task
Open accessAbstract
We present the results of the first Fact Extraction and VERification (FEVER) Shared Task.The task challenged participants to classify whether human-written factoid claims could be SUPPORTED or REFUTED using evidence retrieved from Wikipedia.We received entries from 23 competing teams, 19 of which scored higher than the previously published baseline.The best performing system achieved a FEVER score of 64.21%.In this paper, we present the results of the shared task and a summary of the systems, highlighting commonalities and innovations among participating systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing fact-checking approaches either require access to the output probability distribution or external databases interfaced via separate modules.
“Existing fact-checking approaches either require access to the output probability distribution (which may not be available for systems such as ChatGPT) or external databases that are interfaced via separate, often complex, modules.”
From SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023) · cites this paper as Thorne et al., 2018
The Introduction and §4.1 describe FEVER systems checking claims against evidence retrieved from Wikipedia, and Appendix A.2 describes a multistage retrieval and inference system. This supports the external-source-and-modules part of the claim, but the paper does not establish that existing approaches generally require either that setup or access to an output probability distribution; it also does not address ChatGPT. (LLM, conf=0.85)
Experimental — accuracy not yet measuredFact-verification approaches require evidence retrieved from an external database, and facts can only be assessed relative to the knowledge present in the database.
“An alternate approach is to leverage fact-verification approaches, where evidence is retrieved from an external database to assess the veracity of a claim (Thorne et al., 2018; Guo et al., 2022). However, facts can only be assessed relative to the knowledge present in the database.”
From SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023) · cites this paper as Thorne et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 3× · see citing works
“, 2018) and WordNet (Miller, 1995) and Ohio State report using vector representations of named entities.”
BackgroundMethodDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
A Decomposable Attention Model for Natural Language Inference
Parikh, A. P., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1244 · Cited by 1,453 · see citing works
Extended Boolean information retrieval
Salton, G., Fox, E. A., & Wu, H. (1983) · Communications of the ACM · doi:10.1145/182.358466 · Cited by 1,058 · see citing works
SciTaiL: A Textual Entailment Dataset from Science Question Answering
Khot, T., Sabharwal, A., & Clark, P. E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12022 · Cited by 416 · see citing works
Recognizing textual entailment: Rational, evaluation and approaches
Dagan, I., Dolan, B., Magnini, B., & Roth, D. (2009) · Natural Language Engineering · doi:10.1017/s1351324909990209 · Cited by 194 · see citing works
Flagged first, then by citations
Research works printed but not linked: 12
- Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. A simple but tough to beat baseline for sentence em- beddings. In Iclr, pages 1-14.
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
- Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to Answer Open- Domain Questions. Proc. of ACL'17.
- Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2016. Enhanced LSTM for Natural Language Inference. In Proceedings of the 55th Annual Meeting of the Association for Compu- tational Linguistics, pages 1657-1668, Vancouver, Canada.
- Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. In Proceed- ings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670-680, Copenhagen, Denmark.
- Matt J Kusner, Yu Sun, Nicholas I Kolkin, and Kilian Q Weinberger. 2015. From Word Embeddings To Doc- ument Distances. Proceedings of The 32nd Inter- national Conference on Machine Learning, 37:957- 966.
- Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of NAACL-HLT, pages 2227-2237, New Orleans, Louisiana.
- Alec Radford and Tim Salimans. 2018. Improv- ing Language Understanding by Generative Pre- Training. arXiv, pages 1-12.
- Vladimir Svetnik, Andy Liaw, Christopher Tong, J Christopher Culberson, Robert P Sheridan, and Bradley P Feuston. 2003. Random forest: a clas- sification and regression tool for compound classi- fication and qsar modeling. Journal of chemical in- formation and computer sciences, 43(6):1947-1958.
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. In NAACL-HLT.
- Yi Yang and Ming-Wei Chang. 2015. S-MART: Novel Tree-based Structured Learning Algorithms Applied to Tweet Entity Linking. In Proceedings of the 53rd Annual Meeting of the Association for Compu- tational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 504-513, Beijing, China.
- Wenpeng Yin, Hinrich Schütze, and Dan Roth. 2018. End-Task Oriented Textual Entailment via Deep Ex- plorations of Inter-Sentence Interactions. In Pro- ceedings of the 56th Annual Meeting of the Associ- ation for Computational Linguistics (Short Papers), pages 540-545, Melbourne, Australia.
Cited by
Sorted by citations
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257 · see citing works
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.557 · Cited by 557 · see citing works
A Survey on Automated Fact-Checking
Guo, Z., Schlichtkrull, M., & Vlachos, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00454 · Cited by 500 · see citing works
Self-citationApproximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Xiong, L., Xiong, C., Li, Y., Tang, K., Liu, J., Bennett, P. N., Ahmed, J., & Overwikj, A. (2021) · International Conference on Learning Representations · Cited by 376 · see citing works
KILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.200 · Cited by 357 · see citing works
Self-citationCOVID-19 Misinformation Online and Health Literacy: A Brief Overview
Naeem, S. B., & Boulos, M. N. K. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18158091 · Cited by 296 · see citing works
Automated Fact-Checking for Assisting Human Fact-Checkers
Nakov, P., Corney, D. P. A., Hasanain, M., Alam, F., Elsayed, T., Barrón‐Cedeño, A., Papotti, P., Shaar, S., & Martino, G. D. S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/619 · Cited by 231 · see citing works
Fine-grained Fact Verification with Kernel Graph Attention Network
Liu, Z., Xiong, C., Sun, M., & Liu, Z. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.655 · Cited by 220 · see citing works
Interpretable Rumor Detection in Microblogs by Attending to User Interactions
Khoo, L. M. S., Chieu, H. L., Zhong, Q., & Jiang, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6405 · Cited by 210 · see citing works
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification
Zhou, J., Han, X., Yang, C. H., Liu, Z., Wang, L., Li, C., & Sun, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1085 · Cited by 206 · see citing works
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
Xiong, L., Xiong, C., Li, Y., Tang, K.-F., Liu, J., Bennett, P., Ahmed, J., & Overwijk, A. (2020) · arXiv · doi:10.48550/arxiv.2007.00808 · Cited by 204 · see citing works
COVIDLies: Detecting COVID-19 Misinformation on Social Media
Hossain, T., Logan, R. L., Ugarte, D. A., Matsubara, Y., Young, S. D., & Singh, S. (2020) · Workshop on NLP for COVID-19 (NLP-COVID19) · doi:10.18653/v1/2020.nlpcovid19-2.11 · Cited by 204 · see citing works
Digital media and misinformation: An outlook on multidisciplinary strategies against manipulation
Caled, D., & Silva, M. J. (2021) · Journal of Computational Social Science · doi:10.1007/s42001-021-00118-8 · Cited by 170 · see citing works
Reasoning Over Semantic-Level Graph for Fact Checking
Zhong, W., Xu, J., Tang, D., Xu, Z., Duan, N., Zhou, M., Wang, J., & Yin, J. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.549 · Cited by 151 · see citing works
Re2G: Retrieve, Rerank, Generate
Glass, M., Rossiello, G., Chowdhury, M. F. M., Naik, A., Cai, P., & Gliozzo, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.194 · Cited by 122 · see citing works
Sentence-Level Evidence Embedding for Claim Verification with Hierarchical Attention Networks
Ma, J., Gao, W. G., Joty, S., & Wong, K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1244 · Cited by 110 · see citing works
The limitations of stylometry for detecting machine-generated fake news
Schuster, T., Schuster, R., Shah, D. J., & Barzilay, R. (2020) · Computational Linguistics · doi:10.1162/coli_a_00380 · Cited by 109 · see citing works
Fighting post-truth using natural language processing: A review and open challenges
Saquete, E., Tomás, D., Moreda, P., Martínez-Barco, P., & Palomar, M. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112943 · Cited by 106 · see citing works
Zoom Out and Observe: News Environment Perception for Fake News Detection
Sheng, Q., Cao, J., Zhang, X., Li, R., Wang, D., & Zhu, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.311 · Cited by 103 · see citing works
Towards Debiasing NLU Models from Unknown Biases
Utama, P. A., Moosavi, N. S., & Gurevych, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.613 · Cited by 100 · see citing works
Automated fact‐checking: A survey
Zeng, X., Abumansour, A. S., & Zubiaga, A. (2021) · Language and Linguistics Compass · doi:10.1111/lnc3.12438 · Cited by 100 · see citing works
Aly, R., Guo, Z., Schlichtkrull, M., Thorne, J. P., Vlachos, A., Christodoulopoulos, C., Cocarascu, O., & Mittal, A. (2021) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2021.fever-1.1 · Cited by 100 · see citing works
Self-citationA Richly Annotated Corpus for Different Tasks in Automated Fact-Checking
Hanselowski, A., Stab, C., Schulz, C., Li, Z., & Gurevych, I. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1046 · Cited by 99 · see citing works
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction
Nishida, K., Nishida, K., Nagata, M., Otsuka, A., Saito, I., Asano, H., & Tomita, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1225 · Cited by 97 · see citing works
Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance
Utama, P. A., Moosavi, N. S., & Gurevych, I. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.770 · Cited by 94 · see citing works
CoVerifi: A COVID-19 news verification system
Kolluri, N. L., & Murthy, D. (2021) · Online Social Networks and Media · doi:10.1016/j.osnem.2021.100123 · Cited by 81 · see citing works
The perils and promises of fact-checking with large language models
Quelle, D., & Bovet, A. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1341697 · Cited by 80 · see citing works
TRUE: Re-evaluating Factual Consistency Evaluation
Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., Scialom, T., Szpektor, I., Hassidim, A., & Matias, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.287 · Cited by 75 · see citing works
Crowdsourced Fact-Checking at Twitter
Saeed, M., Traub, N., Nicolas, M., Demartini, G., & Papotti, P. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557279 · Cited by 72 · see citing works
Fake news detection on social media using a natural language inference approach
Sadeghi, F., Bidgoly, A. J., & Amirkhani, H. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12428-8 · Cited by 71 · see citing works
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval
Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., & Wei, F. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.125 · Cited by 71 · see citing works
The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models
Li, J., Chen, J., Ren, R., Cheng, X., Zhao, X., Nie, J.-Y., & Wen, J.-R. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.586 · Cited by 70 · see citing works
Thorne, J. P., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., & Mittal, A. (2019) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/d19-6601 · Cited by 68 · see citing works
Self-citationGERE: Generative Evidence Retrieval for Fact Verification
Chen, J., Zhang, R., Guo, J., Fan, Y., & Cheng, X. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531827 · Cited by 66 · see citing works
Integrating Pattern- and Fact-based Fake News Detection via Model Preference Learning
Sheng, Q., Zhang, X., Cao, J., & Zhong, L. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482440 · Cited by 63 · see citing works
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60 · see citing works
TRUE: Re-evaluating Factual Consistency Evaluation
Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., Scialom, T., Szpektor, I., Hassidim, A., & Matias, Y. (2022) · Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering · doi:10.18653/v1/2022.dialdoc-1.19 · Cited by 56 · see citing works
Park, C. Y., Mendelsohn, J., Field, A., & Tsvetkov, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.382 · Cited by 53 · see citing works
ProoFVer: Natural Logic Theorem Proving for Fact Verification
Krishna, A., Riedel, S., & Vlachos, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00503 · Cited by 51 · see citing works
Self-citationThe impact of emotional signals on credibility assessment
Giachanou, A., Rosso, P., & Crestani, F. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24480 · Cited by 50 · see citing works
(2020) · doi:10.18653/v1/2020.fever-1.5 · Cited by 49 · see citing works
A Review on Fact Extraction and Verification
Bekoulis, G., Papagiannopoulou, C., & Deligiannis, N. (2023) · ACM Computing Surveys · doi:10.1145/3485127 · Cited by 48 · see citing works
Jhamtani, H., & Clark, P. E. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.10 · Cited by 43 · see citing works
Ravichandran, B. D., & Keikhosrokiani, P. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07797-y · Cited by 39 · see citing works
A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems
Thomson, C., & Reiter, E. (2020) · Proceedings of the 13th International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.22 · Cited by 38 · see citing works
An intelligent cybersecurity system for detecting fake news in social media websites
Mughaid, A. S., AlZu’bi, S. M. F., Al-Arjan, A., AL-Amrat, R., AL-Ajmi, R., Zitar, R. A., & Abualigah, L. (2022) · Soft Computing · doi:10.1007/s00500-022-07080-1 · Cited by 38 · see citing works
Thanasi-Boçe, M., & Hoxha, J. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12775-z · Cited by 38 · see citing works
Team DOMLIN: Exploiting Evidence Enhancement for the FEVER Shared Task
Stammbach, D., & Neumann, G. (2019) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/d19-6616 · Cited by 37 · see citing works
HHGN: A Hierarchical Reasoning-based Heterogeneous Graph Neural Network for fact verification
Chen, C., Cai, F., Hu, X., Chen, W., & Chen, H. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102659 · Cited by 37 · see citing works
FACE-KEG: Fact Checking Explained using KnowledgE Graphs
Vedula, N., & Parthasarathy, S. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441828 · Cited by 35 · see citing works
When classification accuracy is not enough: Explaining news credibility assessment
Przybyła, P., & Soto, A. J. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102653 · Cited by 35 · see citing works
Constraint 2021: Machine Learning Models for COVID-19 Fake News Detection Shared TaskPreprint
Felber, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03717 · Cited by 35 · see citing works
Debiasing NLU Models via Causal Intervention and Counterfactual Reasoning
Tian, B., Cao, Y., Zhang, Y., & Xing, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21389 · Cited by 34 · see citing works
AraStance: A Multi-Country and Multi-Domain Dataset of Arabic Stance Detection for Fact Checking
Alhindi, T., Alabdulkarim, A., Alshehri, A., Abdul-Mageed, M., & Nakov, P. (2021) · Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda (NLP4IF) · doi:10.18653/v1/2021.nlp4if-1.9 · Cited by 33 · see citing works
TFGIN: Tight-Fitting Graph Inference Network for Table-based Fact Verification
Wu, L., Wang, K., Nie, K., Guo, S., Gao, C., Wang, Z., & Li, S. (2025) · ACM Transactions on Information Systems · doi:10.1145/3734520 · Cited by 32 · see citing works
Online Disinformation and the Role of WikipediaPreprint
Sáez-Trumper, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12596 · Cited by 32 · see citing works
Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verification
Subramanian, S., & Lee, K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.627 · Cited by 30 · see citing works
Natural language inference for curation of structured clinical registries from unstructured text
Percha, B., Pisapati, K. V., Gao, C., & Schmidt, H. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab243 · Cited by 29 · see citing works
More Robust Dense Retrieval with Contrastive Dual Learning
Li, Y., Liu, Z., Xiong, C., & Liu, Z. (2021) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3471158.3472245 · Cited by 28 · see citing works
Early multi-class ensemble-based fake news detection using content features
Rezaei, S., Kahani, M., Behkamal, B., & Jalayer, A. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-01019-y · Cited by 28 · see citing works
FaVIQ: FAct Verification from Information-seeking Questions
Park, J., Min, S., Kang, J., Zettlemoyer, L., & Hajishirzi, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.354 · Cited by 28 · see citing works
ExpertQA: Expert-Curated Questions and Attributed Answers
Malaviya, C., Lee, S., Chen, S., Sieber, E., Yatskar, M., & Roth, D. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.167 · Cited by 28 · see citing works
An Empirical Comparison of BERT, RoBERTa, and Electra for Fact Verification
Naseer, M., Asvial, M., & Sari, R. F. (2021) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic51459.2021.9415192 · Cited by 27 · see citing works
Sheng, Q., Cao, J., Zhang, X., Li, X., & Zhong, L. (2021) · doi:10.18653/v1/2021.acl-long.425 · Cited by 27 · see citing works
An entity-graph based reasoning method for fact verification
Chen, C., Cai, F., Hu, X., Zheng, J., Ling, Y., & Chen, H. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2020.102472 · Cited by 26 · see citing works
Factoring Fact-Checks: Structured Information Extraction from Fact-Checking Articles
Jiang, S., Baumgartner, S., Ittycheriah, A., & Yu, C. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380231 · Cited by 24 · see citing works
Scientific Fact-Checking: A Survey of Resources and Approaches
Vladika, J., & Matthes, F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.387 · Cited by 24 · see citing works
Multimodal Automated Fact-Checking: A Survey
Akhtar, M., Schlichtkrull, M., Guo, Z., Cocarascu, O., Simperl, E. P. B., & Vlachos, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.361 · Cited by 24 · see citing works
Self-citationAnswerFact: Fact Checking in Product Question Answering
Zhang, W., Deng, Y., Ma, J., & Lam, W. P. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.188 · Cited by 23 · see citing works
Linked Credibility Reviews for Explainable Misinformation Detection
Denaux, R., & Gómez-Pérez, J. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62419-4_9 · Cited by 23 · see citing works
Li, J., Faragó, D., Petrov, C., & Ahmed, I. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643760 · Cited by 22 · see citing works
Adversarial attacks against Fact Extraction and VERificationPreprint
Thorne, J. P., & Vlachos, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.05543 · Cited by 22 · see citing works
Self-citationFocusing on Relevant Responses for Multi-Modal Rumor Detection
Li, J., Bin, Y., Peng, L., Yang, Y., Li, Y., Jin, H., & Huang, Z. H. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3389694 · Cited by 21 · see citing works
Scientific Discourse Tagging for Evidence Extraction
Li, X., Burns, G., & Peng, N. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.218 · Cited by 20 · see citing works
The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
Li, M., Popa, D. N., Chagnon, J., Cinar, Y. G., & Gaussier, É. (2022) · ACM Transactions on Information Systems · doi:10.1145/3568394 · Cited by 18 · see citing works
OpenMatch: An Open Source Library for Neu-IR Research
Liu, Z., Zhang, K., Xiong, C., Liu, Z., & Sun, M. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462789 · Cited by 17 · see citing works
Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data
Li, X., Liu, Z., Xiong, C., Shi, Y., Gu, Y., Liu, Z., & Yu, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.734 · Cited by 16 · see citing works
Joint Models for Answer Verification in Question Answering Systems
Zhang, Z., Vu, T., & Moschitti, A. (2021) · doi:10.18653/v1/2021.acl-long.252 · Cited by 15 · see citing works
Enhancing Descriptive Image Captioning with Natural Language Inference
Shi, Z., Liu, H., & Zhu, X. (2021) · doi:10.18653/v1/2021.acl-short.36 · Cited by 15 · see citing works
Chernyavskiy, A., Ilvovsky, D., & Nakov, P. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481987 · Cited by 15 · see citing works
Barrón‐Cedeño, A., Alam, F., Chakraborty, T., Elsayed, T., Nakov, P., Przybyła, P., Struß, J. M., Haouari, F., Hasanain, M., Ruggeri, F., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56069-9_62 · Cited by 14 · see citing works
A DQN-based Approach to Finding Precise Evidences for Fact Verification
Wan, H., Chen, H., Du, J., Luo, W., & Ye, R. (2021) · doi:10.18653/v1/2021.acl-long.83 · Cited by 14 · see citing works
On the Role of Relevance in Natural Language Processing Tasks
Sauchuk, A., Thorne, J. P., Halevy, A. Y., Tonellotto, N., & Silvestri, F. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532034 · Cited by 14 · see citing works
Self-citationELECTRA-based graph network model for multi-hop question answering
Zhu, P., Yuan, Y., & Chen, L. (2023) · Journal of Intelligent Information Systems · doi:10.1007/s10844-023-00800-5 · Cited by 14 · see citing works
Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder
Liu, Z., Zhang, H., Xiong, C., Liu, Z., Gu, Y., & Li, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.384 · Cited by 13 · see citing works
Marche, C., Cabiddu, I., Castangia, C. G., Serreli, L., & Nitti, M. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3311629 · Cited by 13 · see citing works
Evaluating factual accuracy in complex data-to-text
Thomson, C., Reiter, E., & Sundararajan, B. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101482 · Cited by 12 · see citing works
Extract and Aggregate: A Novel Domain-Independent Approach to Factual Data Verification
Chernyavskiy, A., & Ilvovsky, D. (2019) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/d19-6612 · Cited by 12 · see citing works
A Deep Transfer Learning Approach for Fake News Detection
Saikh, T., Haripriya, B., Ekbal, A., & Bhattacharyya, P. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207477 · Cited by 12 · see citing works
Trokhymovych, M., & Sáez-Trumper, D. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481961 · Cited by 12 · see citing works
Explainability of Automated Fact Verification Systems: A Comprehensive Review
Vallayil, M., Nand, P., Yan, W., & Allende‐Cid, H. (2023) · Applied Sciences · doi:10.3390/app132312608 · Cited by 11 · see citing works
Joint Rumour Stance and Veracity Prediction
Lillie, A. E., Middelboe, E. R., & Derczynski, L. (2019) · DSpace repository (University of Tartu) · Cited by 11 · see citing works
Technological Approaches to Detecting Online Disinformation and Manipulation
Horák, A., Baisa, V., & Herman, O. (2021) · Political campaigning and communication · doi:10.1007/978-3-030-58624-9_5 · Cited by 11 · see citing works
Bouziane, M., Perrin, H., Sadeq, A., Nguyen, T., Cluzeau, A., & Mardas, J. (2021) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2021.fever-1.4 · Cited by 11 · see citing works
Ashktorab, Z., Desmond, M., Pan, Q., Johnson, J., Brachman, M., Dugan, C., Danilevsky, M., & Geyer, W. (2025) · Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work · doi:10.1145/3729176.3729179 · Cited by 10 · see citing works
Simple Compounded-Label Training for Fact Extraction and Verification
Nie, Y., Bauer, L., & Bansal, M. (2020) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/2020.fever-1.1 · Cited by 10 · see citing works
A Neural Model to Jointly Predict and Explain Truthfulness of Statements
Brand, E., Roitero, K., Soprano, M., Rahimi, A., & Demartini, G. (2022) · Journal of Data and Information Quality · doi:10.1145/3546917 · Cited by 10 · see citing works
CsFEVER and CTKFacts: acquiring Czech data for fact verification
Ullrich, H., Drchal, J., Rýpar, M., Vincourová, H., & Moravec, V. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09654-3 · Cited by 10 · see citing works
Fast and Accurate Factual Inconsistency Detection Over Long Documents
Lattimer, B. M., Chen, P. M., Zhang, X., & Yang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.105 · Cited by 10 · see citing works
ECOL: Early Detection of COVID Lies Using Content, Prior Knowledge and Source Information
Schlicht, I. B., & Boukhers, Z. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-73696-5_14 · Cited by 10 · see citing works
- Authors
- Thorne, J., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., & Mittal, A.
- Affiliations
- University of Sheffield (GB)
- Imperial College London (GB)
- Amazon (United States) (US)
- Venue
- Proceedings of the First Workshop on Fact Extraction and VERification (FEVER)
- Published
- 2018
- Pages
- 1-9
- OpenAlex
- W2902421513
- Cited by
- 212
- References
- 6
- Citations by year
- 2026: 7 · 2025: 19 · 2024: 21 · 2023: 27 · 2022: 32 · 2021: 53 · 2020: 27 · 2019: 25 · 2018: 1
- FWCI
- 10.82
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Fact Extraction and VERification Workshop (FEVER) (publishedVersion, cc-by)
- arXiv (Cornell University) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Horizon 2020 Framework Programme (EU‐H2020)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- fact-checking, fact extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
