The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
The Fact Extraction and VERification (FEVER) Shared Task
Open accessAbstract
We present the results of the first Fact Extraction and VERification (FEVER) Shared Task.The task challenged participants to classify whether human-written factoid claims could be SUPPORTED or REFUTED using evidence retrieved from Wikipedia.We received entries from 23 competing teams, 19 of which scored higher than the previously published baseline.The best performing system achieved a FEVER score of 64.21%.In this paper, we present the results of the shared task and a summary of the systems, highlighting commonalities and innovations among participating systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing fact-checking approaches either require access to the output probability distribution (which may not be available for systems such as ChatGPT) or external databases that are interfaced via separate, often complex, modules.”
From SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023) · cites this paper as Thorne et al., 2018
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“An alternate approach is to leverage fact-verification approaches, where evidence is retrieved from an external database to assess the veracity of a claim (Thorne et al., 2018; Guo et al., 2022). However, facts can only be assessed relative to the knowledge present in the database.”
From SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023) · cites this paper as Thorne et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 3×
“, 2018) and WordNet (Miller, 1995) and Ohio State report using vector representations of named entities.”
BackgroundMethodDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
A Decomposable Attention Model for Natural Language Inference
Parikh, A. P., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1244 · Cited by 1,351
Extended Boolean information retrieval
Salton, G., Fox, E. A., & Wu, H. (1983) · Communications of the ACM · doi:10.1145/182.358466 · Cited by 1,058
SciTaiL: A Textual Entailment Dataset from Science Question Answering
Khot, T., Sabharwal, A., & Clark, P. E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12022 · Cited by 419
Recognizing textual entailment: Rational, evaluation and approaches
Dagan, I., Dolan, B., Magnini, B., & Roth, D. (2009) · Natural Language Engineering · doi:10.1017/s1351324909990209 · Cited by 194
Research works printed but not linked: 12
- Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. A simple but tough to beat baseline for sentence em- beddings. In Iclr, pages 1-14.
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
- Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to Answer Open- Domain Questions. Proc. of ACL'17.
- Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2016. Enhanced LSTM for Natural Language Inference. In Proceedings of the 55th Annual Meeting of the Association for Compu- tational Linguistics, pages 1657-1668, Vancouver, Canada.
- Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. In Proceed- ings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670-680, Copenhagen, Denmark.
- Matt J Kusner, Yu Sun, Nicholas I Kolkin, and Kilian Q Weinberger. 2015. From Word Embeddings To Doc- ument Distances. Proceedings of The 32nd Inter- national Conference on Machine Learning, 37:957- 966.
- Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of NAACL-HLT, pages 2227-2237, New Orleans, Louisiana.
- Alec Radford and Tim Salimans. 2018. Improv- ing Language Understanding by Generative Pre- Training. arXiv, pages 1-12.
- Vladimir Svetnik, Andy Liaw, Christopher Tong, J Christopher Culberson, Robert P Sheridan, and Bradley P Feuston. 2003. Random forest: a clas- sification and regression tool for compound classi- fication and qsar modeling. Journal of chemical in- formation and computer sciences, 43(6):1947-1958.
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. In NAACL-HLT.
- Yi Yang and Ming-Wei Chang. 2015. S-MART: Novel Tree-based Structured Learning Algorithms Applied to Tweet Entity Linking. In Proceedings of the 53rd Annual Meeting of the Association for Compu- tational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 504-513, Beijing, China.
- Wenpeng Yin, Hinrich Schütze, and Dan Roth. 2018. End-Task Oriented Textual Entailment via Deep Ex- plorations of Inter-Sentence Interactions. In Pro- ceedings of the 56th Annual Meeting of the Associ- ation for Computational Linguistics (Short Papers), pages 540-545, Melbourne, Australia.
Cited by
Sorted by citations
Items 201–219 of 219
Czech-ing the News: Article Trustworthiness Dataset for Czech
Boháček, M., Bravansky, M., Trhlík, F., & Moravec, V. (2023) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2023.wassa-1.10
CNGT: Co-attention Networks with Graph Transformer for Fact Verification
Yuan, J., Chen, C., Hou, C., & Yuan, X. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46664-9_39
Recent Advancements in Misinformation Detection
Ekbal, A., & Kumari, R. (2024) · The information retrieval series · doi:10.1007/978-3-031-54834-5_2
TAR: A Think-Action-Reflection Framework for Complex Problem Solving with Large Language Models
Wei, Z., Zhang, Q., Duan, Q., Wang, G., Li, R., Li, X., Chen, Q., Yang, T., Zhang, L., & Jiang, K. (2024) · International Conference on Modeling, Natural Language Processing and Machine Learning (CMNM) · doi:10.1145/3677779.3677825
Chernyavskiy, A., Ilvovsky, D., & Nakov, P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_20
A Comprehensive Review of Fake News Detection Approaches Using Machine Learning
Okorie, K. M., Okebanama, U. F., & Ellam, I. V. (2025) · European Journal of Theoretical and Applied Sciences · doi:10.59324/ejtas.2025.3(2).41
Deep One-Directional Neural Semantic Siamese Network for High-Accuracy Fact Verification
Naseer, M., Windiatmaja, J. H., Asvial, M., & Sari, R. F. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9070172
Learning to mine all minimal evidences for unverified claims
Liu, Y., Wan, H., Du, J., Wang, Y., Qi, K., & Luo, W. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122505
Enhancing Hallucination Detection in Large Language Models by Internal-External Feature Fusion
Wang, Q., Liu, F., Li, L., & Wang, J. (2025) · International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) · doi:10.1109/cyberc66434.2025.00039
MIRAGE: A Metrics lIbrary for Rating hAllucinations in Generated tExt
Vendeville, B., Магдановна, Е. Л., Loor, P. D., & Kamps, J. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761644 · Cited by 2
Using Machine Learning to Detect Scientific Misinformation
Impey, C., Danehy, A., Wenger, M. C., & Buxner, S. R. (2025) · Astronomy Education Journal · doi:10.32374/2025.5.1.138aer
Decoding Deception: Machine Learning for Fake News Detection in Arab Social Media
Nimer, M. A., Alshraideh, M., & AlDajani, I. M. (2026) · Springer eBooks · doi:10.1007/978-3-032-01118-3_32
Fact-Checking with Large Language Models via Cost-Effective First-Order Logic Reformulation
Asghari, S., Lakshmanan, L. V. S., Srinivasan, V., & Thomo, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-14107-1_6
Employing Transformers and Humans for Textual-Claim Verification
Saeed, M. (2022) · theses.fr (ABES) · doi:10.70675/3e02a3a6z8281z4d84z8f38ze617eaaf969b
InScribe: Intelligent Augmentation of Natural-Language Statements with Data Facts
Chen, C., Liu, Y., Shi, D., Cao, S., & Cao, N. (2026) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis68791.2026.00021
Tracing the truth: a critical review of propagation tracking strategies in social networks
Ganguly, M., & Dey, P. (2026) · Journal of Computational Social Science · doi:10.1007/s42001-026-00497-w
Aarthy, N., & Velammal, B. L. (2026) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2026.2699912
Metadata
- Authors
- Thorne, J., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., & Mittal, A.
- Affiliations
- University of Sheffield (GB)
- Imperial College London (GB)
- Amazon (United States) (US)
- Venue
- Proceedings of the First Workshop on Fact Extraction and VERification (FEVER)
- Published
- 2018
- Pages
- 1-9
- OpenAlex
- W2902421513
- Cited by
- 212
- References
- 6
- Citations by year
- 2026: 7 · 2025: 19 · 2024: 21 · 2023: 27 · 2022: 32 · 2021: 53 · 2020: 27 · 2019: 25 · 2018: 1
- FWCI
- 10.82
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Fact Extraction and VERification Workshop (FEVER) (publishedVersion, cc-by)
- arXiv (Cornell University) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Horizon 2020 Framework Programme (EU‐H2020)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- fact-checking, fact extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
