FEVER: a Large-scale Dataset for Fact Extraction and VERification
Open accessAbstract
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Arpit Mittal. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some works heuristically select and pair real sentences or tables as source and target when collecting large-scale datasets.
“When collecting large-scale datasets, some works heuristically select and pair real sentences or tables as the source and target [132, 283].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [132]
In §3.1 (Claim Generation; page not available), the paper says it sampled real Wikipedia sentences and asked annotators to generate claims from them. This supports selecting source sentences for a large-scale dataset, but the targets were newly written claims, not selected real sentences or tables. (LLM, conf=0.86)
Experimental — accuracy not yet measuredFEVER claims are synthetic, created by annotators by mutating sentences from Wikipedia articles that will serve as evidence.
“On the other hand, claims in the popular FEVER dataset (Thorne et al., 2018) are synthetic , since they are created by annotators by mutating sentences from the Wikipedia articles that will serve as evidence.”
From Fact or Fiction: Verifying Scientific Claims (2020) · cites this paper as Thorne et al., 2018
Section 3.1 (page unavailable) says annotators generated claims from sampled Wikipedia sentences and produced mutated versions, supporting the description of FEVER claims as synthetic. But it also says the dataset includes both extracted and mutated claims. The Introduction and Section 3.2 explain that evidence was selected in a separate verification step, so the source sentences used to create claims were not necessarily the sentences that served as evidence. (LLM, conf=0.87)
Experimental — accuracy not yet measuredScientific claim verification datasets typically treat claims as standalone statements and assume open-domain retrieval or citation-based reasoning across multiple sources, with evaluation centered on short claims whose supporting evidence is relatively concentrated.
“While these settings share similarities with review verification, they typically treat claims as standalone statements and assume open-domain retrieval or citation-based reasoning across multiple sources, with evaluation centered on short claims whose supporting evidence is relatively concentrated.”
From Peerify: Benchmarking Peer-Review Claim Verification (2026) · cites this paper as Thorne et al., 2018
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredPast benchmarks for information retrieval mostly focus on easily found information and have become saturated by recent language models.
“While past benchmarks have measured the ability to retrieve information (Joshi et al., 2017;Yang et al., 2018;Thorne et al., 2018;Dinan et al., 2019;Fan et al., 2019;Mialon et al., 2023), most of these benchmarks focus on retrieving information that can be found easily, and hence have become saturated by recent language models.”
From BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents (2025) · cites this paper as Thorne et al., 2018
The Introduction identifies FEVER as a claim-verification dataset requiring systems to retrieve supporting or refuting sentences, so it supports citing FEVER as a retrieval benchmark. The paper does not support the broader assertion that most such benchmarks involve easily found information or have become saturated by recent language models; its Introduction reports that some FEVER claims require evidence from multiple pages. (LLM, conf=0.86)
Experimental — accuracy not yet measuredRetrieval helps evaluation by providing up-to-date knowledge.
“It is commonly used in factual error detection (Min et al., 2023; Wei et al., 2024; Thorne et al., 2018) and helps evaluation by providing up-to-date knowledge.”
From VeriScore: Evaluating the factuality of verifiable claims in long-form text generation (2024) · cites this paper as Thorne et al., 2018
Sections 5.2 (Evaluation) and 5.3 (Document Retrieval) describe retrieving evidence to verify claims and evaluate whether a system returns the correct evidence. They do not say retrieval provides up-to-date knowledge; Section 3.1 describes using a June 2017 Wikipedia dump. Page numbers are unavailable in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Evidence-based Factual Error Correction (EFEC) implements a full research-and-revise workflow trained on Wikipedia passages.
“Evidence-based Factual Error Correction (EFEC; Thorne and Vlachos, 2021) also implements a full research-and-revise workflow trained on Wikipedia passages (Thorne et al., 2018).”
From RARR: Researching and revising what language models say, using language models (2023) · cites this paper as Thorne et al., 2018
The Introduction and §3.1 (page numbers unavailable) establish that FEVER’s claims and verification evidence come from Wikipedia. The cited paper does not address EFEC or establish that EFEC was trained on those passages. The research-and-revise workflow is attributed to the separate 2021 citation, not this one. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe approach is inspired by works on fact-checking where simple research-and-revise workflows are effective at attributing or correcting unattributed claims made by humans.
“The approach is inspired by works on fact-checking 2 where simple research-and-revise workflows are effective at attributing or correcting unattributed claims made by humans (Thorne et al., 2018;Schuster et al., 2021;Thorne and Vlachos, 2021).”
From RARR: Researching and revising what language models say, using language models (2023) · cites this paper as Thorne et al., 2018
Section 3.2 (Task 2—Claim Labeling; page not supplied) describes annotators finding Wikipedia evidence to support or refute human-generated claims. This supports citing FEVER as fact-checking work, but the paper does not establish that a research-and-revise workflow is effective at attributing or correcting unattributed human claims. (LLM, conf=0.82)
Experimental — accuracy not yet measuredMost literature leverages Wikipedia as the knowledge source, which is only a small part of world knowledge.
“Most of the literature leverages Wikipedia as the knowledge source [135, 249, 300], which is only a small part of world knowledge.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [249]
SPOT extends verification to the full complexity of frontier-level scientific research, unlike prior evaluation suites that focus on sentence-level fact checks or reproducing noisy peer-review feedback.
“It should be noted that, whereas prior evaluation suites focus on sentence-level fact checks of everyday knowledge [26, 27] or on reproducing noisy peer-review feedback [28, 29], SPOT extends verification to the full complexity of frontier-level scientific research.”
From When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research (2025) · cites this paper as [26]
Prior fact verification benchmarks concentrate on claims at the sentence level and rely on text inputs only to assess consistency with reference documents.
“Prior fact verification benchmarks [26, 27] concentrate on claims at the sentence level and rely on text inputs only to assess consistency with reference documents.”
From When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research (2025) · cites this paper as [26]
General-domain fact-checking datasets like FEVER are abundantly available, but generalize poorly to scientific claims.
“General-domain fact-checking datasets like FEVER (Thorne et al., 2018) are abundantly available, but generalize poorly to scientific claims (see §6.1).”
From MultiVerS: Improving scientific claim verification with weak supervision and full-document context (2022) · cites this paper as Thorne et al., 2018
Label agreement for 232 independently re-annotated claim-abstract pairs is 0.75 Cohen's κ.
“We assign 232 claim-abstract pairs for independent re-annotation. The label agreement is 0.75 Cohen's κ , comparable with the 0.68 Fleiss' κ reported in Thorne et al. (2018), and 0.70 Cohen's κ reported in Hanselowski et al. (2019).”
From Fact or Fiction: Verifying Scientific Claims (2020) · cites this paper as Thorne et al. (2018)
FEVER is a large-scale benchmark designed for evaluating factual consistency and claim verification in natural language.
“FEVER (Fact Extraction and VERification) dataset (Thorne et al., 2018) is a large-scale benchmark designed for evaluating factual consistency and claim verification in natural language.”
From UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models (2025) · cites this paper as Thorne et al., 2018
Several benchmarks have been proposed to evaluate the performance of fact checking in both the general domain and the scientific domain.
“Building on this pipeline, several benchmarks have been proposed to evaluate the performance of fact checking in both the general domain Thorne et al. (2018); Ma et al. (2024) and the scientific domain Wadden et al. (2020; 2022); Ho et al. (2025).”
From ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks (2025) · cites this paper as Thorne et al. (2018)
This released mixture uses SNLI, MNLI, FEVER, SciTail, PAWS, and VitaminC rather than the original TRUE paper's ANLI recipe.
“This released mixture uses SNLI, MNLI, FEVER, SciTail, PAWS, and VitaminC (Bowman et al., 2015;Williams et al., 2018;Thorne et al., 2018;Khot et al., 2018;Zhang et al., 2019;Schuster et al., 2021), rather than the original TRUE paper's ANLI recipe.”
From The Attribution-Compression Frontier in Retrieval-Augmented Generation (2026) · cites this paper as Thorne et al., 2018
Retrieval is commonly used in factual error detection.
“It is commonly used in factual error detection (Min et al., 2023; Wei et al., 2024; Thorne et al., 2018) and helps evaluation by providing up-to-date knowledge.”
From VeriScore: Evaluating the factuality of verifiable claims in long-form text generation (2024) · cites this paper as Thorne et al., 2018
FEVER features synthesized incorrect sentences from Wikipedia.
“FEVER (Thorne et al., 2018) features synthesized incorrect sentences from Wikipedia.”
From VeriScore: Evaluating the factuality of verifiable claims in long-form text generation (2024) · cites this paper as Thorne et al., 2018
In WIKIBIO, the authors took the Wikipedia infobox as the source and the first sentence of the Wikipedia page as the target ground-truth reference.
“For instance, when constructing WIKIBIO [132], a dataset for generating biographical notes based on the infoboxes of Wikipedia, the authors took the Wikipedia infobox as the source and the first sentence of the Wikipedia page as the target ground-truth reference.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [132]
Section 3.1 describes generating FEVER claims from sentences in Wikipedia introductory sections; section 3.2 describes using those sections as evidence. Searches for WIKIBIO, infoboxes, and biographical-note references found no passage describing WIKIBIO’s source–target construction. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
Reading Wikipedia to answer open-domain questions
Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · arXiv · doi:10.48550/arxiv.1606.05250 · Cited by 678 · see citing works
Flagged first, then by citations
Research works printed but not linked: 18
- Gabor Angeli and Christopher D. Manning. 2014. Nat- uralLI: Natural logic inference for common sense reasoning. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Pro- cessing. pages 534-545.
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large anno- tated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
- Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. 2009. Recognizing textual entail- ment: Rational, evaluation and approaches. Natural Language Engineering 15(4):i-xvii. https://doi.org/10.1017/S1351324909990209. · DOI
- Joe Ellis, Jeremy Getman, Dana Fore, Neil Kuster, Zhiyi Song, Ann Bies, and Stephanie Strassel. 2016. Overview of Linguistic Resources for the TAC KBP 2016 Evaluations : Methodologies and Results. Pro- ceedings of TAC KBP 2016 Workshop, National In- stitute of Standards and Technology, Maryland, USA (Ldc).
- William Ferreira and Andreas Vlachos. 2016. Emer- gent: a novel data-set for stance classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. San Diego, California, pages 1163-1168.
- Michael Heilman and Noah A. Smith. 2010. Good Question! statistical ranking for question genera- tion. In Proceedings of the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. pages 609-617.
- Mio Kobayashi, Ai Ishii, Chikara Hoshino, Hiroshi Miyashita, and Takuya Matsuzaki. 2017. Auto- mated historical fact-checking by passage retrieval, word statistics, and virtual question-answering. In Proceedings of the Eighth International Joint Con- ference on Natural Language Processing (Volume 1: Long Papers). volume 1, pages 967-975.
- Edward Loper and Steven Bird. 2002. Nltk: The natural language toolkit. In Proceedings of the ACL-02 Workshop on Effective Tools and Method- ologies for Teaching Natural Language Process- ing and Computational Linguistics -Volume 1. As- sociation for Computational Linguistics, Strouds- burg, PA, USA, ETMTNLP '02, pages 63-70. https://doi.org/10.3115/1118108.1118117. · DOI
- Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc- Closky. 2014. The Stanford CoreNLP natural lan- guage processing toolkit. In ACL (System Demon- strations). pages 55-60.
- Mausam, Michael Schmitz, Robert Bart, Stephen Soderland, and Oren Etzioni. 2012. Open language learning for information extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. pages 523-534.
- Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceed- ings of the 2016 Conference on Empirical Meth- ods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2249-2255. https://aclweb.org/anthology/D16- 1244.
- Benjamin Riedel, Isabelle Augenstein, George Sp- ithourakis, and Sebastian Riedel. 2017. A simple but tough-to-beat baseline for the Fake News Chal- lenge stance detection task. CoRR abs/1707.03264. http://arxiv.org/abs/1707.03264. · arXiv
- Tim Rocktäschel and Sebastian Riedel. 2017. End- to-end differentiable proving. In Advances in Neu- ral Information Processing Systems 31: Annual Conference on Neural Information Processing Sys- tems 2017, December 4-9, 2017, Long Beach, California, United States. volume abs/1705.11040. http://arxiv.org/abs/1705.11040. · arXiv
- Álvaro Rodrigo, Anselmo Peñas, and Felisa Verdejo. 2009. Overview of the answer validation exercise 2008. In Carol Peters, Thomas Deselaers, Nicola Ferro, Julio Gonzalo, Gareth J. F. Jones, Mikko Ku- rimo, Thomas Mandl, Anselmo Peñas, and Vivien Petras, editors, Evaluating Systems for Multilingual and Multimodal Information Access: 9th Workshop of the Cross-Language Evaluation Forum. pages 296-313.
- Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional at- tention flow for machine comprehension. CoRR abs/1611.01603. http://arxiv.org/abs/1611.01603. · arXiv
- Niket Tandon, Gerard de Melo, and Gerhard Weikum. 2011. Deriving a Web-scale common sense fact database. In Proceedings of the 25th AAAI Confer- ence on Artificial Intelligence (AAAI 2011). AAAI Press, Palo Alto, CA, USA, pages 152-157.
- Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Lan- guage Technologies and Computational Social Sci- ence. ACL. http://www.aclweb.org/anthology/W14- 2508.
- William Yang Wang. 2017. "Liar, Liar Pants on Fire": A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). http://aclweb.org/anthology/P17- 2067.
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
“Most of the literature leverages Wikipedia as the knowledge source [72, 132], which is only a small part of world knowledge.”
BackgroundRecent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387 · see citing works
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876 · see citing works
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · see citing works
“A related line of research is automated fact-checking (Thorne et al., 2018; Aly et al., 2021; Baly et al., 2018), where the focus is on evaluation of statements rather than generation.”
BackgroundA survey on fake news and rumour detection techniques
Bondielli, A., & Marcelloni, F. (2019) · Information Sciences · doi:10.1016/j.ins.2019.05.035 · Cited by 658 · see citing works
Adversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656 · see citing works
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569 · see citing works
ERASER: A Benchmark to Evaluate Rationalized NLP Models
DeYoung, J., Jain, S., Rajani, N. F., Lehman, E., Xiong, C., Socher, R., & Wallace, B. C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.408 · Cited by 561 · see citing works
Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach
Yin, W., Hay, J., & Roth, D. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1404 · Cited by 537 · see citing works
Combating Fake News: A Survey on Identification and Mitigation Techniques
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3305260 · Cited by 514 · see citing works
Fact or Fiction: Verifying Scientific Claims
Wadden, D., Lin, S., Lo, K., Wang, L. L., van Zuylen, M., Cohan, A., & Hajishirzi, H. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.609 · Cited by 382 · see citing works
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.741 · Cited by 407 · see citing works
Deep Learning applications for COVID-19
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal of Big Data · doi:10.1186/s40537-020-00392-9 · Cited by 350 · see citing works
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., & Zaharia, M. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.272 · Cited by 353 · see citing works
KILT: a Benchmark for Knowledge Intensive Language Tasks
Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., De Cao, N., Thorne, J., Jernite, Y., Karpukhin, V., Maillard, J., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.200 · Cited by 357 · see citing works
Self-citationDynabench: Rethinking Benchmarking in NLP
Kiela, D., Bartolo, M., Nie, Y., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.324 · Cited by 254 · see citing works
<scp>SummaC</scp>: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization
Laban, P., Schnabel, T., Bennett, P. N., & Hearst, M. A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00453 · Cited by 238 · see citing works
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2304.06364 · Cited by 211 · see citing works
Enabling Large Language Models to Generate Text with Citations
Gao, T., Yen, H., Yu, J., & Chen, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.398 · Cited by 216 · see citing works
Editing Factual Knowledge in Language Models
De Cao, N., Aziz, W., & Titov, I. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.522 · Cited by 180 · see citing works
The Internal State of an LLM Knows When It’s Lying
Azaria, A., & Mitchell, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.68 · Cited by 175 · see citing works
MetaICL: Learning to Learn In Context
Min, S., Lewis, M., Zettlemoyer, L., & Hajishirzi, H. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.201 · Cited by 169 · see citing works
Expel: LLM agents are experiential learners
Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29936 · Cited by 153 · see citing works
Reasoning Over Semantic-Level Graph for Fact Checking
Zhong, W., Xu, J., Tang, D., Xu, Z., Duan, N., Zhou, M., Wang, J., & Yin, J. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.549 · Cited by 151 · see citing works
Improving Text Embeddings with Large Language Models
Wang, L., Yang, N., Huang, X., Yang, L., Majumder, R., & Wei, F. (2024) · arXiv · doi:10.48550/arxiv.2401.00368 · Cited by 149 · see citing works
RARR: Researching and revising what language models say, using language models
Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V., Lao, N., Lee, H., Juan, D.-C., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.910 · Cited by 141 · see citing works
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141 · see citing works
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.149 · Cited by 211 · see citing works
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale
Nie, Y., Wang, S., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1258 · Cited by 132 · see citing works
Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.11511 · Cited by 101 · see citing works
“…from a couple of knowledge-intensive datasets, Natural Questions (Kwiatkowski et al., 2019), Wizard of Wikipedia (Dinan et al., 2019) and FEVER (Thorne et al., 2018) from the KILT benchmark (Petroni et al., 2021), ASQA (Stel-makh et al., 2022) and multiple QA datasets including ARC-Easy and…”
BackgroundAnnotating and Modeling Fine-grained Factuality in Summarization
Goyal, T., & Durrett, G. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.114 · Cited by 100 · see citing works
COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic
Saakyan, A., Chakrabarty, T., & Muresan, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.165 · Cited by 91 · see citing works
AlignScore: Evaluating factual consistency with a unified alignment function
Zha, Y., Yang, Y., Li, R., & Hu, Z. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.634 · Cited by 81 · see citing works
GERE: Generative Evidence Retrieval for Fact Verification
Chen, J., Zhang, R., Guo, J., Fan, Y., & Cheng, X. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531827 · Cited by 66 · see citing works
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation
Liu, T., Zhang, Y., Brockett, C., Mao, Y., Sui, Z., Chen, W., & Dolan, B. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.464 · Cited by 63 · see citing works
Measuring Attribution in Natural Language Generation Models
Rashkin, H., Nikolaev, V., Lamm, M., Aroyo, L., Collins, M., Das, D., Petrov, S., Tomar, G. S., Turc, I., & Reitter, D. (2023) · Computational Linguistics · doi:10.1162/coli_a_00486 · Cited by 71 · see citing works
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.11738 · Cited by 57 · see citing works
“Our method is also inspired by fact-checking in journalism (Wang, 2017) that assesses whether a claim made by a human is true (Thorne et al., 2018).”
MethodCorpusBrain: Pre-train a Generative Retrieval Model for Knowledge-Intensive Language Tasks
Chen, J., Zhang, R., Guo, J., Liu, Y., Fan, Y., & Cheng, X. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557271 · Cited by 57 · see citing works
MultiVerS: Improving scientific claim verification with weak supervision and full-document context
Wadden, D., Lo, K., Wang, L., Cohan, A., Beltagy, I., & Hajishirzi, H. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.6 · Cited by 60 · see citing works
“F EVER (Thorne et al., 2018) P UBMED QA (Jin et al., 2019) was released to facilitate question-answering over biomedical research abstracts.”
BackgroundInfluentialCan We Edit Factual Knowledge by In-Context Learning?
Zheng, C., Li, L., Dong, Q., Fan, Y., Wu, Z., Xu, J., & Chang, B. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.296 · Cited by 54 · see citing works
Teaching language models to support answers with verified quotes
Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., Song, F., Chadwick, M., Glaese, M., Young, S., Campbell-Gillingham, L., Irving, G., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.11147 · Cited by 53 · see citing works
Towards Continual Knowledge Learning of Language Models
Jang, J., Ye, S., Yang, S., Shin, J., Han, J., Kim, G., Choi, S. J., & Seo, M. (2021) · arXiv · doi:10.48550/arxiv.2110.03215 · Cited by 49 · see citing works
“Configuration KILT (Petroni et al., 2021) consists of 5 different tasks and 11 datasets: OpenDomain Question Answering (Joshi et al., 2017; Kwiatkowski et al., 2019; Fan et al., 2019; Yang et al., 2018), Fact Checking (Thorne et al., 2018), Entity Linking (Hoffart et al., 2011; Guo & Barbosa, 2018), Slot-filling (Levy…”
BackgroundDialFact: A Benchmark for Fact-Checking in Dialogue
Gupta, P., Wu, C.-S., Liu, W., & Xiong, C. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.263 · Cited by 48 · see citing works
Task-aware Retrieval with Instructions
Asai, A., Schick, T., Lewis, P., Chen, X., Izacard, G., Riedel, S., Hajishirzi, H., & Yih, W.-T. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.225 · Cited by 46 · see citing works
From language to action: a review of large language models as autonomous agents and tool users
Chowa, S. S., Alvi, R., Rahman, S. S., Rahman, M. A., Raiaan, M. A. K., Islam, M. R., Hussain, M., & Azam, S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11471-9 · Cited by 50 · see citing works
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38 · see citing works
BREAKING! Presenting Fake News Corpus for Automated Fact Checking
Pathak, A., & Srihari, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/p19-2050 · Cited by 39 · see citing works
A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems
Thomson, C., & Reiter, E. (2020) · Proceedings of the 13th International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.22 · Cited by 38 · see citing works
Read before Generate! Faithful Long Form Question Answering with Machine Reading
Su, D., Li, X., Zhang, J., Shang, L., Jiang, X., Liu, Q., & Fung, P. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.61 · Cited by 37 · see citing works
A Unified Generative Retriever for Knowledge-Intensive Language Tasks via Prompt Learning
Chen, J., Zhang, R., Guo, J., de Rijke, M., Liu, Y., Fan, Y., & Cheng, X. (2023) · Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3539618.3591631 · Cited by 32 · see citing works
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Laban, P., Vig, J., Hearst, M., Xiong, C., & Wu, C.-S. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676419 · Cited by 31 · see citing works
SciFact-Open: Towards open-domain scientific claim verification
Wadden, D., Lo, K., Kuehl, B., Cohan, A., Beltagy, I., Wang, L. L., & Hajishirzi, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.347 · Cited by 35 · see citing works
RA-DIT: Retrieval-Augmented Dual Instruction Tuning
Lin, X. V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01352 · Cited by 26 · see citing works
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29 · see citing works
“fact-checking (Thorne et al., 2018b) (where the knowledge base is a large source like Wikipedia) and summarization (Kryscinski et al.”
BackgroundStretching Sentence-pair NLI Models to Reason over Long Documents and Clusters
Schuster, T., Chen, S., Buthpitiya, S., Fabrikant, A., & Metzler, D. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.28 · Cited by 17 · see citing works
The Internal State of an LLM Knows When It's Lying
Azaria, A., & Mitchell, T. (2023) · arXiv · doi:10.48550/arxiv.2304.13734 · Cited by 179 · see citing works
PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
Zou, W., Geng, R., Wang, B., & Jia, J. (2024) · arXiv · doi:10.48550/arxiv.2402.07867 · Cited by 16 · see citing works
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2305.14251 · Cited by 14 · see citing works
“Our work is closely related to prior work on fact verification (Thorne et al., 2018; Wadden et al., 2020) where claim sentences are automatically checked against a large knowledge source like Wikipedia or scientific literature.”
BackgroundA Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
Jacovi, A., Bitton, Y., Bohnet, B., Herzig, J., Honovich, O., Tseng, M., Collins, M., Aharoni, R., & Geva, M. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.254 · Cited by 14 · see citing works
Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection
Sheokand, T., Jain, G., Bahga, A., & Madisetti, V. K. (2025) · IEEE Access · doi:10.1109/access.2025.3561693 · Cited by 7 · see citing works
When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Research
Son, G., Hong, J., Fan, H., Nam, H., Ko, H., Lim, S., Song, J., Choi, J., Paulo, G., Yu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.11855 · Cited by 5 · see citing works
“Prior fact verification benchmarks [26, 27] concentrate on claims at the sentence level and rely on text inputs only to assess consistency with reference documents.”
Eldifrawi, I., Wang, S., & Trabelsi, A. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.361 · Cited by 5 · see citing works
Understanding Faithfulness and Reasoning of Large Language Models on Plain Biomedical Summaries
Fang, B., Dai, X., & Karimi, S. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.578 · Cited by 4 · see citing works
SciClaimHunt: A Large Dataset for Evidence-based Scientific Claim Verification
Kumar, S., Sharma, A., Khincha, S. H., Shroff, G., Singh, S. R., & Mishra, R. (2025) · arXiv · doi:10.48550/arxiv.2502.10003 · Cited by 3 · see citing works
Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers
Wang, Y., Reddy, R. G., Mujahid, Z. M., Arora, A., Rubashevskii, A., Geng, J., Afzal, O. M., Pan, L., Borenstein, N., Pillai, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.09000 · Cited by 3 · see citing works
Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations
Tilwani, D., Saxena, Y., Mohammadi, S., Padia, A., Raff, E., Sheth, A., Parthasarathy, S., & Gaur, M. (2024) · arXiv · doi:10.48550/arxiv.2405.02228 · Cited by 2 · see citing works
“Benchmarks such as Attributed Question Answering (AQA) [12], ALCE [13], and related knowledge-grounding and fact-verification benchmarks [14, 19] assess whether generated responses or claims are supported by relevant evidence.”
BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents
Wei, J., Sun, Z., Papay, S., McKinney, S., Han, J., Fulford, I., Chung, H. W., Passos, A. T., Fedus, W., & Glaese, A. (2025) · arXiv · doi:10.48550/arxiv.2504.12516 · Cited by 2 · see citing works
“…prior work that focuses on information that could be easily found by a human within ten minutes of searching (Joshi et al., 2017; Yang et al., 2018; Thorne et al., 2018; Dinan et al., 2019; Lee et al., 2019; Fan et al., 2019; Petroni et al., 2021; Yao et al., 2022; Mialon et al., 2023), BrowseComp…”
Web Retrieval Agents for Evidence-Based Misinformation Detection
Tian, J.-J., Yu, H., Orlovskiy, Y., Vergho, T., Rivera, M., Goel, M., Yang, Z., Godbout, J.-F., Rabbany, R., & Pelrine, K. (2024) · arXiv · doi:10.48550/arxiv.2409.00009 · Cited by 2 · see citing works
ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks
Li, M., Zeng, Y., Cheng, Z., Ma, C., & Jia, K. (2025) · arXiv · doi:10.48550/arxiv.2508.15804 · Cited by 1 · see citing works
“Building on this pipeline, several benchmarks have been proposed to evaluate the performance of fact checking in both the general domain Thorne et al. (2018); Ma et al. (2024) and the scientific domain Wadden et al. (2020; 2022); Ho et al. (2025).”
SEDM: Scalable Self-Evolving Distributed Memory for Agents
Xu, H., Hu, J., Zhang, K., Yu, L., Tang, Y., Song, X., Duan, Y., Ai, L., & Shi, B. (2025) · arXiv · doi:10.48550/arxiv.2509.09498
“Experiments are conducted on two widely used benchmark datasets: FEVER [43] and HotpotQA [49].”
Phantom References: Hallucinated Citations That Survive Peer Review at Top-Tier Conferences
Russinovich, M., Kumar, R. S. S., & Salem, A. (2026) · arXiv · doi:10.48550/arxiv.2607.00738
“…whether a statement is supported by its source [25], evaluating verifiability in generative search engines [26], and training models to emit inline citations grounded in retrieved evidence [27], [28], on the foundation of automated fact verification [29] and alongside AI-text detection [30], [31].”
Zhang, C., Wan, Z., Yu, X., Wu, J., Zhou, Y., Tsang, I., & An, B. (2026) · arXiv
“For transfer, we use 292 balanced FEVER claims [Thorne et al., 2018], where a random guess is right half the time.”
InfluentialPeerify: Benchmarking Peer-Review Claim Verification
Daghighfarsoodeh, A., Ebrahimi, S., Ghorbanpour, A., Sadeghian, S., Cheraghi, R., Arabzadeh, N., & Bagheri, E. (2026) · arXiv · doi:10.48550/arxiv.2609.25046
“Peerify is also related to scientific claim verification and evidence-based fact checking, studied extensively through textual entailment and natural language inference, from early RTE-style formulations (Dagan et al., 2006) to large-scale benchmarks such as FEVER (Thorne et al., 2018), SciFact (Wadden et al., 2020),…”
InfluentialAccurate and Efficient Long-Term Memory for LLM Agents
Zhao, Z., Guo, X., Lv, L., Wang, M., Li, M., & Li, S. (2026) · arXiv · doi:10.48550/arxiv.2607.16211
“Fact verification systems [Thorne et al., 2018] and fine-grained factual evaluation [Min et al., 2023] address hallucination at generation time.”
BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
Zhou, P., Leon, B., Ying, X., Zhang, C., Shao, Y., Ye, Q., Chong, D., Jin, Z., Xie, C., Cao, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.19314
“To assess retrieval capabilities, a variety of widely used English benchmarks have been proposed, including TriviaQA Joshi et al. (2017), HotpotQA Yang et al. (2018), FEVER Thorne et al. (2018), KILT Petroni et al. (2020), and GAIA Mialon et al. (2023).”
UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models
Gu, X., Huang, Z., Gu, J.-C., & Zhang, K. (2025) · arXiv · doi:10.48550/arxiv.2505.14679
Synapse: Empowering LLM Agents with Episodic-Semantic Memory via Spreading Activation
Jiang, H., Chen, J., Pan, Y., Chen, L., You, W., Zhou, Y., Zhang, R., Abate, Y., & Liu, T. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1108
AlignCheck: a Semantic Open-Domain Metric for Factual Consistency Assessment
Aghaebrahimian, A. (2025) · arXiv · doi:10.48550/arxiv.2512.03634
“Decomposing text into constituents such as entities (Lee et al., 2022) or triples (subject, predicate, object) is exercised in several other works (Goodrich et al., 2019; Thorne et al., 2018; Min et al., 2023) as a direct and model-free alternative to validate factual consistency of generated texts.”
A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation
Chen, J., Chen, W., Lin, Z., & Vong, C. M. (2026) · arXiv · doi:10.48550/arxiv.2608.24419
“The adjacent verification literature asks whether a claim is supported [49, 51]; our construct begins where that one ends, since every item in the T C arm is supported on both sides of the pair and differs only in what it covers or how hard it commits.”
Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attribution
Saxena, Y., Bommireddy, R., Padia, A., & Gaur, M. (2025) · arXiv · doi:10.48550/arxiv.2509.21557
Fu, Y., Kang, Y., & Zhao, Y. (2026) · arXiv · doi:10.48550/arxiv.2606.27383
Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models
Qiu, Y., Embar, V. R., Zhang, Y., Jaitly, N., Cohen, S. B., & Han, B. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.165
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Krishna, K., Bransom, E., Kuehl, B., Iyyer, M., Dasigi, P., Cohan, A., & Lo, K. (2023) · arXiv · doi:10.48550/arxiv.2301.13298 · Cited by 3
“Faithfulness in summarization differs from fact verification in three ways: (1) summaries are paragraph-long and contextual compared to single sentence stand-alone claims in fact verification; (2) summaries are grounded to a source document, compared to a large knowledge source in fact verification; (3) summaries are…”
Background
- Authors
- Thorne, J., Vlachos, A., Christodoulopoulos, C., & Mittal, A.
- Affiliations
- University of Sheffield (GB)
- University of Cambridge (GB)
- Amazon (United States) (US)
- Venue
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
- Published
- 2018
- Pages
- 809-819
- OpenAlex
- W2789566302
- Cited by
- 1,343
- References
- 3
- Citations by year
- 2026: 177 · 2025: 177 · 2024: 128 · 2023: 230 · 2022: 196 · 2021: 197 · 2020: 115 · 2019: 87 · 2018: 35 · 2013: 1
- FWCI
- 44.39
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Apollo (cc-by)
- Funders
- Horizon 2020 Framework Programme (EU‐H2020)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- fact extraction, fact-checking, large-scale datasets, fever
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
