Generating fact checking briefs
Open accessAbstract
Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, Sebastian Riedel. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In real-world scenarios such as the one RARR operates in, relevant evidence may not be provided, necessitating retrieval.
“In real-world scenarios such as the one which RARR operates in, relevant evidence may not be provided, necessitating retrieval (Fan et al., 2020;Piktus et al., 2021).”
From RARR: Researching and revising what language models say, using language models (2023) · cites this paper as Fan et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575 · see citing works
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · see citing works
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · see citing works
Unified Language Model Pre-training for Natural Language Understanding and Generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019) · arXiv · doi:10.48550/arxiv.1905.03197 · Cited by 1,225 · see citing works
Learning to Ask: Neural Question Generation for Reading Comprehension
Du, X., Shao, J., & Cardie, C. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1123 · Cited by 696 · see citing works
Fact Checking: Task definition and dataset construction
Vlachos, A., & Riedel, S. (2014) · Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science · doi:10.3115/v1/w14-2508 · Cited by 402 · see citing works
Self-citationELI5: long form question answering
Fan, A., Jernite, Y., Perez, E., Grangier, D., Weston, J., & Auli, M. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1346 · Cited by 319 · see citing works
Self-citationLearning Together Slowly: Bayesian Learning about Political Facts
Hill, S. J. (2017) · The Journal of Politics · doi:10.1086/692739 · Cited by 159 · cited 1× · see citing works
“Analysis shows accuracy can improve by providing feedback (Hill, 2017), additional time (Bago et al., 2020), tooling (Kar-duni et al., 2019), or training (Zhang et al., 2018).”
Flagged first, then by citations
Research works printed but not linked: 31
- Michelle A Amazeen. 2016. Checking the fact- checkers in 2008: Predicting political ad scrutiny and assessing consistency. Journal of Political Mar- keting, 15(4):433-464.
- Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Chris- tian Hansen, and Jakob Grue Simonsen. 2019. MultiFC: A real-world multi-domain dataset for evidence-based fact checking of claims. In Proceed- ings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Inter- national Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), pages 4685-4697, Hong Kong, China. Association for Computational Lin- guistics. · DOI
- Bence Bago, David G Rand, and Gordon Pennycook. 2020. Fake news, fast and slow: Deliberation re- duces belief in false (but not true) news headlines. Journal of experimental psychology: general.
- Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. 2017. Question generation for question answering. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 866-874.
- Andreas Hanselowski, Christian Stab, Claudia Schulz, Zile Li, and Iryna Gurevych. 2019. A richly anno- tated corpus for different tasks in automated fact- checking. In Proceedings of the 23rd Confer- ence on Computational Natural Language Learning (CoNLL), pages 493-503, Hong Kong, China. Asso- ciation for Computational Linguistics. · DOI
- Naeemul Hassan, Bill Adair, James Hamilton, Chengkai Li, Mark Tremayne, Jun Yang, and Cong Yu. 2015. The quest to automate fact-checking. Pro- ceedings of the 2015 Computation + Journalism Symposium.
- Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Trans- former architectures and pre-training strategies for fast and accurate multi-sentence scoring. CoRR abs/1905.01969. External Links: Link Cited by, 2:2- 2. · arXiv
- Mayank Jobanputra. 2019. Unsupervised question an- swering for fact-checking. In Proceedings of the Second Workshop on Fact Extraction and VERifica- tion (FEVER), pages 52-56, Hong Kong, China. As- sociation for Computational Linguistics. · DOI
- Alireza Karduni. 2019. Human-misinformation in- teraction: Understanding the interdisciplinary ap- proach needed to computationally combat false in- formation. arXiv preprint arXiv:1903.07136. · arXiv
- Alireza Karduni, Isaac Cho, Ryan Wesslen, Sashank Santhanam, Svitlana Volkova, Dustin L Arendt, Samira Shaikh, and Wenwen Dou. 2019. Vulnera- ble to misinformation? verifi! In Proceedings of the 24th International Conference on Intelligent User In- terfaces, pages 312-323.
- Elena Kochkina, Maria Liakata, and Arkaitz Zubi- aga. 2018. All-in-one: Multi-task learning for ru- mour verification. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 3402-3413, Santa Fe, New Mexico, USA. As- sociation for Computational Linguistics.
- Nayeon Lee, Chien-Sheng Wu, and Pascale Fung. 2018. Improving large-scale fact-checking using decom- posable attention models and lexical tagging. In Pro- ceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pages 1133- 1138, Brussels, Belgium. Association for Computa- tional Linguistics. · DOI
- Sizhen Li, Shuai Zhao, Bo Cheng, and Hao Yang. 2018. An end-to-end multi-task learning model for fact checking. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pages 138-144, Brussels, Belgium. Association for Com- putational Linguistics. · DOI
- Chloe Lim. 2018. Checking how fact-checkers check. Research & Politics, 5(3):2053168018786848.
- Morgan Marietta, David C Barker, and Todd Bowser. 2015. Fact-checking polarized politics: Does the fact-check industry provide consistent guidance on disputed realities? In The Forum, volume 13, pages 577-596.
- Sebastião Miranda, David Nogueira, Afonso Mendes, Andreas Vlachos, Andrew Secker, Rebecca Garrett, Jeff Mitchel, and Zita Marinho. 2019. Automated fact checking in the news room. In The World Wide Web Conference, pages 3579-3583.
- Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics (Demonstrations), pages 48-53.
- Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2018. DeClarE: Debunking fake news and false claims using evidence-aware deep learning. In Proceedings of the 2018 Confer- ence on Empirical Methods in Natural Language Processing, pages 22-32, Brussels, Belgium. Asso- ciation for Computational Linguistics. · DOI
- Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 2383-2392.
- Sudha Rao and Hal Daumé III. 2018. Learning to ask good questions: Ranking clarification questions us- ing neural expected value of perfect information. In Proceedings of the 56th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 2737-2746, Melbourne, Aus- tralia. Association for Computational Linguistics. · DOI
- Benjamin Riedel, Isabelle Augenstein, Georgios P Sp- ithourakis, and Sebastian Riedel. 2017. A sim- ple but tough-to-beat baseline for the fake news challenge stance detection task. arXiv preprint arXiv:1707.03264. · arXiv
- Kevin Roitero, Michael Soprano, Shaoyang Fan, Dami- ano Spina, Stefano Mizzaro, and Gianluca Demar- tini. 2020. Can the crowd identify misinformation objectively? the effects of judgment scale and asses- sor's background. In Proceedings of the 43rd Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR).
- Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing sys- tems, pages 3104-3112.
- Duyu Tang, Nan Duan, Tao Qin, Zhao Yan, and Ming Zhou. 2017. Question answering and ques- tion generation as dual tasks. arXiv preprint arXiv:1706.02027. · arXiv
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics. · DOI · Paper page
- Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Ja- son Weston. 2019. Learning to speak and act in a fantasy text adventure game. In Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 673-683.
- William Yang Wang. 2017. "liar, liar pants on fire": A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the As- sociation for Computational Linguistics (Volume 2: Short Papers), pages 422-426, Vancouver, Canada. Association for Computational Linguistics. · DOI
- Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2019. Zero-shot entity linking with dense entity retrieval. arXiv preprint arXiv:1911.03814. · arXiv
- Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2369-2380.
- Amy X Zhang, Aditya Ranganathan, Sarah Emlen Metz, Scott Appling, Connie Moon Sehat, Norman Gilmore, Nick B Adams, Emmanuel Vincent, Jen- nifer Lee, Martin Robbins, et al. 2018. A struc- tured response to misinformation: Defining and an- notating credibility indicators in news articles. In Companion Proceedings of the The Web Conference 2018, pages 603-612.
- Yao Zhao, Xiaochuan Ni, Yuanyuan Ding, and Qifa Ke. 2018. Paragraph-level neural question genera- tion with maxout pointer and gated self-attention net- works. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3901-3910.
Cited by
Sorted by citations
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.741 · Cited by 407 · see citing works
Internet-Augmented Dialogue Generation
Komeili, M., Shuster, K., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.579 · Cited by 192 · see citing works
RARR: Researching and revising what language models say, using language models
Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V., Lao, N., Lee, H., Juan, D.-C., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.910 · Cited by 141 · see citing works
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
Schuster, T., Fisch, A., & Barzilay, R. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.52 · Cited by 126 · see citing works
Large Language Models Help Humans Verify Truthfulness – Except When They Are Convincingly Wrong
Si, C., Goyal, N., Wu, T., Zhao, C., Feng, S., Daumé Iii, H., & Boyd-Graber, J. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.81 · Cited by 26 · see citing works
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2305.14251 · Cited by 14 · see citing works
“There also has been work that verifies a longer sentence or text through decomposition to atomic facts (Fan et al., 2020; Wright et al., 2022; Chen et al., 2022; Kamoi et al., 2023) from which we take inspiration.”
BackgroundFactcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers
Wang, Y., Reddy, R. G., Mujahid, Z. M., Arora, A., Rubashevskii, A., Geng, J., Afzal, O. M., Pan, L., Borenstein, N., Pillai, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.09000 · Cited by 3 · see citing works
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Krishna, K., Bransom, E., Kuehl, B., Iyyer, M., Dasigi, P., Cohan, A., & Lo, K. (2023) · arXiv · doi:10.48550/arxiv.2301.13298 · Cited by 3
“Prior work (Nakov et al., 2021) attempts to simplify the human fact checking process by methods like knowledge source snip-pets (Fan et al., 2020), similar to hint highlights (§3.3).”
MethodInfluential
- Authors
- Fan, A., Piktus, A., Petroni, F., Wenzek, G., Saeidi, M., Vlachos, A., Bordes, A., & Riedel, S.
- Affiliations
- Meta (United States) (US)
- University of Cambridge (GB)
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 7147-7161
- OpenAlex
- W3105753178
- Cited by
- 42
- References
- 10
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 6 · 2023: 7 · 2022: 10 · 2021: 9
- FWCI
- 13.79
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- fact-checking, automated fact-checking, summarization, abstractive summarization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
