The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Evaluating the Factual Consistency of Abstractive Text Summarization
Open accessAbstract
Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents.We propose a weakly-supervised, model-based approach for verifying factual consistency and identifying conflicts between source documents and a generated summary.Training data is generated by applying a series of rule-based transformations to the sentences of source documents.The factual consistency model is then trained jointly for three tasks: 1) identify whether sentences remain factually consistent after transformation, 2) extract a span in the source documents to support the consistency prediction, 3) extract a span in the summary sentence that is inconsistent if one exists.Transferring this model to summaries generated by several state-of-the art models reveals that this highly scalable approach substantially outperforms previous models, including those trained with strong supervision using standard datasets for natural language inference and fact checking.Additionally, human evaluation shows that the auxiliary span extraction tasks provide useful assistance in the process of verifying factual consistency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In most previous works, factuality has been defined as whether a claim in a text can be supported by evidence from a separate, trustworthy knowledge base, with applications in fact-checking (Thorne et al., 2018b) (where the knowledge base is a large source like Wikipedia) and summarization (Kryscinski et al., 2020) (where the knowledge base is an input document or documents).”
From FacTool: Factuality Detection in Generative AI - A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) · cites this paper as Kryscinski et al., 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, hallucinations are observed over a wide range of tasks beyond pure fact verification (Kryscinski et al., 2020; Maynez et al., 2020).”
From SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023) · cites this paper as Kryscinski et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Research suggests that up to 30% of machine-generated abstractive summaries may contain factual inconsistencies [52, 53].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [53]
The cited source says what the claim attributes to it.
In the citing paper
“Results show that the performance of their FactCC model surpasses the classifiers trained on the MNLI or FEVER datasets.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [126]
The cited source says what the claim attributes to it.
In the citing paper
“These methods, such as entity swapping and sentence negation (Kryscinski et al., 2020;Goyal and Durrett, 2021), were designed to target specific error types that occurred in claims from earlier summarization models.”
From MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents (2024) · cites this paper as Kryscinski et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“In response to this challenge, a surge of research explores leveraging data-augmentation methods to construct synthetical data for fine-tuning the classifier, either by rule-based perturbation [79, 152, 266] or generation [389].”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [152]
The cited source says what the claim attributes to it.
In the citing paper
“most prior studies have primarily concentrated on task-specific hallucinations, such as abstractive summarization [81, 102, 152, 208, 236, 310], data-to-text[240, 296], and machine translation [389].”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [152]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This approach has been widely studied in the summarization literature and recent work in factual precision (Kryscinski et al., 2020; Wang et al., 2020; Gao et al., 2022; Manakul et al., 2023).”
From FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023) · cites this paper as Kryscinski et al., 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
(2017) · Cited by 11,625
(2004) · Cited by 9,718
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1044 · Cited by 2,932
Research works printed but not linked: 9
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large an- notated corpus for learning natural language infer- ence. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguis- tics.
- Kyunghyun Cho, Bart van Merrienboer, C ¸aglar Gülc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Nat- ural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1724- 1734.
- Alexis Conneau, Ruty Rinott, Guillaume Lample, Ad- ina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross- lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Nat- ural Language Processing. Association for Compu- tational Linguistics.
- Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. CoRR, abs/1808.09381. · arXiv
- Ramesh Nallapati, Bowen Zhou, C ¸aglar Gülc ¸ehre, Bing Xiang, et al. 2016. Abstractive text summa- rization using sequence-to-sequence rnns and be- yond. Proceedings of SIGNLL Conference on Com- putational Natural Language Learning.
- Ramakanth Pasunuru and Mohit Bansal. 2018. Multi- reward reinforced summarization with saliency and entailment. CoRR, abs/1804.06451. · arXiv
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proc. of NAACL.
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. CoRR, abs/1803.05355. · arXiv
Cited by
Sorted by citations
Items 301–400 of 693
Wang, X., Liu, B., Tang, S., & Wu, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.37 · Cited by 12
Nutri-bullets: Summarizing Health Studies by Composing Segments
Shah, D., Yu, L., Lei, T., & Barzilay, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17624 · Cited by 7
TODSum: Task-Oriented Dialogue Summarization with State TrackingPreprint
Zhao, L., Zheng, F., He, K., Zeng, W., Lei, Y., Jiang, H., Wu, W., Xu, W., Guo, J. Q., & Meng, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.12680 · Cited by 19
PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition
Chen, S., Buthpitiya, S., Fabrikant, A., Roth, D., & Schuster, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.565 · Cited by 10
Can Pre-trained Language Models Generate Titles for Research Papers?
Rehman, T., Sanyal, D. K., & Chattopadhyay, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0865-2_13 · Cited by 10
Cl2sum: abstractive summarization via contrastive prompt constructed by LLMs hallucination
Huang, X., Nong, Q., Wang, X., Zhang, H., Du, K., Yin, C., Li, Y., Yan, B., & Zhang, X. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01795-y · Cited by 7
Trends and Prospects of Retrieval-Augmented Generation (RAG) for Generative AI
Yoon, Y. C., & Kim, S. K. (2025) · The Journal of Korean Association of Computer Education · doi:10.32431/kace.2025.28.2.007 · Cited by 10
Liu, Z., Kou, J., Zhang, W., Gu, C., Fang, X., Huang, Z., Hao, Y., Li, H., Lu, X., Yin, A., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189137 · Cited by 9
Learning with Instance Bundles for Reading Comprehension
Dua, D., Dasigi, P., Singh, S., & Gardner, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.584 · Cited by 9
Differentiable N -gram objective on abstractive summarization
Zhu, Y., Yang, X., Wu, Y., Zhu, M., & Zhang, W. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119367 · Cited by 11
Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Model
Jia, Q., Ren, S., Liu, Y., & Zhu, K. Q. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.679 · Cited by 20
Resource for Error Analysis in Text Simplification: New Taxonomy and Test Collection
Vendeville, B., Магдановна, Е. Л., & Loor, P. D. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730304 · Cited by 7
Is Human Scoring the Best Criteria for Summary Evaluation?
Vasilyev, O., & Bohannon, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.192 · Cited by 8
Summary Explorer: Visualizing the State of the Art in Text Summarization
Syed, S., Yousef, T., Al‐Khatib, K., Jänicke, S., & Potthast, M. (2021) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2021.emnlp-demo.22 · Cited by 8
Leveraging speaker-aware structure and factual knowledge for faithful dialogue summarization
Zhao, L., Xu, W., Zhang, C., & Guo, J. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108550 · Cited by 7
Should We Trust This Summary? Bayesian Abstractive Summarization to The Rescue
Gidiotis, A., & Tsoumakas, G. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.325 · Cited by 10
Lee, H., Yoo, K. M., Park, J., Lee, H., & Jung, K. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.76 · Cited by 9
SWING: Balancing Coverage and Faithfulness for Dialogue Summarization
Huang, K.-H., Singh, S., Ma, X., Xiao, W., Nan, F., Dingwall, N., Wang, W. Y., & McKeown, K. R. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.37 · Cited by 12
Expository Text Generation: Imitate, Retrieve, Paraphrase
Balepur, N., Huang, J., & Chang, K. C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.729 · Cited by 16
Xue, B., Wang, W., Wang, H., Mi, F., Wang, R., Wang, Y., Shang, L., Jiang, X., Liu, Q., & Wong, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.525 · Cited by 17
WebRED: Effective Pretraining And Finetuning For Relation Extraction On The WebPreprint
Ormándi, R., Saleh, M., Winter, E., & Rao, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09681 · Cited by 10
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding
Shi, W., Han, X., Lewis, M., Tsvetkov, Y., Zettlemoyer, L., & Yih, S. W.-T. (2023) · arXiv · doi:10.48550/arxiv.2305.14739 · Cited by 6
ScholarMate: A Mixed-Initiative Tool for Qualitative Knowledge Work and Information Sensemaking
Ye, R., Lee, P. Y. K., Varona, M., Huang, O., & Nobre, C. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3707640.3731913 · Cited by 13
EvalAssist: Insights on Task-Specific Evaluations and AI-Assisted Judgment Strategy Preferences
Ashktorab, Z., Desmond, M. J., Pan, Q., Johnson, J. M., Cooper, M. S., Daly, E., Nair, R., Pedapati, T., Do, H. J., & Geyer, W. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747740 · Cited by 6
LENS: Layers of Evaluation of Hallucination in GenAI Systems
Gu, C., Zhang, W., Huang, Z., Kou, J., Liu, Z., Zhao, C., Liu, C., Zhang, L., Lin, W., Wang, Z., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189150 · Cited by 32
Responsible AI Considerations in Text Summarization Research: A Review of Current Practices
Liu, Y. H., Cao, M., Blodgett, S. L., Cheung, J. C. K., Olteanu, A., & Trischler, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.413 · Cited by 7
Hallucination Reduction in Long Input Text Summarization
Rehman, T., Mandal, R., Agarwal, A., & Sanyal, D. K. (2024) · CRC Press eBooks · doi:10.1201/9781003428459-35 · Cited by 12
Generative AI for Consumer Communications: Classification, Summarization, Response Generation
Correa, N., Correa, A., & Zadrozny, W. (2024) · IEEE ANDESCON · doi:10.1109/andescon61840.2024.10755794 · Cited by 7
Evaluation and Comparison of Open-Source LLMs Using Natural Language Generation Quality Metrics
Hamzic, D., Wurzenberger, M., Skopik, F., Landauer, M., & Rauber, A. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825576 · Cited by 8
Barron, R., Eren, M. E., Serafimova, O. M., Matuszek, C., & Alexandrov, B. S. (2025) · International Conference on Artificial Intelligence and Law · doi:10.1145/3769126.3769215 · Cited by 36
TESA: A Task in Entity Semantic Aggregation for Abstractive Summarization
Jumel, C., Louis, A. P., & Cheung, J. C. K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.646 · Cited by 6
Explainable Assessment of Healthcare Articles with QA
Boissonnet, A., Saeidi, M., Plachouras, V., & Vlachos, A. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.1 · Cited by 6
CORPURES: Benchmark corpus for urdu extractive summaries and experiments using supervised learning
Humayoun, M., & Akhtar, N. (2022) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2022.200129 · Cited by 6
SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages
Laban, P., Vig, J., Kryściński, W., Joty, S., Xiong, C., & Wu, C.-S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.596 · Cited by 7
Self-citationNonFactS: NonFactual Summary Generation for Factuality Evaluation in Document Summarization
Soleimani, A., Monz, C., & Worring, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.400 · Cited by 7
Questioning the Validity of Summarization Datasets and Improving Their Factual Consistency
Guo, Y., Clavel, C., Eddine, M. K., & Vazirgiannis, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.386 · Cited by 9
Siriwardhana, S., Gupta, C., Kaluarachchi, T., Dissanayake, V., Ellawela, S., & Nanayakkara, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2286090 · Cited by 6
Models See Hallucinations: Evaluating the Factuality in Video Captioning
Liu, H., & Wan, X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.723 · Cited by 6
Fast and Accurate Factual Inconsistency Detection Over Long Documents
Lattimer, B. M., Chen, P. M., Zhang, X., & Yang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.105 · Cited by 15
What Makes a Good Podcast Summary?
Rezapour, R., Reddy, S., Jones, R., & Soboroff, I. M. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531802 · Cited by 8
Joint Generator-Ranker Learning for Natural Language Generation
Shen, W., Gong, Y., Shen, Y., Wang, S., Quan, X., Duan, N., & Chen, W. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.486 · Cited by 6
Evaluating and Improving Factuality in Multimodal Abstractive Summarization
Wan, D., & Bansal, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.654 · Cited by 12
Rebuffel, C., Soulier, L., Scoutheeten, G., & Gallinari, P. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.18 · Cited by 7
Sensitivity of BLANC to human-scored qualities of text summariesPreprint
Vasilyev, O., Dharnidharka, V., Egan, N., Chambliss, C., & Bohannon, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06716 · Cited by 8
Accurate, yet inconsistent? Consistency Analysis on Language Understanding ModelsPreprint
Jang, M., Kwon, D. S., & Lukasiewicz, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06665 · Cited by 14
Inspecting the Factuality of Hallucinated Entities in Abstractive Summarization.Preprint
Cao, M., Dong, Y., & Cheung, J. C. K. (2021) · arXiv (Cornell University) · Cited by 6
Proceedings of The 4th Workshop on e-Commerce and NLP
(2021) · doi:10.18653/v1/2021.ecnlp-1 · Cited by 6
Abstractive Text Summarization Using Bert and Adversarial Learning
Prakash, A., Rohith, G., & Umamageswaran, J. (2024) · International Conference on Technological Advancements in Computational Sciences (ICTACS) · doi:10.1109/ictacs62700.2024.10840865 · Cited by 7
Wu, Q., Khan, M. A., Das, S., Nanda, V., Ghosh, B., Kolling, C., Speicher, T., Bindschaedler, L., Gummadi, K. P., & Terzi, E. (2025) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3701551.3703562 · Cited by 14
Massive-scale Decoding for Text Generation using Lattices
Xu, J., Jonnalagadda, S., & Durrett, G. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.344 · Cited by 7
Improving Factual Consistency of Abstractive Summarization on Customer Feedback
Liu, Y., Sun, Y., & Gao, V. (2021) · Workshop on e-Commerce and NLP (ECNLP) · doi:10.18653/v1/2021.ecnlp-1.19 · Cited by 7
Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization
Zeng, Z., Chen, J., Xu, W., & Li, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.337 · Cited by 5
EchoGen: Generating Conclusions from Echocardiogram Notes
Tang, L., Kooragayalu, S., Wang, Y., Ding, Y., Durrett, G., Rousseau, J. F., & Peng, Y. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.35 · Cited by 5
Wang, X., & Zhao, T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.180 · Cited by 5
Recent Advances in Chatbot Algorithms, Techniques, and Technologies
Caldarini, G., & Jaf, S. (2023) · Advances in web technologies and engineering book series · doi:10.4018/978-1-6684-6234-8.ch011 · Cited by 6
Rong, H., Chen, G., Ma, T., Sheng, V. S., & Bertino, E. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3286097 · Cited by 5
SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes
Xu, W., Qian, X., Wang, M., Li, L., & Wang, W. Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.283 · Cited by 7
Revisiting text decomposition methods for NLI-based factuality scoring of summaries
Glover, J., Fancellu, F., Jagannathan, V., Gormley, M. R., & Schaaf, T. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.7 · Cited by 7
Improving Faithfulness by Augmenting Negative Summaries from Fake Documents
Wang, T., Ladhak, F., Durmus, E., & He, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.816 · Cited by 5
From task to evaluation: an automatic text summarization review
Lu, L., Liu, Y., Xu, W., Li, H., & Sun, G. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10582-5 · Cited by 7
Put Your Voice on Stage: Personalized Headline Generation for News Articles
Ao, X., Luo, L., Wang, X., Yang, Z., Chen, J.-H., Qiao, Y., He, Q., & Xie, X. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3629168 · Cited by 5
Mitigating Intrinsic Named Entity-Related Hallucinations of Abstractive Text Summarization
Shen, J., Xuan, J., & Liang, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1059 · Cited by 5
Giabbanelli, P. J., Padilla, J. J., & Agrawal, A. (2024) · Winter Simulation Conference · doi:10.1109/wsc63780.2024.10838754 · Cited by 6
“I’ve Seen Things You People Wouldn’t Believe”: Hallucinating Entities in GuessWhat?!
Testoni, A., & Bernardi, R. (2021) · doi:10.18653/v1/2021.acl-srw.11 · Cited by 5
Towards improving coherence and diversity of slogan generation
Jin, Y., Bhatia, A., Wanvarie, D., & Le, P. T. V. (2022) · Natural Language Engineering · doi:10.1017/s1351324921000474 · Cited by 6
Precisely the Point: Adversarial Augmentations for Faithful and Informative Text Generation
Wu, W., Li, W., Liu, J., Xiao, X., Li, S., & Lyu, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.482 · Cited by 6
Narrate Dialogues for Better Summarization
Xu, R., Zhu, C., & Zeng, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.261 · Cited by 6
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization
Kamoi, R., Goyal, T., & Durrett, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.11 · Cited by 8
Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization
Cheang, C., Chan, H. P., Wong, D. F., Liu, X., Li, Z., Sun, Y., Liu, S., & Chao, L. S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1007 · Cited by 9
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models
Chen, W., Yen, A.-Z., Wu, C.-K., Huang, H., & Chen, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.310 · Cited by 9
Event Graph based Sentence Fusion
Yuan, R., Wang, Z., & Li, W. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.334 · Cited by 5
Zhang, M., Zhou, G., Yu, W., Liu, W., Huang, N., & Yu, Z. (2022) · Neural Computing and Applications · doi:10.1007/s00521-021-06880-0 · Cited by 5
Multi-Fact Correction in Abstractive Text SummarizationPreprint
Dong, Y., Wang, S., Gan, Z., Cheng, Y., Cheung, J. C. K., & Liu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02443 · Cited by 5
Entity-level Factual Consistency of Abstractive Text SummarizationPreprint
Nan, F., Nallapati, R. M., Wang, Z., dos Santos, C. N., Zhu, H., Zhang, D., McKeown, K. R., & Xiang, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09130 · Cited by 5
Nutribullets Hybrid: Multi-document Health SummarizationPreprint
Shah, D., Yu, L., Lei, T., & Barzilay, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.03465 · Cited by 5
MoFE: Mixture of Factual Experts for Controlling Hallucinations in Abstractive Summarization.Preprint
Choubey, P. K., Vig, J., Liu, W., & Rajani, N. F. (2021) · arXiv (Cornell University) · Cited by 5
Li, Y., Li, L., Hu, D., Li, Y., Zhou, Y., Litvak, M., & Vanetik, N. (2023) · Research Square · doi:10.21203/rs.3.rs-3695932/v1 · Cited by 7
Chiang, C.-H., & Lee, H.-Y. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.160 · Cited by 5
The Compaction Cliff in Long-Running AI Agent Memory
Zerhoudi, S., Mitrović, J., & Granitzer, M. (2026) · ACM · doi:10.1145/3799682.3840567 · Cited by 4
Evaluating Text Summarization Techniques and Factual Consistency with Language Models
Islam, M. M., Muhammad, U., & Oussalah, M. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826032 · Cited by 6
Motghare, M., Agarwal, M., & Agrawal, A. (2025) · Computers · doi:10.3390/computers14120508 · Cited by 5
A hallucination detection and mitigation framework for faithful text summarization using LLMs
Liu, S., Gao, Y., Li, S., Wang, P., & Wang, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-31075-1 · Cited by 9
Data Augmentation for Fake News Detection by Combining Seq2seq and NLI
Glazkova, A. V. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_048 · Cited by 4
A Survey of Factual Consistency in Summarization from 2021 to 2023
Li, Y., Hao, X., & Li, L. (2023) · International Conference on Computers and Artificial Intelligence Technology (CAIT) · doi:10.1109/cait59945.2023.10468958 · Cited by 4
Deutsch, D., & Roth, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.296 · Cited by 5
Hierarchical3D Adapters for Long Video-to-text Summarization
Papalampidi, P., & Lapata, M. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.96 · Cited by 7
Entity-based De-noising Modeling for Controllable Dialogue Summarization
Liu, Z., & Chen, N. F. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.40 · Cited by 4
Eyes Show the Way: Modelling Gaze Behaviour for Hallucination Detection
Maharaj, K., Saxena, A., Kumar, R., Mishra, A., & Bhattacharyya, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.764 · Cited by 6
Dar, R. A., & Dileep, A. D. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03583-1 · Cited by 4
Improving named entity correctness of abstractive summarization by generative negative sampling
Chen, Z., & Lin, H. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101504 · Cited by 4
CoP: Factual Inconsistency Detection by Controlling the Preference
She, S., Geng, X., Huang, S., & Chen, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26589 · Cited by 4
Aligning Factual Consistency for Clinical Studies Summarization through Reinforcement Learning
Tang, X., Cohan, A., & Gerstein, M. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.7 · Cited by 4
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning
Wu, W., Li, W., Xiao, X., Liu, J., Li, S., & Lyu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.18 · Cited by 11
HistAlign: Improving Context Dependency in Language Generation by Aligning with History
Wan, D., Zhang, S., & Bansal, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.179 · Cited by 6
Template Filling for Controllable Commonsense Reasoning
Rajagopal, D., Khetan, V., Sacaleanu, B., Gershman, A. V., Fano, A., & Hovy, E. H. (2023) · doi:10.18653/v1/2023.findings-ijcnlp.23 · Cited by 9
Summarizing Long-Form Document with Rich Discourse Information
Zhu, T., Hua, W., Qu, J., & Zhou, X. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482396 · Cited by 4
Query-focused Abstractive Summarization via Question-answering Model
Du, J., & Gao, Y. (2021) · IEEE International Conference on Big Knowledge (ICBK) · doi:10.1109/ickg52313.2021.00065 · Cited by 4
Metadata
- Authors
- Kryscinski, W., McCann, B., Xiong, C., & Socher, R.
- Affiliations
- Salesforce (United States) (US)
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 9332-9346
- OpenAlex
- W2981456979
- Cited by
- 669
- References
- 41
- Citations by year
- 2026: 94 · 2025: 86 · 2024: 56 · 2023: 171 · 2022: 106 · 2021: 118 · 2020: 37
- FWCI
- 41.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- abstractive summarization, factual consistency, weakly supervised learning, rule-based transformation, span extraction, natural language inference, fact-checking, summarization evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
