The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Evaluating the Factual Consistency of Abstractive Text Summarization
Open accessAbstract
Currently used metrics for assessing summarization algorithms do not account for whether summaries are factually consistent with source documents.We propose a weakly-supervised, model-based approach for verifying factual consistency and identifying conflicts between source documents and a generated summary.Training data is generated by applying a series of rule-based transformations to the sentences of source documents.The factual consistency model is then trained jointly for three tasks: 1) identify whether sentences remain factually consistent after transformation, 2) extract a span in the source documents to support the consistency prediction, 3) extract a span in the summary sentence that is inconsistent if one exists.Transferring this model to summaries generated by several state-of-the art models reveals that this highly scalable approach substantially outperforms previous models, including those trained with strong supervision using standard datasets for natural language inference and fact checking.Additionally, human evaluation shows that the auxiliary span extraction tasks provide useful assistance in the process of verifying factual consistency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In most previous works, factuality has been defined as whether a claim in a text can be supported by evidence from a separate, trustworthy knowledge base, with applications in fact-checking (Thorne et al., 2018b) (where the knowledge base is a large source like Wikipedia) and summarization (Kryscinski et al., 2020) (where the knowledge base is an input document or documents).”
From FacTool: Factuality Detection in Generative AI - A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) · cites this paper as Kryscinski et al., 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, hallucinations are observed over a wide range of tasks beyond pure fact verification (Kryscinski et al., 2020; Maynez et al., 2020).”
From SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (2023) · cites this paper as Kryscinski et al., 2020
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Research suggests that up to 30% of machine-generated abstractive summaries may contain factual inconsistencies [52, 53].”
From All Public Voices Are Equal, But Are Some More Equal Than Others to LLMs? (2026) · cites this paper as [53]
The cited source says what the claim attributes to it.
In the citing paper
“Results show that the performance of their FactCC model surpasses the classifiers trained on the MNLI or FEVER datasets.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [126]
The cited source says what the claim attributes to it.
In the citing paper
“These methods, such as entity swapping and sentence negation (Kryscinski et al., 2020;Goyal and Durrett, 2021), were designed to target specific error types that occurred in claims from earlier summarization models.”
From MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents (2024) · cites this paper as Kryscinski et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“In response to this challenge, a surge of research explores leveraging data-augmentation methods to construct synthetical data for fine-tuning the classifier, either by rule-based perturbation [79, 152, 266] or generation [389].”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [152]
The cited source says what the claim attributes to it.
In the citing paper
“most prior studies have primarily concentrated on task-specific hallucinations, such as abstractive summarization [81, 102, 152, 208, 236, 310], data-to-text[240, 296], and machine translation [389].”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [152]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This approach has been widely studied in the summarization literature and recent work in factual precision (Kryscinski et al., 2020; Wang et al., 2020; Gao et al., 2022; Manakul et al., 2023).”
From FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023) · cites this paper as Kryscinski et al., 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
(2017) · Cited by 11,625
(2004) · Cited by 9,718
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1044 · Cited by 2,932
Research works printed but not linked: 9
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large an- notated corpus for learning natural language infer- ence. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguis- tics.
- Kyunghyun Cho, Bart van Merrienboer, C ¸aglar Gülc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Nat- ural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1724- 1734.
- Alexis Conneau, Ruty Rinott, Guillaume Lample, Ad- ina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross- lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Nat- ural Language Processing. Association for Compu- tational Linguistics.
- Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. CoRR, abs/1808.09381. · arXiv
- Ramesh Nallapati, Bowen Zhou, C ¸aglar Gülc ¸ehre, Bing Xiang, et al. 2016. Abstractive text summa- rization using sequence-to-sequence rnns and be- yond. Proceedings of SIGNLL Conference on Com- putational Natural Language Learning.
- Ramakanth Pasunuru and Mohit Bansal. 2018. Multi- reward reinforced summarization with saliency and entailment. CoRR, abs/1804.06451. · arXiv
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proc. of NAACL.
- James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. CoRR, abs/1803.05355. · arXiv
Cited by
Sorted by citations
Items 201–300 of 693
LLM-infused bi-level semantic enhancement for corporate credit risk prediction
Lu, S., Su, Y., Zhang, X., Chai, J., & Yu, L. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104091 · Cited by 15
DialogSum Challenge: Summarizing Real-Life Scenario Dialogues
Chen, Y., Liu, Y., & Zhang, Y. (2021) · International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.33 · Cited by 14
WikiHowQA: A Comprehensive Benchmark for Multi-Document Non-Factoid Question Answering
Bolotova-Baranova, V., Blinov, V., Филиппова, С. В., Scholer, F., & Sanderson, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.290 · Cited by 15
Towards Argument-Aware Abstractive Summarization of Long Legal Opinions with Summary Reranking
Elaraby, M., Zhong, Y., & Litman, D. J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.481 · Cited by 17
Improving Factuality of Abstractive Summarization via Contrastive Reward Learning
Chern, I.-C., Wang, Z., Das, S. K., Sharma, B., Liu, P., & Neubig, G. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.6 · Cited by 20
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.4 · Cited by 14
IterSum: Iterative summarization based on document topological structure
Yu, S., Gao, W., Qin, Y., Yang, C., Huang, R., Chen, Y., & Lin, C. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103918 · Cited by 14
Truth or Error? Towards systematic analysis of factual errors in abstractive summaries
Lux, K.-M., Sappelli, M., & Larson, M. (2020) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2020.eval4nlp-1.1 · Cited by 14
A Graph-to-Sequence Learning Framework for Summarizing Opinionated Texts
Wei, P., Zhao, J., & Mao, W. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3071667 · Cited by 15
X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization
Chaudhury, S., Swaminathan, S., Gunasekara, C., Crouse, M., Ravishankar, S., Kimura, D., Murugesan, K., Astudillo, R. F., Naseem, T., Kapanipathi, P., et al. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.478 · Cited by 15
Generating Sports News from Live Commentary: A Chinese Dataset for Sports Game Summarization
Huang, K.-H., Li, C., & Chang, K.-W. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.61 · Cited by 15
Abstractive text summarization: A comprehensive survey of techniques, systems, and challenges
Almohaimeed, N., & Azmi, A. M. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100762 · Cited by 18
Table-based Fact Verification with Self-adaptive Mixture of Experts
Zhou, Y., Liu, X., Zhou, K., & Wu, J. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.13 · Cited by 13
Entity-Driven Fact-Aware Abstractive Summarization of Biomedical Literature
Alambo, A., Banerjee, T., Thirunarayan, K., & Raymer, M. L. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956656 · Cited by 13
Cross-modal knowledge guided model for abstractive summarization
Wang, H., Liu, J., Duan, M., Gong, P., Wu, Z., Wang, J., & Han, B. (2023) · Complex & Intelligent Systems · doi:10.1007/s40747-023-01170-9 · Cited by 14
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation
He, T., Zhang, J., Wang, T., Kumar, S., Cho, K., Glass, J. R., & Tsvetkov, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.674 · Cited by 37
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge
Feng, S., Balachandran, V., Bai, Y., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.59 · Cited by 40
Generating Summaries with Controllable Readability Levels
Ribeiro, L., Bansal, M., & Dreyer, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.714 · Cited by 24
Single-Document Abstractive Text Summarization: A Systematic Literature Review
Rao, A. B., Aithal, S. G., & Singh, S. (2024) · ACM Computing Surveys · doi:10.1145/3700639 · Cited by 16
Active Learning for Abstractive Text Summarization
Tsvigun, A., Lysenko, I., Sedashov, D., Lazichny, I., Damirov, E., Va, K., Belousov, A., Sanochkin, L., Panov, M., Panchenko, A., et al. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.377 · Cited by 14
Bian, J., Huang, X., Zhou, H., Huang, T., & Zhu, S. (2024) · Knowledge and Information Systems · doi:10.1007/s10115-024-02195-3 · Cited by 18
Yadav, S., Sarrouti, M., & Gupta, D. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.34 · Cited by 15
GSum: A General Framework for Guided Neural Abstractive SummarizationPreprint
Dou, Z.-Y., Liu, P., Hayashi, H., Jiang, Z., & Neubig, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08014 · Cited by 13
(2021) · arXiv (Cornell University) · Cited by 13
MedKA: A knowledge graph-augmented approach to improve factuality in medical Large Language Models
Deng, Y., Zhao, S., Miao, Y., Zhu, J., & Li, J. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104871 · Cited by 18
BioLaySumm 2023 Shared Task: Lay Summarisation of Biomedical Research Articles
Goldsack, T., Luo, Z., Xie, Q., Scarton, C., Shardlow, M., Ananiadou, S., & Lin, C. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.44 · Cited by 13
On Improving Summarization Factual Consistency from Natural Language Feedback
Liu, Y., Deb, B., Teruel, M., Halfaker, A., Radev, D., & Awadallah, A. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.844 · Cited by 25
Sudoh, K., Takahashi, K., & Nakamura, S. (2021) · Cited by 12
Data Augmentation for Low-Resource Dialogue Summarization
Liu, Y., Maynez, J., Simões, G., & Narayan, S. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.53 · Cited by 13
From spoken dialogue to formal summary: An utterance rewriting for dialogue summarization
Fang, Y., Zhang, H., Chen, H., Ding, Z., Long, B., Lan, Y., & Zhou, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.283 · Cited by 13
Improving Automatic Summarization for Browsing Longform Spoken Dialog
Li, D., Chen, T., Zadikian, A., Tung, A., & Chilton, L. B. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581339 · Cited by 13
Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges
Ramprasad, S., Mcinerney, J., Marshall, I. J., & Wallace, B. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-demo.27 · Cited by 12
How “Multi” is Multi-Document Summarization?
Wolhandler, R., Cattan, A., Ernst, O., & Dagan, I. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.389 · Cited by 16
Factual Error Correction for Abstractive Summarization ModelsPreprint
Cao, M., Dong, Y., Wu, J., & Cheung, J. C. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08712 · Cited by 12
Enhancing Factual Consistency of Abstractive SummarizationPreprint
Zhu, C., Hinthorn, W., Xu, R., Zeng, Q., Zeng, M., Huang, X., & Jiang, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08612 · Cited by 12
Proceedings of the 4th New Frontiers in Summarization Workshop
(2023) · doi:10.18653/v1/2023.newsum-1 · Cited by 12
Adhik, C., Lakshmi, S., & Muralidharan, C. (2024) · AIP conference proceedings · doi:10.1063/5.0217004 · Cited by 11
Shah, A., Shah, H., Bafna, V., Khandor, C., & Nair, S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110284 · Cited by 19
AgreeSum: Agreement-Oriented Multi-Document Summarization
Pang, R. Y., Lelkes, A. D., Tran, V., & Yu, C. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.299 · Cited by 13
WikiDes: A Wikipedia-based dataset for generating short descriptions from paragraphs
Ta, H. T., Rahman, A. B. S., Majumder, N., Hussain, A., Najjar, L., Howard, N., Poria, S., & Gelbukh, A. F. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.09.022 · Cited by 12
Faithful Abstractive Summarization via Fact-aware Consistency-constrained Transformer
Lyu, Y., Zhu, C., Xu, T., Yin, Z., & Chen, E. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557319 · Cited by 11
Generating Explainable Product Comparisons for Online Shopping
Vedula, N., Collins, M., Agichtein, E., & Rokhlenko, O. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3570489 · Cited by 11
Entity-based SpanCopy for Abstractive Summarization to Improve the Factual Consistency
Wen, X., & Carenini, G. (2023) · Workshop on Computational Approaches to Discourse (CODI) · doi:10.18653/v1/2023.codi-1.9 · Cited by 14
Templated Text Synthesis for Expert-Guided Multi-Label Extraction from Radiology Reports
Schrempf, P., Watson, H. I., Park, E., Pajak, M., MacKinnon, H., Muir, K. W., Harris‐Birtill, D., & O’Neil, A. Q. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3020015 · Cited by 11
Fact-Driven Abstractive Summarization by Utilizing Multi-Granular Multi-Relational Knowledge
Mao, Q., Li, J., Peng, H. M., He, S., Wang, L., Yu, P. S., & Wang, Z. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3161157 · Cited by 11
Efficient Attentions for Long Document SummarizationPreprint
Huang, L., Cao, S., Parulian, N. N., Ji, H., & Wang, L. (2021) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2104.02112 · Cited by 507
Abstractive summarization with deep reinforcement learning using semantic similarity rewards
Fikri, F. B., Oflazer, K., & Yanikoglu, B. A. (2023) · Natural Language Engineering · doi:10.1017/s1351324923000505 · Cited by 13
Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworks
Haldar, R., & Hockenmaier, J. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.1361 · Cited by 11
“SummaC CONV performs the best at FactCC, which is unsurprising since that baseline was specifically fine-tuned on training examples from that dataset (Laban et al., 2022).”
InfluentialA Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarization
Adams, G., Zucker, J. E., & Elhadad, N. (2023) · PubMed · doi:10.48550/arxiv.2303.03948 · Cited by 22
Frisoni, G., Italiani, P., Salvatori, S., & Moro, G. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26503 · Cited by 11
Contrastive Hierarchical Discourse Graph for Scientific Document Summarization
Zhang, H., Liu, X., & Zhang, J. (2023) · Workshop on Computational Approaches to Discourse (CODI) · doi:10.18653/v1/2023.codi-1.4 · Cited by 13
From Chaos to Clarity: Claim Normalization to Empower Fact-Checking
Sundriyal, M., Chakraborty, T., & Nakov, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.439 · Cited by 24
Give the Truth: Incorporate Semantic Slot into Abstractive Dialogue Summarization
Zhao, L., Zeng, W., Xu, W., & Guo, J. H. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.209 · Cited by 10
Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality
Dixit, T., Wang, F., & Chen, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.78 · Cited by 12
Factual Error Correction for Abstractive Summaries Using Entity Retrieval
Lee, H., Park, C., Yoon, S., Bui, T., Dernoncourt, F., Kim, J., & Jung, K. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.41 · Cited by 12
Explainable Fact-Checking Through Question Answering
Yang, J., Vega‐Oliveros, D. A., Seibt, T., & Rocha, A. D. R. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747214 · Cited by 10
Improving Unsupervised Extractive Summarization by Jointly Modeling Facet and Redundancy
Liang, X., Li, J., Wu, S., Li, M., & Li, Z. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3138673 · Cited by 10
Token-Level Fact Correction in Abstractive Summarization
Shin, J., Park, S.-B., & Song, H.-J. (2023) · IEEE Access · doi:10.1109/access.2022.3233854 · Cited by 10
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis
Lu, Q., Ding, L., Xie, L., Zhang, K., Wong, D. F., & Tao, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.324 · Cited by 20
Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator
Cheng, Q., Li, L., Quan, G., Gao, F., Mou, X., & Qiu, X. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.90 · Cited by 14
Learning with Rejection for Abstractive Text Summarization
Cao, M., Dong, Y., He, J., & Cheung, J. C. K. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.663 · Cited by 11
Dong, Y., Romascanu, A., & Cheung, J. C. K. (2020) · arXiv (Cornell University) · Cited by 8
Adversarial NLI for Factual Correctness in Text Summarisation Models
Barrantes, M., Herudek, B., & Wang, R. (2020) · arXiv · doi:10.48550/arxiv.2005.11739 · Cited by 10
Sometimes We Want Ungrammatical Translations
Parthasarathi, P., Sinha, K., Pineau, J., & Williams, A. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.275 · Cited by 9
HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
Ravichander, A., Ghela, S., Wadden, D., & Choi, Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.71 · Cited by 13
Zhong, Y., & Litman, D. J. (2022) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2022.nllp-1.30 · Cited by 13
Transformer-Based Approach for Automatic Semantic Financial Document Verification
Toprak, A., & Turan, M. (2024) · IEEE Access · doi:10.1109/access.2024.3477270 · Cited by 10
An End-to-End Dialogue Summarization System for Sales Calls
Asi, A., Wang, S., Eisenstadt, R., Geckt, D., Kuper, Y., Mao, Y., & Ronen, R. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-industry.6 · Cited by 14
ESSumm: Extractive Speech Summarization from Untranscribed Meeting
Wang, J. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-945 · Cited by 10
Bastan, M., Surdeanu, M., & Balasubramanian, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.374 · Cited by 19
To Point or Not to Point: Understanding How Abstractive Summarizers Paraphrase Text
Wilber, M., Timkey, W., & van Schijndel, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.298 · Cited by 9
Fine-grained Factual Consistency Assessment for Abstractive Summarization Models
Zhang, S., Niu, J., & Wei, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.9 · Cited by 9
ALS-MRS: Incorporating aspect-level sentiment for abstractive multi-review summarization
Zhao, Q., Niu, J., & Liu, X. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109942 · Cited by 10
Improving abstractive summarization with energy-based re-ranking
Pernes, D., Mendes, A., & Martins, A. F. T. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.1 · Cited by 9
FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness
Wu, W., Li, W., Liu, J., Xiao, X., Cao, Z., Li, S., & Wu, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.267 · Cited by 9
Reliability Check: An Analysis of GPT-3’s Response to Sensitive Topics and Prompt Wording
Khatun, A., & Brown, D. G. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.8 · Cited by 18
Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution
Xu, J., & Durrett, G. (2021) · doi:10.18653/v1/2021.acl-long.539 · Cited by 11
Cooperative Generator-Discriminator Networks for Abstractive Summarization with Narrative FlowPreprint
Gabriel, S., Bosselut, A., Holtzman, A., Lo, K., Çelikyılmaz, A., & Choi, Y. (2019) · arXiv (Cornell University) · Cited by 9
Analyzing the Abstractiveness-Factuality Tradeoff With Nonlinear Abstractiveness Constraints.Preprint
Dreyer, M., Liu, M., Feng, N., Atluri, S., & Ravi, S. (2021) · arXiv (Cornell University) · Cited by 9
Fine-tuning BERT Models for Summarizing German Radiology Findings
Liang, S., Kades, K., Fink, M., Full, P., Weber, T., Kleesiek, J., Strube, M., & Maier-Hein, K. (2022) · Proceedings of the 4th Clinical Natural Language Processing Workshop · doi:10.18653/v1/2022.clinicalnlp-1.4 · Cited by 8
Evaluating the Tradeoff Between Abstractiveness and Factuality in Abstractive Summarization
Dreyer, M., Liu, M., Nan, F., Atluri, S., & Ravi, S. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.156 · Cited by 16
Xiong, J., Pan, L., Liu, Y., Zhu, L., Zhang, L., & Tan, S. (2025) · Applied Sciences · doi:10.3390/app15073850 · Cited by 12
Performance Evaluation of Modern GPU Accelerator-Based Edge Systems: A Holistic Approach
Lee, H., & Kang, P. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3616166 · Cited by 8
Nutri-bullets Hybrid: Consensual Multi-document Summarization
Shah, D., Yu, L., Lei, T., & Barzilay, R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.411 · Cited by 14
Learning as Conversation: Dialogue Systems Reinforced for Information Acquisition
Cai, P., Wan, H., Liu, F., Yu, M., Yu, H. Q., & Joshi, S. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.352 · Cited by 8
Automated Summarization of Stack Overflow Posts
Kou, B., Chen, M., & Zhang, T. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00158 · Cited by 13
Generating User-Engaging News Headlines
Cai, P., Song, K., Cho, S., Wang, H., Wang, X., Yu, H., Liu, F., & Dong, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.183 · Cited by 12
Han, C., Feng, J., & Qi, H. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121873 · Cited by 10
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation
Clark, E., Rijhwani, S., Gehrmann, S., Maynez, J., Aharoni, R., Nikolaev, V. A., Sellam, T., Siddhant, A., Das, D., & Parikh, A. P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.584 · Cited by 25
Learning Cluster Patterns for Abstractive Summarization
Jo, S.-G., Park, S.-H., Kim, J.-J., & On, B.-W. (2023) · IEEE Access · doi:10.1109/access.2023.3346911 · Cited by 9
Do Multi-Document Summarization Models Synthesize?
DeYoung, J., Martinez, S. C., Marshall, I. J., & Wallace, B. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00687 · Cited by 12
The Effect of Pretraining on Extractive Summarization for Scientific Documents
Gupta, Y., Ammanamanchi, P. S., Bordia, S., Manoharan, A., Mittal, D., Pasunuru, R., Shrivastava, M., Singh, M., Bansal, M., & Jyothi, P. (2021) · Workshop on Scholarly Document Processing (SDP) · doi:10.18653/v1/2021.sdp-1.9 · Cited by 8
Building Adaptive Acceptability Classifiers for Neural NLG
Batra, S., Jain, S. S. B. C., Heidari, P., Arun, A., Youngs, C., Li, X., Dönmez, P., Mei, S., Kuo, S., Bhardwaj, V. S., et al. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.53 · Cited by 8
Towards Abstractive Grounded Summarization of Podcast Transcripts
Song, K., Li, C., Wang, X., Yu, D., & Liu, F. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.302 · Cited by 12
An Exploration of Post-Editing Effectiveness in Text Summarization
Lai, V. W.-M., Smith, A., Zhang, K., Cheng, R., Zhang, W., Tetreault, J., & Jaimes-Larrarte, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.35 · Cited by 8
Data-Centric and Model-Centric Approaches for Biomedical Question Answering
Yoon, W., Yoo, J., Seo, S., Sung, M., Jeong, M., Kim, G., & Kang, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13643-6_16 · Cited by 8
Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization
Chan, H. P., Zeng, Q., & Ji, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.402 · Cited by 8
Chang, H.-S., Yao, Z., Gon, A., Yu, H. Z., & McCallum, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.805 · Cited by 8
Adams, G., Nguyen, B. H., Smith, J. A., Xia, Y., Xie, S., Ostropolets, A., Deb, B., Chen, Y.-J., Naumann, T., & Elhadad, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.587 · Cited by 10
Metadata
- Authors
- Kryscinski, W., McCann, B., Xiong, C., & Socher, R.
- Affiliations
- Salesforce (United States) (US)
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 9332-9346
- OpenAlex
- W2981456979
- Cited by
- 669
- References
- 41
- Citations by year
- 2026: 94 · 2025: 86 · 2024: 56 · 2023: 171 · 2022: 106 · 2021: 118 · 2020: 37
- FWCI
- 41.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- abstractive summarization, factual consistency, weakly supervised learning, rule-based transformation, span extraction, natural language inference, fact-checking, summarization evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
