Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection
Open accessAbstract
Despite significant progress in neural abstractive summarization, recent studies have shown that the current models are prone to generating summaries that are unfaithful to the original context.To address the issue, we study contrast candidate generation and selection as a model-agnostic post-processing technique to correct the extrinsic hallucinations (i.e.information not present in the source text) in unfaithful summaries.We learn a discriminative correction model by generating alternative candidate summaries where named entities and quantities in the generated summary are replaced with ones with compatible semantic types from the source document.This model is then used to select the best candidate as the final output summary.Our experiments and analysis across a number of neural summarization systems show that our proposed method is effective in identifying and correcting extrinsic hallucinations.We analyze the typical hallucination phenomenon by different types of neural summarization systems, in hope to provide insights for future work on the direction.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For noisy datasets where a large proportion of ground truth references suffer from hallucinations, modeling correction is a competitive choice to handle the hallucination problem.
“Especially for noisy datasets where a large proportion of the ground truth references suffer from hallucinations, modeling correction is a competitive choice to handle the hallucination problem [32].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [32]
Planning as a separate step in a two-step generator is prone to progressive amplification of the hallucination problem.
“Planning can be a separate step in a two-step generator [32, 162, 208, 239, 269], which is prone to progressive amplification of the hallucination problem.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [32]
The Introduction describes candidate generation followed by selection as a post-processing method, not planning as a separate step in a two-step generator. Section 4.2 says incorrect entity replacements can introduce intrinsic hallucinations, but does not describe progressive amplification. Searches for planning and amplification found no passage addressing the attributed claim. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
(2004) · Cited by 9,718 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775 · see citing works
Supervised Contrastive Learning
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., & Krishnan, D. (2020) · arXiv · doi:10.48550/arxiv.2004.11362 · Cited by 3,569 · see citing works
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · see citing works
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,879 · see citing works
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · see citing works
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., & Xiang, B. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1028 · Cited by 2,361 · see citing works
Text Summarization with Pretrained Encoders
Liu, Y., & Lapata, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1387 · Cited by 1,779 · see citing works
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., & Manning, C. D. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.14 · Cited by 1,505 · see citing works
Narayan, S., Cohen, S. B., & Lapata, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1206 · Cited by 1,207 · see citing works
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · see citing works
Evaluating the Factual Consistency of Abstractive Text Summarization
Kryscinski, W., McCann, B., Xiong, C., & Socher, R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.750 · Cited by 669 · see citing works
Leveraging Pre-trained Checkpoints for Sequence Generation Tasks
Rothe, S., Narayan, S., & Severyn, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00313 · Cited by 400 · see citing works
Neural Text Summarization: A Critical Evaluation
Kryscinski, W., Keskar, N. S., McCann, B., Xiong, C., & Socher, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1051 · Cited by 335 · see citing works
Durmus, E., He, H., & Diab, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.454 · Cited by 306 · see citing works
Evaluating Models’ Local Decision Boundaries via Contrast Sets
Gardner, M., Artzi, Y., Basmov, V., Berant, J., Bogin, B., Chen, S., Dasigi, P., Dua, D., Elazar, Y., Gottumukkala, A., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.117 · Cited by 265 · see citing works
Self-citationFalke, T., Ribeiro, L. F. R., Utama, P. A., Dagan, I., & Gurevych, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1213 · Cited by 265 · see citing works
Factual Error Correction for Abstractive Summarization Models
Cao, M., Dong, Y., Wu, J., & Cheung, J. C. K. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.506 · Cited by 132 · see citing works
Annotating and Modeling Fine-grained Factuality in Summarization
Goyal, T., & Durrett, G. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.114 · Cited by 100 · see citing works
Evaluating Factuality in Generation with Dependency-level Entailment
Goyal, T., & Durrett, G. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.322 · Cited by 93 · see citing works
Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data
Filippova, K. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.76 · Cited by 79 · see citing works
Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary
Deutsch, D., Bedrax-Weiss, T., & Roth, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00397 · Cited by 75 · see citing works
Self-citationImproved Natural Language Generation via Loss Truncation
Kang, D. D., & Hashimoto, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.66 · Cited by 71 · see citing works
Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a SummaryPreprint
Deutsch, D., Bedrax-Weiss, T., & Roth, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00490 · Cited by 10 · see citing works
Self-citationDeutsch, D., & Roth, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12495 · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
2 of the 30 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Tanya Goyal and Greg Durrett. 2020. Evaluating fac- tuality in generation with dependency-level entail- ment. In Findings of the Association for Computa- tional Linguistics: EMNLP 2020, pages 3592-3603, Online. Association for Computational Linguistics.
- Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Su- pervised contrastive learning. Proceedings of the 34th Conference on Neural Information Processing Systems.
- Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018b. Ranking sentences for extractive summariza- tion with reinforcement learning. In Proceedings of the 2018 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa- pers), pages 1747-1759, New Orleans, Louisiana. Association for Computational Linguistics. · DOI
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141 · see citing works
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization
Cao, S., & Wang, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.532 · Cited by 135 · see citing works
A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models
Zhang, H., Yu, P. S., & Zhang, J. (2025) · ACM Computing Surveys · doi:10.1145/3731445 · Cited by 159 · see citing works
Measure and Improve Robustness in NLP Models: A Survey
Wang, X., Wang, H., & Yang, D. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.339 · Cited by 90 · see citing works
Improving Factual Consistency of Abstractive Summarization via Question Answering
Nan, F., Nogueira dos Santos, C., Zhu, H., Ng, P., McKeown, K., Nallapati, R., Zhang, D., Wang, Z., Arnold, A. O., & Xiang, B. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.536 · Cited by 78 · see citing works
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization
Wan, D., & Bansal, M. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.74 · Cited by 58 · see citing works
Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors
Tang, L., Goyal, T., Fabbri, A. R., Laban, P., Xu, J., Yavuz, S., Kryściński, W., Rousseau, J. F., & Durrett, G. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.650 · Cited by 62 · see citing works
AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarization
Eddine, M. K., Tomeh, N., Habash, N. Y., Roux, J. L., & Vazirgiannis, M. (2022) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/2022.wanlp-1.4 · Cited by 63 · see citing works
Ladhak, F., Durmus, E., He, H., Cardie, C., & McKeown, K. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.100 · Cited by 46 · see citing works
RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding
Ji, Z., Liu, Z., Lee, N., Yu, T., Wilie, B., Zeng, M., & Fung, P. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.275 · Cited by 37 · see citing works
Ando, K., Okumura, T., Komachi, M., Horiguchi, H., & Matsumoto, Y. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000158 · Cited by 36 · see citing works
Faithfulness-Aware Decoding Strategies for Abstractive Summarization
Wan, D., Liu, M., McKeown, K. R., Dreyer, M., & Bansal, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.210 · Cited by 35 · see citing works
Learning to Revise References for Faithful Summarization
Adams, G., Shing, H.-C., Sun, Q., Winestock, C., McKeown, K. R., & Elhadad, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.296 · Cited by 30 · see citing works
Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control
Zhang, H., Yavuz, S., Kryściński, W., Hashimoto, K., & Zhou, Y. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.40 · Cited by 28 · see citing works
Mutual Information Alleviates Hallucinations in Abstractive Summarization
van der Poel, L., Cotterell, R., & Meister, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.399 · Cited by 35 · see citing works
Zhang, S., Wan, D., & Bansal, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.120 · Cited by 24 · see citing works
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters
Schuster, T., Chen, S., Buthpitiya, S., Fabrikant, A., & Metzler, D. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.28 · Cited by 17 · see citing works
Self-citationDocument Summarization with Latent Queries
Xu, Y., & Lapata, M. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00480 · Cited by 17 · see citing works
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Fabbri, A. R., Choubey, P. K., Vig, J., Wu, C.-S., & Xiong, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.623 · Cited by 19 · see citing works
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization
Choubey, P. K., Fabbri, A. R., Vig, J., Wu, C.-S., Liu, W., & Rajani, N. F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.685 · Cited by 23 · see citing works
Challenges in Domain-Specific Abstractive Summarization and How to Overcome Them
Afzal, A., Vladika, J., Braun, D., & Matthes, F. (2023) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0011744500003393 · Cited by 19 · see citing works
X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization
Chaudhury, S., Swaminathan, S., Gunasekara, C., Crouse, M., Ravishankar, S., Kimura, D., Murugesan, K., Astudillo, R. F., Naseem, T., Kapanipathi, P., et al. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.478 · Cited by 14 · see citing works
Single-Document Abstractive Text Summarization: A Systematic Literature Review
Rao, A. B., Aithal, S. G., & Singh, S. (2024) · ACM Computing Surveys · doi:10.1145/3700639 · Cited by 16 · see citing works
“Why is this misleading?”: Detecting News Headline Hallucinations with Explanations
Shen, J., Liu, J., Finnie, D., Rahmati, N., Bendersky, M., & Najork, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583375 · Cited by 13 · see citing works
On Improving Summarization Factual Consistency from Natural Language Feedback
Liu, Y., Deb, B., Teruel, M., Halfaker, A., Radev, D., & Awadallah, A. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.844 · Cited by 17 · see citing works
Evaluation of Question-Answering Based Text Summarization using LLM Invited Paper
Ding, J., Nguyen, H., & Chen, H. (2024) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest62860.2024.00025 · Cited by 12 · see citing works
Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges
Ramprasad, S., Mcinerney, J., Marshall, I. J., & Wallace, B. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-demo.27 · Cited by 12 · see citing works
R2D2: Robust Data-to-Text with Replacement Detection
Nan, L., Flores, L. J., Zhao, Y., Liu, Y., Benson, L., Zou, W., & Radev, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.464 · Cited by 14 · see citing works
Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality
Dixit, T., Wang, F., & Chen, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.78 · Cited by 12 · see citing works
Factual Error Correction for Abstractive Summaries Using Entity Retrieval
Lee, H., Park, C., Yoon, S., Bui, T., Dernoncourt, F., Kim, J., & Jung, K. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.41 · Cited by 10 · see citing works
Fine-grained Factual Consistency Assessment for Abstractive Summarization Models
Zhang, S., Niu, J., & Wei, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.9 · Cited by 9 · see citing works
Generating EDU Extracts for Plan-Guided Summary Re-Ranking
Adams, G., Fabbri, A. R., Ladhak, F., Elhadad, N., & McKeown, K. R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.151 · Cited by 9 · see citing works
FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness
Wu, W., Li, W., Liu, J., Xiao, X., Cao, Z., Li, S., & Wu, H. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.267 · Cited by 9 · see citing works
CO2Sum:Contrastive Learning for Factual-Consistent Abstractive SummarizationPreprint
Liu, W., Wu, H., Mu, W., Li, Z., Chen, T., & Nie, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01147 · Cited by 13 · see citing works
Adams, G., Nguyen, B. H., Smith, J. A., Xia, Y., Xie, S., Ostropolets, A., Deb, B., Chen, Y.-J., Naumann, T., & Elhadad, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.587 · Cited by 9 · see citing works
PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition
Chen, S., Buthpitiya, S., Fabrikant, A., Roth, D., & Schuster, T. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.565 · Cited by 10 · see citing works
Self-citationScientific document processing: challenges for modern learning methods
Kashyap, A. R., Yang, Y., & Kan, M. (2023) · International Journal on Digital Libraries · doi:10.1007/s00799-023-00352-7 · Cited by 9 · see citing works
Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation
Lango, M., & Dušek, O. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.172 · Cited by 10 · see citing works
Quality Evaluation of Summarization Models for Patent Documents
Ding, J., Chen, H., Kolapudi, S. T., Pobbathi, L., & Nguyen, H. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00033 · Cited by 6 · see citing works
Focus on the Action: Learning to Highlight and Summarize Jointly for Email To-Do Items Summarization
Zhang, K., Chen, J., & Yang, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.323 · Cited by 5 · see citing works
Wang, X., & Zhao, T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.180 · Cited by 5 · see citing works
Improving Faithfulness by Augmenting Negative Summaries from Fake Documents
Wang, T., Ladhak, F., Durmus, E., & He, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.816 · Cited by 5 · see citing works
Mitigating Intrinsic Named Entity-Related Hallucinations of Abstractive Text Summarization
Shen, J., Xuan, J., & Liang, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1059 · Cited by 5 · see citing works
Towards improving coherence and diversity of slogan generation
Jin, Y., Bhatia, A., Wanvarie, D., & Le, P. T. V. (2022) · Natural Language Engineering · doi:10.1017/s1351324921000474 · Cited by 6 · see citing works
Milintsevich, K., & Agarwal, N. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.56 · Cited by 4 · see citing works
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization
Kamoi, R., Goyal, T., & Durrett, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.11 · Cited by 5 · see citing works
MoFE: Mixture of Factual Experts for Controlling Hallucinations in Abstractive Summarization.Preprint
Choubey, P. K., Vig, J., Liu, W., & Rajani, N. F. (2021) · arXiv (Cornell University) · Cited by 5 · see citing works
A Survey of Factual Consistency in Summarization from 2021 to 2023
Li, Y., Hao, X., & Li, L. (2023) · International Conference on Computers and Artificial Intelligence Technology (CAIT) · doi:10.1109/cait59945.2023.10468958 · Cited by 4 · see citing works
Suri, K., Saha, S., & Singh, A. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.50 · Cited by 3 · see citing works
Gao, M., Wan, X., Su, J., Wang, Z., & Huai, B. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.779 · Cited by 5 · see citing works
Improving named entity correctness of abstractive summarization by generative negative sampling
Chen, Z., & Lin, H. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101504 · Cited by 4 · see citing works
Aligning Factual Consistency for Clinical Studies Summarization through Reinforcement Learning
Tang, X., Cohan, A., & Gerstein, M. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.7 · Cited by 4 · see citing works
HistAlign: Improving Context Dependency in Language Generation by Aligning with History
Wan, D., Zhang, S., & Bansal, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.179 · Cited by 5 · see citing works
Improving Factual Consistency of Abstractive Summarization via Question AnsweringPreprint
Nan, F., dos Santos, C. N., Zhu, H., Ng, P., McKeown, K. R., Nallapati, R. M., Zhang, D., Wang, Z., Arnold, A. O., & Xiang, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04623 · Cited by 4 · see citing works
Zhu, Y., Li, Y., Li, L., Hu, D., Hao, X., Chen, D., Zhang, X., Zhang, Z., Zhou, Y., Litvak, M., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3567744 · Cited by 4 · see citing works
A Survey of Hallucination in Large Language Models
何, 金. (2026) · Artificial Intelligence and Robotics Research · doi:10.12677/airr.2026.151016 · Cited by 3 · see citing works
Reducing named entity hallucination risk to ensure faithful summary generation
Akani, E., Favre, B., Béchet, F., & Gemignani, R. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.33 · Cited by 3 · see citing works
Chen, W., Wu, C.-K., Chen, H., & Chen, C.-C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.54 · Cited by 4 · see citing works
Atri, Y. K., Iyer, A. S., Chakraborty, T., & Goyal, V. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.133 · Cited by 3 · see citing works
A Pretraining Strategy to Improve Faithfulness in Abstractive Text Summarization
Alrefaai, M., & Akgün, D. (2023) · International Symposium on Innovative Approaches in Smart Technologies (ISAS) · doi:10.1109/isas60782.2023.10391437 · Cited by 3 · see citing works
Lan, Z., Li, W., Su, J., Xiao, X., Liu, J., Wu, W., & Lyu, Y. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14267 · Cited by 3 · see citing works
Fact-Aware Generative Text Summarization with Dependency Graphs
Chen, R., Li, Y., Jiang, Y., Sun, B., Wang, J., & Li, Z. (2024) · Electronics · doi:10.3390/electronics13163230 · Cited by 2 · see citing works
Automatic Text Summarization-Based Transformers Architecture
Al-Banna, A. A., & Al-Mashhadany, A. K. (2023) · International Conference on Information Technology, Applied Mathematics and Statistics (ICITAMS) · doi:10.1109/icitams57610.2023.10525560 · Cited by 2 · see citing works
Improving Faithfulness and Factuality with Contrastive Learning in Explainable Recommendation
Zhuang, H., Zhang, W. E., Chen, W., Yang, J. Y., & Sheng, Q. Z. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3653984 · Cited by 5 · see citing works
Personalized Product Description Generation With Gated Pointer-Generator Transformer
Liang, Y.-S., Chen, C.-Y., Li, C., & Chang, S. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3396840 · Cited by 2 · see citing works
Integrating Entities in Text Summarization: A Review
Zhao, Y. Q., Ding, Z., Wang, F., Zou, L., & Nian, A. (2024) · doi:10.1145/3695080.3695170 · Cited by 2 · see citing works
Dataset Construction and Verification for Detecting Factual Inconsistency in Japanese Summarization
Iwamoto, K., & Shimada, K. (2024) · IIAI International Congress on Advanced Applied Informatics (IIAI-AAI) · doi:10.1109/iiai-aai63651.2024.00054 · Cited by 1 · see citing works
Improving Next Tokens via Second-to-Last Predictions with Generate and Refine
Schneider, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91398-3_17 · Cited by 1 · see citing works
LexiSem: A re-ranker balancing lexical and semantic quality for enhanced abstractive summarization
Aloraini, E., Kassab, H., Hamdi, A., & Shaban, K. B. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130816 · Cited by 1 · see citing works
Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarization
Zhu, R., Qi, J., & Lau, J. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.377 · Cited by 4 · see citing works
Hallucination Detection for Grounded Instruction Generation
Zhao, L., Nguyen, K., & Daumé, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.266 · Cited by 1 · see citing works
PhraseSumm: Abstractive Short Phrase Summarization
Bhattacharjee, K., McKeown, K. R., & Gangadharaiah, R. (2023) · doi:10.18653/v1/2023.findings-ijcnlp.8 · Cited by 1 · see citing works
Generating Coherent and Diverse Slogans with Sequence-to-Sequence Transformer.Preprint
Jin, Y., Bhatia, A., Wanvarie, D., & Le, P. T. V. (2021) · arXiv (Cornell University) · Cited by 1 · see citing works
CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive SummarizationPreprint
Cao, S., & Wang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.09209 · Cited by 14 · see citing works
Toward Fact-aware Abstractive Summarization Method Using Joint LearningPreprint
Mao, X., Chen, L., Ju, S., Wan, X., & Liu, Y. (2022) · Research Square · doi:10.21203/rs.3.rs-2206382/v1
Factual Error Correction in Summarization with Retriever-Reader Pipeline
Li, W., Liu, J., & Gao, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30105-6_33
A Multi-Hop Reasoning Knowledge Selection Module for Dialogue Generation
Ma, Z., Liu, J., Xu, B., Lv, K., & Guo, S. (2024) · Electronics · doi:10.3390/electronics13163275
Chen, D., Hu, D., & Li, L. (2024) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3708657.3708772
Tong, X., Zhu, Y., Fan, S., & Xu, L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0014-7_12
GraphFact-Summ: Graph-Augmented Factual Summarization of Hospital Courses from Clinical Notes
Niloy, M., Islam, M. T., Ullah, M. S., Alom, J., Sultana, S. R., & Nur, K. M. (2025) · International Conference on Artificial Intelligence, Blockchain, and Internet of Things (AIBThings) · doi:10.1109/aibthings66987.2025.11296232 · Cited by 3 · see citing works
Contribution to natural language generation : systems and evaluation
Eddine, M. K. (2022) · theses.fr (ABES) · doi:10.70675/d1be4d48z831bz40cdzb61az9296693d2f3a
Diverse and efficient ensembling of deep networks
Ramé, A. (2023) · theses.fr (ABES) · doi:10.70675/fa295dcfz969ez4448z9a30z7c7bbfabb740
- Authors
- Chen, S., Zhang, F., Sone, K., & Roth, D.
- Affiliations
- University of Pennsylvania (US)
- Google (United States) (US)
- Venue
- Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2021
- Pages
- 5935-5941
- OpenAlex
- W3172210712
- Cited by
- 83
- References
- 30
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 11 · 2023: 36 · 2022: 20 · 2021: 8
- FWCI
- 8.66
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- U.S. Department of Defense
- Tencent
- Defense Advanced Research Projects Agency (FA8750-19-2-1004, FA8750)
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- abstractive summarization, faithfulness, hallucination correction, candidate selection, post-processing, neural summarization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
