The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey
Open accessAbstract
Recently, various neural encoder-decoder models pioneered by Seq2Seq framework have been proposed to achieve the goal of generating more abstractive summaries by learning to map input text to output text. At a high level, such neural models can freely generate summaries without any constraint on the words or phrases used. Moreover, their format is closer to human-edited summaries and output is more readable and fluent. However, the neural model's abstraction ability is a double-edged sword. A commonly observed problem with the generated summaries is the distortion or fabrication of factual information in the article. This inconsistency between the original text and the summary has caused various concerns over its applicability, and the previous evaluation methods of text summarization are not suitable for this issue. In response to the above problems, the current research direction is predominantly divided into two categories, one is to design fact-aware evaluation metrics to select outputs without factual inconsistency errors, and the other is to develop new summarization systems towards factual consistency. In this survey, we focus on presenting a comprehensive review of these fact-specific evaluation methods and text summarization models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The main research focus has been on dealing with intrinsic hallucination, while extrinsic hallucination has been somewhat overlooked as it is more challenging to reduce.
“Moreover, the main research focus has been on dealing with intrinsic hallucination, while extrinsic hallucination has been somewhat overlooked as it is more challenging to reduce [104].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [104]
Section 2.1 (p. 2) says existing optimization methods mainly focus on intrinsic errors, and the Conclusion (p. 7) says they ignore extrinsic errors. The Conclusion discusses a challenge in addressing extrinsic errors, but does not say they have been overlooked because they are more challenging to reduce than intrinsic errors. (LLM, conf=0.87)
Experimental — accuracy not yet measuredHallucinations in conventional NLG tasks are categorized into intrinsic and extrinsic types.
“Notably, hallucinations in conventional natural language generation (NLG) tasks have been extensively studied [125, 136], with hallucinations defined as generated content that is either nonsensical or unfaithful to the provided source content. These hallucinations are categorized into two types: intrinsic hallucination , where the generated output contradicts the source content, and extrinsic Hall”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [125]
Section 2.1 (p. 2) divides factual inconsistency errors in abstractive summarization into intrinsic and extrinsic types and explicitly calls them hallucinations. The source does not establish that this classification applies across conventional NLG tasks generally; its broader discussion of other conditional generation tasks in Section 5 (p. 7) does not extend the classification to them. (LLM, conf=0.84)
Experimental — accuracy not yet measuredFollowing previous works, there are two main types of hallucinations: intrinsic hallucination and extrinsic hallucination.
“Following the categorization from previous works [60, 104, 177], there are two main types of hallucinations, namely intrinsic hallucination and extrinsic hallucination.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [104]
Existing surveys on hallucination have only focused on specific tasks like abstractive summarization and translation.
“To the best of our knowledge, existing surveys have only focused specific tasks like abstractive summarization [104, 177] and translation [133].”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [104]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
(2004) · Cited by 9,718 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775 · see citing works
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · see citing works
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,779 · see citing works
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., & Xiang, B. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1028 · Cited by 2,361 · see citing works
Open information extraction from the web
Banko, M., Cafarella, M., Soderland, S. G., Broadhead, M., & Etzioni, O. (2007) · Cited by 1,410 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242 · see citing works
Narayan, S., Cohen, S. B., & Lapata, M. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1206 · Cited by 1,207 · see citing works
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · see citing works
Advances in Automatic Text Summarization
Liddy, E. D. (2001) · Information Retrieval · doi:10.1023/a:1011476409104 · Cited by 689 · see citing works
Evaluating the Factual Consistency of Abstractive Text Summarization
Kryscinski, W., McCann, B., Xiong, C., & Socher, R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.750 · Cited by 669 · see citing works
Faithful to the Original: Fact Aware Neural Abstractive Summarization
Cao, Z., Wei, F., Li, W., & Li, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11912 · Cited by 359 · see citing works
(2020) · doi:10.18653/v1/2020.acl-main.450 · Cited by 356 · see citing works
The challenges of automatic summarization
Hahn, U., & Mani, I. (2000) · Computer · doi:10.1109/2.881692 · Cited by 352 · see citing works
Durmus, E., He, H., & Diab, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.454 · Cited by 306 · see citing works
Falke, T., Ribeiro, L. F. R., Utama, P. A., Dagan, I., & Gurevych, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1213 · Cited by 265 · see citing works
Abstractive summarization: An overview of the state of the art
Gupta, S., & Gupta, S. K. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.12.011 · Cited by 236 · see citing works
Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports
Zhang, Y., Merck, D., Tsai, E., Manning, C. D., & Langlotz, C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.458 · Cited by 171 · see citing works
Assessing The Factual Accuracy of Generated Text
Goodrich, B., Rao, V., Liu, P. J., & Saleh, M. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330955 · Cited by 162 · see citing works
Knowledge Graph-Augmented Abstractive Summarization with Semantic-Driven Cloze Reward
Huang, L., Wu, L., & Wang, L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.457 · Cited by 159 · see citing works
Detecting Hallucinated Content in Conditional Neural Sequence Generation
Zhou, C., Neubig, G., Gu, J., Diab, M., Guzmán, F., Zettlemoyer, L., & Ghazvininejad, M. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.120 · Cited by 122 · see citing works
Reducing Quantity Hallucinations in Abstractive Summarization
Zhao, Z., Cohen, S. B., & Webber, B. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.203 · Cited by 99 · see citing works
Li, H., Zhu, J., Zhang, J., & Zong, C. (2018) · International Conference on Computational Linguistics · Cited by 89 · see citing works
Zajic, D. M., Dorr, B. J., & Schwartz, R. M. (2004) · Cited by 71 · see citing works
Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained GenerationPreprint
Mao, Y., Ren, X., Ji, H., & Han, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12723 · Cited by 33 · see citing works
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
Gunel, B., Zhu, C., Zeng, M., & Huang, X. (2020) · arXiv · doi:10.48550/arxiv.2006.15435 · Cited by 28 · see citing works
FFCI: A Framework for Interpretable Automatic Evaluation of Summarization
Koto, F., Baldwin, T., & Lau, J. H. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13167 · Cited by 19 · see citing works
Factual Error Correction for Abstractive Summarization ModelsPreprint
Cao, M., Dong, Y., Wu, J., & Cheung, J. C. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08712 · Cited by 12 · see citing works
On the Faithfulness for E-commerce Product Summarization
Yuan, P., Li, H., Xu, S., Wu, Y., He, X., & Zhou, B. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.502 · Cited by 9 · see citing works
Nutri-bullets: Summarizing Health Studies by Composing Segments
Shah, D., Yu, L., Lei, T., & Barzilay, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17624 · Cited by 7 · see citing works
Looking Beyond Sentence-Level Natural Language Inference for Downstream TasksPreprint
Mishra, A., Patel, D., Vijayakumar, A., Li, X., Kapanipathi, P., & Talamadupula, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.09099 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- [Bordes et al. , 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Dur´ an, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multirelational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 , NIPS'13, page 2787-2795, Red Hook, NY, USA, 2013. Curran Associates Inc.
- Second AAAI Conference on Artificial Intelligence, (AAAI18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018 , pages 4784-4791, 2018.
- [Dong et al. , 2020] Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, and Jingjing Liu. Multifact correction in abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 93209331, Online, November 2020. Association for Computational Linguistics.
- [Matsumaru et al. , 2020] Kazuki Matsumaru, Sho Takase, and Naoaki Okazaki. Improving truthfulness of headline generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 1335-1346, Online, July 2020. Association for Computational Linguistics.
- [Norouzi et al. , 2016] Mohammad Norouzi, Samy Bengio, Navdeep Jaitly, Mike Schuster, Yonghui Wu, Dale Schuurmans, et al. Reward augmented maximum likelihood for neural structured prediction. Advances In Neural Information Processing Systems , 29:1723-1731, 2016.
- [Wang et al. , 2020] Alex Wang, Kyunghyun Cho, and Mike Lewis. Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 5008-5020, Online, July 2020. Association for Computational Linguistics.
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
“These metrics are based on the idea that only the source knowledge reference should entail the entirety of the information in faithful and hallucination-free generation [37, 41, 44, 72, 75, 86, 89, 123, 190].”
BackgroundMethodInfluentialHuang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
Self-citationEvaluating Object Hallucination in Large Vision-Language Models
Li, Y., Du, Y., Zhou, K., Wang, J., Xin, Z., & Wen, J.-R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.20 · Cited by 580 · see citing works
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.557 · Cited by 557 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516 · see citing works
“Generally, hallucinations in natural language generation tasks can be categorized into two primary types: intrinsic hallu-cination and extrinsic hallucination (Huang et al., 2021; Li et al., 2022b; Ji et al., 2023a).”
Self-citationHallucination is Inevitable: An Innate Limitation of Large Language Models
Xu, Z., Jain, S., & Kankanhalli, M. (2024) · arXiv · doi:10.48550/arxiv.2401.11817 · Cited by 180 · see citing works
“A conventional classification of hallucination is the intrinsic-extrinsic dichotomy [14, 25, 29, 75].”
BackgroundKabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642596 · Cited by 160 · see citing works
Multi-Modal Hallucination Control by Visual Information Grounding
Favero, A., Zancato, L., Trager, M., Choudhary, S., Perera, P., Achille, A., Swaminathan, A., & Soatto, S. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.01356 · Cited by 299 · see citing works
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.11738 · Cited by 57 · see citing works
“[43] Yichong Huang, Xiachong Feng, Xiaocheng Feng, and Bing Qin.”
BackgroundAbstractive Meeting Summarization: A Survey
Rennard, V., Shang, G., Hunter, J., & Vazirgiannis, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00578 · Cited by 41 · see citing works
KI-MAG: A knowledge-infused abstractive question answering system in medical domain
Zafar, A., Sahoo, S. K., Bhardawaj, H., Das, A., & Ekbal, A. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127141 · Cited by 33 · see citing works
Balachandran, V., Hajishirzi, H., Cohen, W. W., & Tsvetkov, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.667 · Cited by 32 · see citing works
Angel, M. C., Rinehart, J. B., Canneson, M. P., & Baldi, P. (2024) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000006892 · Cited by 26 · see citing works
Huang, T.-Y., Hsieh, P. H., & Chang, Y. (2024) · JMIR Medical Education · doi:10.2196/59902 · Cited by 22 · see citing works
Towards Unifying Multi-Lingual and Cross-Lingual Summarization
Wang, J., Meng, F., Zheng, D., Liang, Y., Li, Z., Qu, J., & Zhou, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.843 · Cited by 29 · see citing works
Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus
Zhang, T., Qiu, L., Guo, Q., Deng, C., Zhang, Y., Zhang, Z., Zhou, C., Wang, X., & Fu, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.58 · Cited by 39 · see citing works
Mapping the Design Space of Human-AI Interaction in Text Summarization
Cheng, R., Smith, A., Zhang, K., Tetreault, J., & Jaimes-Larrarte, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.33 · Cited by 22 · see citing works
LUNA: A Model-Based Universal Analysis Framework for Large Language Models
Song, D., Xie, X., Song, J., Zhu, D., Huang, Y., Juefei-Xu, F., & Ma, L. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3411928 · Cited by 25 · see citing works
Self-citationAngel, M. C., Rinehart, J. B., Canneson, M. P., & Baldi, P. (2023) · medRxiv · doi:10.1101/2023.05.10.23289805 · Cited by 20 · see citing works
DialSummEval: Revisiting Summarization Evaluation for Dialogues
Gao, M., & Wan, X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.418 · Cited by 17 · see citing works
SEASum: Syntax-Enriched Abstractive Summarization
Liu, S., Yang, L., & Cai, X. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116819 · Cited by 17 · see citing works
MSAMSum: Towards Benchmarking Multi-lingual Dialogue Summarization
Feng, X., Feng, X., & Qin, B. (2022) · DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering (DialDoc) · doi:10.18653/v1/2022.dialdoc-1.1 · Cited by 16 · see citing works
Self-citationChallenges in Domain-Specific Abstractive Summarization and How to Overcome Them
Afzal, A., Vladika, J., Braun, D., & Matthes, F. (2023) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0011744500003393 · Cited by 19 · see citing works
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries
Tang, X., Fabbri, A. R., Li, H., Mao, Z., Adams, G., Wang, B., Çelikyılmaz, A., Mehdad, Y., & Radev, D. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.417 · Cited by 17 · see citing works
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.4 · Cited by 14 · see citing works
Evaluation of Hallucination and Robustness for Large Language Models
Hu, R. M., Zhong, J., Ding, M., Ma, Z., & Chen, M. (2023) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c60940.2023.00089 · Cited by 15 · see citing works
Active Learning for Abstractive Text Summarization
Tsvigun, A., Lysenko, I., Sedashov, D., Lazichny, I., Damirov, E., Va, K., Belousov, A., Sanochkin, L., Panov, M., Panchenko, A., et al. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.377 · Cited by 14 · see citing works
Evidential Reasoning and Learning: a Survey
Cerutti, F., Kaplan, L., & Şensoy, M. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/760 · Cited by 13 · see citing works
FaMeSumm: Investigating and Improving Faithfulness of Medical Summarization
Zhang, N., Zhang, Y., Guo, W., Mitra, P., & Zhang, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.673 · Cited by 15 · see citing works
Faithful Abstractive Summarization via Fact-aware Consistency-constrained Transformer
Lyu, Y., Zhu, C., Xu, T., Yin, Z., & Chen, E. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557319 · Cited by 11 · see citing works
Small Language Model Can Self-Correct
Han, H., Liang, J., Shi, J., He, Q., & Xiao, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29774 · Cited by 13 · see citing works
Factored Verification: Detecting and Reducing Hallucination in Summaries of Academic Papers
George, C., & Stuhlmüller, A. (2023) · Workshop on Information Extraction from Scientific Publications (WIESP) · doi:10.18653/v1/2023.wiesp-1.13 · Cited by 10 · see citing works
CO2Sum:Contrastive Learning for Factual-Consistent Abstractive SummarizationPreprint
Liu, W., Wu, H., Mu, W., Li, Z., Chen, T., & Nie, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01147 · Cited by 13 · see citing works
Automated Summarization of Stack Overflow Posts
Kou, B., Chen, M., & Zhang, T. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00158 · Cited by 13 · see citing works
OntoFact: Unveiling Fantastic Fact-Skeleton of LLMs via Ontology-Driven Reinforcement Learning
Shang, Z., Ke, W., Xiu, N., Wang, P., Liu, J., Li, Y., Luo, Z., & Ji, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29859 · Cited by 10 · see citing works
Analyzing Multi-Task Learning for Abstractive Text Summarization
Kirstein, F., Wahle, J. P., Ruas, T., & Gipp, B. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.5 · Cited by 7 · see citing works
Explain-and-Test: An Interactive Machine Learning Framework for Exploring Text Embeddings
Raval, S., Wang, C., Viégas, F. B., & WATTENBERG, M. P. (2023) · Visualization · doi:10.1109/vis54172.2023.00052 · Cited by 7 · see citing works
Enhancing consistency with the fusion of paralleled decoders for text generation
Li, Y., Huang, H., Bai, Y., & Gao, Y. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102652 · Cited by 7 · see citing works
Interactive User Interface for Dialogue Summarization
Jung, J., Seo, H., Jung, S., Chung, R., Ryu, H., & Chang, D.-S. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584057 · Cited by 6 · see citing works
Questioning the Validity of Summarization Datasets and Improving Their Factual Consistency
Guo, Y., Clavel, C., Eddine, M. K., & Vazirgiannis, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.386 · Cited by 6 · see citing works
Tackling Hallucinations in Neural Chart Summarization
Islam, S. O. U., Škrjanec, I., Dušek, O., & Demberg, V. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.30 · Cited by 6 · see citing works
A Comprehensive Evaluation of Large Language Models for Turkish Abstractive Dialogue Summarization
Büyük, O. (2024) · IEEE Access · doi:10.1109/access.2024.3454342 · Cited by 7 · see citing works
Wang, X., & Zhao, T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.180 · Cited by 5 · see citing works
Rong, H., Chen, G., Ma, T., Sheng, V. S., & Bertino, E. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3286097 · Cited by 5 · see citing works
Generative AI in Multimodal Cross-Lingual Dialogue System for Inclusive Communication Support
Nataraj, V., Liao, W.-H., Chang, Y., Chiang, C.-Y., Lin, C., Lin, Y.-A., & Day, M.-Y. (2024) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri62200.2024.00051 · Cited by 7 · see citing works
A Survey of Factual Consistency in Summarization from 2021 to 2023
Li, Y., Hao, X., & Li, L. (2023) · International Conference on Computers and Artificial Intelligence Technology (CAIT) · doi:10.1109/cait59945.2023.10468958 · Cited by 4 · see citing works
Exploring Abstractive Text Summarization: Methods, Dataset, Evaluation, and Emerging Challenges
Sunusi, Y., Omar, N., & Zakaria, L. Q. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.01507130 · Cited by 4 · see citing works
Exploring In-Context Learning for Knowledge Grounded Dialog Generation
Chen, Q., Wu, W., & Li, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.675 · Cited by 4 · see citing works
Li, S., & Xu, J. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191247 · Cited by 3 · see citing works
Axioms for Retrieval-Augmented Generation
Reimer, J. H., Fröbe, M., Stein, B., Potthast, M., & Hagen, M. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744601 · Cited by 2 · see citing works
Wang, X., Wang, J., Xu, B., Lin, H., Zhang, B., & Yang, Z. (2022) · JMIR Medical Informatics · doi:10.2196/38052 · Cited by 2 · see citing works
Improving Factual Consistency of Dialogue Summarization with Fact-Augmentation Mechanism
Li, W., Zhou, X., Bai, X., & Pan, S. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9891885 · Cited by 2 · see citing works
Beyond Pure Text: Summarizing Financial Reports Based on Both Textual and Tabular Data
Wang, Z., Jiang, Z., Zhang, X., Soon, J., Zhang, J., Xiaoyao, W., & Du, H. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/581 · Cited by 2 · see citing works
Entity-Based Evaluation of Political Bias in Automatic Summarization
Zhou, K., & Tan, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.696 · Cited by 5 · see citing works
Prompt scoring system for dialogue summarization using GPT-3Preprint
Prodan, G., & Pelican, E. (2022) · doi:10.36227/techrxiv.16652392.v2 · Cited by 2 · see citing works
Factual Accuracy Checking by Validating Numerical Data in Financial Summaries
Bera, A., & Goel, A. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825697 · Cited by 1 · see citing works
Dutta, T., & Liu, X. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650810 · Cited by 1 · see citing works
Language-Based Syllogistic Reasoning Using Deep Neural Networks
Aghahadi, Z., & Talebpour, A. (2022) · Cognitive Semantics · doi:10.1163/23526416-bja10026 · Cited by 1 · see citing works
Evaluating Parameter Efficient Learning for Generation
Xu, P., Patwary, M. A., Prabhumoye, S., Adams, V., Prenger, R., Ping, W., Lee, N., Shoeybi, M., & Catanzaro, B. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.319 · Cited by 1 · see citing works
Prompt scoring system for dialogue summarization using GPT-3Preprint
Prodan, G., & Pelican, E. (2021) · doi:10.36227/techrxiv.16652392.v1 · Cited by 1 · see citing works
Prompt scoring system for dialogue summarization using GPT-3Preprint
Prodan, G., & Pelican, E. (2021) · doi:10.36227/techrxiv.16652392 · Cited by 1 · see citing works
Wang, X., Wang, J., Xu, B., Lin, H., Zhang, B., & Yang, Z. (2022) · doi:10.2196/preprints.38052
Toward Fact-aware Abstractive Summarization Method Using Joint LearningPreprint
Mao, X., Chen, L., Ju, S., Wan, X., & Liu, Y. (2022) · Research Square · doi:10.21203/rs.3.rs-2206382/v1
Gu, D., On, B.-W., & Jeong, D. (2022) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2022.20.12.25
GERAÇÃO DE DESCRIÇÕES DE PRODUTOS A PARTIR DE AVALIAÇÕES DE USUÁRIOS USANDO UM LLM
GUTIERREZ, B. F. M. (2024) · doi:10.17771/pucrio.acad.66911
Bello-Orgaz, G., Dakhama, A., Ramirez‐Atencia, C., & Menéndez, H. D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5593064
LieToMe: A Multi-Agent System for Detecting Hallucinations in Large Language Models
Blitti, K. E. K., Tola, F. G., Shukla, M., Ladva, V., & Dholakia, N. H. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17834-3_29
Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2023) · arXiv · doi:10.48550/arxiv.2308.02312
- Authors
- Huang, Y., Feng, X., Feng, X., & Qin, B.
- Affiliations
- Harbin Institute of Technology (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2104.14839
- OpenAlex
- W3158419028
- Cited by
- 62
- References
- 37
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 13 · 2023: 22 · 2022: 21 · 2021: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- abstractive summarization, factual consistency, Seq2Seq, summarization evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
