The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization
Open accessAbstract
Alexander Fabbri, Chien-Sheng Wu, Wenhao Liu, Caiming Xiong. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Current literature on detecting and mitigating factual errors generated by machine learning models focuses predominantly on a single specific task, for example, retrieval-augmented verification models for QA (Lewis et al., 2020), hallucination detection models for text summarization (Fabbri et al., 2022), and execution-based evaluation for code (Shi et al., 2022).”
From FacTool: Factuality Detection in Generative AI - A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) · cites this paper as Fabbri et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“Building on this foundational work, Fabbri et al. [80] conducted an in-depth evaluation of the components within QA-based metrics, yielding further enhancements in faithfulness evaluation.”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [80]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
The Jackknife, the Bootstrap and Other Resampling Plans
Efron, B. (1982) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611970319 · Cited by 8,120
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
Research works printed but not linked: 4
- Dan Friedman, Ben Dodge, and Danqi Chen. 2021. Single-dataset experts for multi-dataset qa. In Em- pirical Methods in Natural Language Processing (EMNLP).
- Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2020. FFCI: A framework for interpretable auto- matic evaluation of summarization. ArXiv preprint, abs/2011.13662. · arXiv
- Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020b. Asking and answering questions to evaluate the fac- tual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 5008-5020, Online. Asso- ciation for Computational Linguistics. · DOI
- PMLR.
Cited by
Sorted by citations
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 4,688
“Building on this foundational work, Fabbri et al. (2022) conducted an in-depth evaluation of the components within QA-based metrics, yielding further enhancements in faithfulness evaluation.”
Benchmarking Large Language Models for News Summarization
Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00632 · Cited by 415
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.741 · Cited by 407
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516
“Building on this foundational work, Fabbri et al. (2022) conducted an in-depth evaluation of the components within QA-based metrics, yielding further enhancements in faithfulness evaluation.”
Anh-Hoang, D., Tran, V., & Nguyen, L.-M. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1622292 · Cited by 146
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Gehrmann, S., Clark, E., & Sellam, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13715 · Cited by 123
AlignScore: Evaluating factual consistency with a unified alignment function
Zha, Y., Yang, Y., Li, R., & Hu, Z. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.634 · Cited by 81
TRUE: Re-evaluating Factual Consistency Evaluation
Honovich, O., Aharoni, R., Herzig, J., Taitelbaum, H., Kukliansy, D., Cohen, V., Scialom, T., Szpektor, I., Hassidim, A., & Matias, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.287 · Cited by 113
Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation
Liu, Y., Fabbri, A. R., Liu, P., Zhao, Y., Nan, L., Han, R., Han, S., Joty, S., Wu, C.-S., Xiong, C., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.228 · Cited by 108
Self-citationROUGE-SEM: Better evaluation of summarization using ROUGE combined with semantics
Zhang, M., Li, C., Wan, M., Zhang, X., & Zhao, Q. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121364 · Cited by 61
Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation
Bulian, J., Buck, C., Gajewski, W., Börschinger, B., & Schuster, T. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.20 · Cited by 44
Factual consistency evaluation of summarization in the Era of large language models
Luo, Z., Xie, Q., & Ananiadou, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124456 · Cited by 59
SummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization
Laban, P., Kryściński, W., Agarwal, D., Fabbri, A. R., Xiong, C., Joty, S., & Wu, C.-S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.600 · Cited by 44
Self-citationA Critical Evaluation of Evaluations for Long-form Question Answering
Xu, F., Song, Y., Iyyer, M., & Choi, E. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.181 · Cited by 33
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Laban, P., Vig, J., Hearst, M., Xiong, C., & Wu, C.-S. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676419 · Cited by 31
Self-citationA Field Guide to Automatic Evaluation of LLM-Generated Summaries
van Schaik, T. A., & Pugh, B. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3661346 · Cited by 30
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Krishna, K., Bransom, E., Kuehl, B., Iyyer, M., Dasigi, P., Cohan, A., & Lo, K. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.121 · Cited by 29
Multilingual Summarization with Factual Consistency Evaluation
Aharoni, R., Narayan, S., Maynez, J., Herzig, J., Clark, E., & Lapata, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.220 · Cited by 47
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29
“, 2020), hallucination detection models for text summarization (Fabbri et al., 2022), and execution-based evaluation for code (Shi et al.”
BackgroundMarvista: Exploring the Design of a Human-AI Collaborative News Reading Tool
Chen, X., Wu, C.-S., Murakhovs’ka, L., Laban, P., Niu, T., Liu, W., & Xiong, C. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3609331 · Cited by 35
Self-citationZero-shot Faithful Factual Error Correction
Huang, K.-H., Chan, H. P., & Ji, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.311 · Cited by 26
Detecting and Mitigating Hallucinations in Multilingual Summarisation
Qiu, Y., Ziser, Y., Korhonen, A., Ponti, E. M., & Cohen, S. B. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.551 · Cited by 35
Reading Subtext: Evaluating Large Language Models on Short Story Summarization with Writers
Subbiah, M., Zhang, S., Chilton, L. B., & McKeown, K. R. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00702 · Cited by 34
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters
Schuster, T., Chen, S., Buthpitiya, S., Fabrikant, A., & Metzler, D. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.28 · Cited by 17
Lee, S. Y.-T., Bahukhandi, A., Liu, D., & Ma, K. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456398 · Cited by 19
FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence
Joseph, S., Chen, L., Trienes, J., Göke, H., Coers, M., Xu, W., Wallace, B., & Li, J. J. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.459 · Cited by 17
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question
Mohammadshahi, A., Scialom, T., Yazdani, M., Yanki, P., Fan, A., Henderson, J., & Saeidi, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.428 · Cited by 28
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Fabbri, A. R., Choubey, P. K., Vig, J., Wu, C.-S., & Xiong, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.623 · Cited by 21
Self-citationCaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization
Choubey, P. K., Fabbri, A. R., Vig, J., Wu, C.-S., Liu, W., & Rajani, N. F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.685 · Cited by 26
Self-citationFActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2305.14251 · Cited by 14
“This includes model-based evaluation in summarization that considers the consistency between a summary and a source document using QA or NLI (Kryscin-ski et al., 2020; Wang et al., 2020; Fabbri et al., 2022; Deutsch et al., 2021; Laban et al., 2022).”
MethodImproving Factuality of Abstractive Summarization via Contrastive Reward Learning
Chern, I.-C., Wang, Z., Das, S. K., Sharma, B., Liu, P., & Neubig, G. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.6 · Cited by 20
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge
Feng, S., Balachandran, V., Bai, Y., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.59 · Cited by 40
Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension
Agrawal, S., & Carpuat, M. J. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00653 · Cited by 17
MedKA: A knowledge graph-augmented approach to improve factuality in medical Large Language Models
Deng, Y., Zhao, S., Miao, Y., Zhu, J., & Li, J. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104871 · Cited by 18
On Improving Summarization Factual Consistency from Natural Language Feedback
Liu, Y., Deb, B., Teruel, M., Halfaker, A., Radev, D., & Awadallah, A. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.844 · Cited by 25
Automatic Multilingual Question Generation for Health Data Using LLMs
Ackerman, R., & Balyan, R. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-7587-7_1 · Cited by 13
FaMeSumm: Investigating and Improving Faithfulness of Medical Summarization
Zhang, N., Zhang, Y., Guo, W., Mitra, P., & Zhang, R. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.673 · Cited by 18
A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarization
Adams, G., Zucker, J. E., & Elhadad, N. (2023) · PubMed · doi:10.48550/arxiv.2303.03948 · Cited by 22
Attributable and Scalable Opinion Summarization
Hosking, T., Tang, H., & Lapata, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.473 · Cited by 16
Yang, F., Liu, X. C., Lu, L., Wang, B., & Liu, C. (2025) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981251322468 · Cited by 12
Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization
Chan, H. P., Zeng, Q., & Ji, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.402 · Cited by 8
Murakhovs’ka, L., Laban, P., Xie, T., Xiong, C., & Wu, C.-S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.657 · Cited by 9
Self-citationLiu, Z., Kou, J., Zhang, W., Gu, C., Fang, X., Huang, Z., Hao, Y., Li, H., Lu, X., Yin, A., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189137 · Cited by 9
SWING: Balancing Coverage and Faithfulness for Dialogue Summarization
Huang, K.-H., Singh, S., Ma, X., Xiao, W., Nan, F., Dingwall, N., Wang, W. Y., & McKeown, K. R. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.37 · Cited by 12
Expository Text Generation: Imitate, Retrieve, Paraphrase
Balepur, N., Huang, J., & Chang, K. C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.729 · Cited by 16
LENS: Layers of Evaluation of Hallucination in GenAI Systems
Gu, C., Zhang, W., Huang, Z., Kou, J., Liu, Z., Zhao, C., Liu, C., Zhang, L., Lin, W., Wang, Z., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189150 · Cited by 32
Ethical perspectives on deployment of large language model agents in biomedicine: a survey
Nia, N. G., Amiri, A., Luo, Y., & Kline, A. S. (2025) · AI and Ethics · doi:10.1007/s43681-025-00847-w · Cited by 7
Fast and Accurate Factual Inconsistency Detection Over Long Documents
Lattimer, B. M., Chen, P. M., Zhang, X., & Yang, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.105 · Cited by 15
Evaluating and Improving Factuality in Multimodal Abstractive Summarization
Wan, D., & Bansal, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.654 · Cited by 12
Factuality Guided Diffusion-Based Abstractive Summarization
Shin, J., Park, H., & Song, H.-J. (2024) · IEEE Access · doi:10.1109/access.2024.3467063 · Cited by 5
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization
Kamoi, R., Goyal, T., & Durrett, G. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.11 · Cited by 8
Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization
Cheang, C., Chan, H. P., Wong, D. F., Liu, X., Li, Z., Sun, Y., Liu, S., & Chao, L. S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1007 · Cited by 9
Li, Y., Li, L., Hu, D., Li, Y., Zhou, Y., Litvak, M., & Vanetik, N. (2023) · Research Square · doi:10.21203/rs.3.rs-3695932/v1 · Cited by 7
A Survey of Factual Consistency in Summarization from 2021 to 2023
Li, Y., Hao, X., & Li, L. (2023) · International Conference on Computers and Artificial Intelligence Technology (CAIT) · doi:10.1109/cait59945.2023.10468958 · Cited by 4
Gao, M., Wan, X., Su, J., Wang, Z., & Huai, B. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.779 · Cited by 5
Aligning Factual Consistency for Clinical Studies Summarization through Reinforcement Learning
Tang, X., Cohan, A., & Gerstein, M. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.7 · Cited by 4
WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning
Wu, W., Li, W., Xiao, X., Liu, J., Li, S., & Lyu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.18 · Cited by 11
Concise Answers to Complex Questions: Summarization of Long-form Answers
Potluri, A., Xu, F., & Choi, E. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.541 · Cited by 7
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation
Liu, Y., Fabbri, A. R., Zhao, Y., Liu, P., Joty, S., Wu, C.-S., Xiong, C., & Radev, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.1018 · Cited by 14
Self-citationDecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4
Hu, Y., Song, K., Cho, S., Wang, X., Foroosh, H., & Liu, F. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.519 · Cited by 8
Understanding Faithfulness and Reasoning of Large Language Models on Plain Biomedical Summaries
Fang, B., Dai, X., & Karimi, S. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.578 · Cited by 4
Zhu, Y., Li, Y., Li, L., Hu, D., Hao, X., Chen, D., Zhang, X., Zhang, Z., Zhou, Y., Litvak, M., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3567744 · Cited by 4
Walking a Tightrope – Evaluating Large Language Models in High-Risk Domains
Hung, C.-C., Rim, W. B., Frost, L., Bruckner, L., & Lawrence, C. (2023) · GenBench Workshop on Generalisation (Benchmarking) in NLP (GenBench) · doi:10.18653/v1/2023.genbench-1.8 · Cited by 13
CASESUMM: A Large-Scale Dataset for Long-Context Summarization from U.S. Supreme Court OpinionsPreprint
Heddaya, M., MacMillan, K., Malani, A., Mei, H., & Tan, C. (2025) · Qeios · doi:10.32388/ej9r29 · Cited by 4
A comprehensive study on factual consistency in LLMs for data-to-text generation
Mahapatra, J., & Garain, U. (2025) · Sadhana · doi:10.1007/s12046-025-02860-5 · Cited by 6
USB: A Unified Summarization Benchmark Across Tasks and Domains
Krishna, K., Gupta, P., Ramprasad, S., Wallace, B., Bigham, J. P., & Lipton, Z. C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.592 · Cited by 3
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage
Laban, P., Wu, C.-S., Murakhovs’ka, L., Chen, X., & Xiong, C. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.380 · Cited by 7
Self-citationScirè, A., Ghonim, K., & Navigli, R. (2024) · arXiv · doi:10.48550/arxiv.2403.02270 · Cited by 2
Bayesian Calibration of Win Rate Estimation with LLM Evaluators
Gao, Y., Xu, G., Wang, Z., & Cohan, A. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.273 · Cited by 1
“…text include heuristic-based evaluation metrics (Papineni et al., 2002; Lin, 2004; Pillutla et al., 2021), model-based evaluation metrics (Zhang et al., 2019; Fabbri et al., 2022; Zha et al., 2023; Chen and Eger, 2023), and recently, LLM-based evaluations (Kim et al., 2024a,b; Wang et al., 2024).”
MethodVisual Storytelling with Question-Answer Plans
Liu, D., Lapata, M., & Keller, F. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.386 · Cited by 6
Liu, J., Du, W., Li, Q., Wang, X., Zhou, Z., & Liu, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448276 · Cited by 1
Balancing factual consistency and informativeness for abstractive summarization
Li, Z., Ou, L., & Huang, Y. (2025) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02724-8 · Cited by 1
Mind the Link: Discourse Link-Aware Hallucination Detection in Summarization
Lee, D., Jung, H., & Choi, Y. S. (2025) · Applied Sciences · doi:10.3390/app151910506 · Cited by 1
Language Models for Automated Market Commentary from Corporate Disclosures
Assis, G., Dutra, H., Vianna, D., Meira, W., da Silva, A. S., & Paes, A. (2025) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3768292.3770438 · Cited by 1
Quantum-Inspired Cross-Attention Alignment for Turkish Scientific Abstractive Summarization
ALTAY, G., & Küçüksille, E. U. (2025) · Electronics · doi:10.3390/electronics14224474 · Cited by 1
Wu, F., Zhang, Y., Li, G., Li, J., & Ma, J. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3658765 · Cited by 2
Wang, Z., Li, L., Chen, L., Xu, X., Li, M., & Su, Y. (2025) · International Conference on Wireless Communication, Networking and Internet of Things (WCNIoT) · doi:10.1109/wcniot67424.2025.11381315 · Cited by 1
Exploring question answering: metric analysis and evaluation framework for enhanced interpretability
Navarro, L. C., Mucciaccia, S. S., Mutz, F. W., Paixão, T. M., Badue, C. S., Souza, A. F. D., & Oliveira-Santos, T. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12053-8 · Cited by 1
Zhang, H., Vithanage, D., Deng, C., & Yu, P. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105053 · Cited by 2
Unsupervised Summarization Re-ranking
Ravaut, M., Joty, S., & Chen, N. F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.529 · Cited by 1
Annotating and Detecting Fine-grained Factual Errors for Dialogue Summarization
Zhu, R., Qi, J., & Lau, J. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.377 · Cited by 7
Multi-View Source Ablation for Faithful Summarization
Cao, S., Ma, L., Lu, D., Logan, R. L., Tetreault, J., & Jaimes, A. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.151 · Cited by 1
Guo, B., Feng, C., Liu, F., Li, X., & Wang, X. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-7894-6_11 · Cited by 1
Improving Consistency for Text Summarization with Energy Functions
Zeng, Q., Yin, Q., Li, Z., Gao, Y., Nag, S., Wang, Z., Yin, B., Ji, H., & Zhang, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.798 · Cited by 1
Dutta, T., & Liu, X. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650810 · Cited by 1
New Paradigm for Evaluating Scholar Summaries: A Facet-aware Metric and a Meta-evaluation Benchmark
Wang, T., Chen, X., Zhu, Q., Guo, T., Gao, S., Lu, Z., Gao, X., & Zhang, X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3733597 · Cited by 1
You, Z., & Guo, Y. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105019 · Cited by 2
GIER: Gap-Driven Self-Refinement for Large Language Models
Dewri, R. (2025) · arXiv · doi:10.48550/arxiv.2509.00325
“…proposed fine-grained evaluation dimensions such as factual consistency, completeness, relevance, and helpfulness to assess LLM outputs across tasks like summarization and QA (Askell et al., 2021; Durmus et al., 2020; Fabbri et al., 2022; Tam et al., 2023; Wang et al., 2024; Zhang et al., 2024).”
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics
Ma, L., Cao, S., Logan, R. L., Lu, D., Ran, S., Zhang, K., Tetreault, J., & Jaimes, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.716 · Cited by 3
Multi Document Summarization Evaluation in the Presence of Damaging Content
Manevich, A., Carmel, D., Cohen, N., Kravi, E., & Shapira, O. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1
Fact-checking benchmark for the Russian Large Language Models
SberDevices, Kozlova, A., Shevelev, D., & Fenogenova, A. (2023) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2023-22-267-277
Integrating Concept Associations for Query Focused Knowledge Summarization
Wang, J., Liu, Z., Sun, Y., & Li, X. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890022
Mitigating Multi-hop Hallucination in Large Language Models with Non-authoritative Knowledge Sources
Yuan, H., Cao, P., Jin, Z., Chen, Y., Liu, K., & Zhao, J. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-1809-5_17
Methods to Evaluate and Improve the Quality of Text Summarization
Nguyen, H. (2025) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc2443390
Tong, X., Zhu, Y., Fan, S., & Xu, L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0014-7_12
FARE: Factual Alignment for Reliable Evaluation in Text Summarization
Wang, Z., Zhou, C., Zhang, J., & Hu, D. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-94898-5_54
Hao, X., Li, Y., & Li, L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3352-7_10
LLM-Based Abstractive Summarization of PubMed Articles
Ho, E., Heidari, M., Zad, S., Boroojeni, K. G., & Headden, C. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-0433-6_14
Localizing Factual Inconsistencies in Attributable Text Generation
Cattan, A., Roit, P., Zhang, S., Wan, D., Aharoni, R., Szpektor, I., Bansal, M., & Dagan, I. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.598 · Cited by 11
Metadata
- Authors
- Fabbri, A., Wu, C.-S., Liu, W., & Xiong, C.
- Affiliations
- Salesforce (United States) (US)
- Venue
- Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2022
- Pages
- 2587-2601
- OpenAlex
- W4226118367
- Cited by
- 119
- References
- 50
- Citations by year
- 2026: 30 · 2025: 21 · 2024: 14 · 2023: 48 · 2022: 6
- FWCI
- 12.30
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- question answering, summarization, factual accuracy, abstractive summarization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
