Benchmarking Large Language Models for News Summarization
Open accessThe DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have shown promise for automatic summarization but the reasons behind their successes are poorly understood. By conducting a human evaluation on ten LLMs across different pretraining methods, prompts, and model scales, we make two important observations. First, we find instruction tuning, not model size, is the key to the LLM’s zero-shot summarization capability. Second, existing studies have been limited by low-quality references, leading to underestimates of human performance and lower few-shot and finetuning performance. To better evaluate LLMs, we perform human evaluation over high-quality summaries we collect from freelance writers. Despite major stylistic differences such as the amount of paraphrasing, we find that LLM summaries are judged to be on par with human written summaries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For summarization tasks, although LLMs do not have an obvious advantage over fine-tuned models under traditional automatic evaluation metrics, such as ROUGE [70], human evaluation results indicate that humans tend to prefer the results generated by LLMs [42,138] compared to that of fine-tuned models.”
From Harnessing the power of llms in practice: a survey on chatGPT and beyond (2024) · cites this paper as [138]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“LLMs are known to produce errors in many settings, from document summarisation7, to general reasoning tasks as well as more clinically relevant tasks8.”
From A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation (2025) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“LLMs (Bommasani et al., 2021; Chowdhery et al., 2022; Brown et al., 2020) have two distinctive features over previous pretrained models.”
BackgroundSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“In this work, we evaluate the effectiveness of automatic metrics for evaluating LLMs and show that the usefulness of reference-based evaluation is closely linked to the quality of the references.”
MethodNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“In this work, we evaluate the effectiveness of automatic metrics for evaluating LLMs and show that the usefulness of reference-based evaluation is closely linked to the quality of the references.”
Method(2004) · Cited by 9,718
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · cited 2×
“In this work, we evaluate the effectiveness of automatic metrics for evaluating LLMs and show that the usefulness of reference-based evaluation is closely linked to the quality of the references.”
MethodTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 2×
“In this approach, a diverse set of natural language processing tasks are reformulated into the prompting format and the LLM’s parameters are updated for these tasks either through supervised finetuning or reinforcement learning.”
Method(2005) · Cited by 4,152
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775 · cited 2×
“In this work, we evaluate the effectiveness of automatic metrics for evaluating LLMs and show that the usefulness of reference-based evaluation is closely linked to the quality of the references.”
BackgroundMethodTraining Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 2×
“In this approach, a diverse set of natural language processing tasks are reformulated into the prompting format and the LLM’s parameters are updated for these tasks either through supervised finetuning or reinforcement learning.”
Method
Research works printed but not linked: 1
- Conroy, 2006, Back to basics: Classy 2006, 10.12968/sece.2006.11.755s · doi:10.12968/sece.2006.11.755s
Cited by
Items 301–400 of 427
Neural headline generation: A comprehensive survey
Ren, H., Chang, X., & Li, X. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129633 · Cited by 1
Investigating Fine-Tuning of Large Language Models for Text Summarisation
Khaliq, U., & Patel, P. (2025) · Smart innovation, systems and technologies · doi:10.1007/978-981-96-0139-4_6
On Comparing LLM-Generated Causal Networks with a Rule-Based Approach
Sheikh, S. J., & Haider, S. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-5887-9_5
Making language models better human-like learners
Qin, C. (2025) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/183806
Yepez, J., Tavares, B., Peres, F. D. C. L., & Becker, K. E. (2024) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat62293.2024.00026
“It has displayed good performance in a number of NLP tasks, summarization among them [16].”
ArcTEX—a novel clinical data enrichment pipeline to support real-world evidence oncology studies
Tait, K., Cronin, J., Wiper, O., Wallis, J., Davies, J., & Dürichen, R. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1561358 · Cited by 1
Empirical Study on Translating Toxic Sentences with Large Language Model vs. Machine Translation
Supryadi, & Xiong, D. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-5123-8_2
Evaluating the Effectiveness of GPT-3.5 and BERT in Classifying Bangla News Articles
Arif, M., & Salam, A. (2024) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit64611.2024.11022328
“5 excel in zero-shot learning and news summarization, demonstrating high effectiveness even without extensive fine-tuning [7], [8].”
One Arrow, Many Targets: Probing LLMs for Multi-Attribute Controllable Text Summarization
Roy, T., & Mishra, R. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8186-0_23 · Cited by 1
Jiang, F., Yang, L., Lu, Y., & Li, H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6599-0_24
Systematic Review of Large Language Models and Future Directions
Solomon, E., Woubie, A., & Emiru, B. S. (2025) · International Conference on Information and Computer Technologies (ICICT) · doi:10.1109/icict64582.2025.00036
Large-scale transformer-based topic graphs identify thematic links between engineering and biology
Douard, N., Cavallucci, D., Samet, A., & Giakos, G. K. (2025) · Scientific Reports · doi:10.1038/s41598-025-15067-9 · Cited by 1
The Use of Generative AI in Legal and Related Sectors
Zou, M., Poncibò, C., Ebers, M., & Calo, R. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009492553.028
Li, R. (2025) · SHS Web of Conferences · doi:10.1051/shsconf/202522004007
Automated detection of affected libraries from vulnerability reports
Xu, J., Zhang, H., Zhou, X., Yang, Y., Mao, R., Li, X., Yang, L., & Shen, H. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00540-6 · Cited by 1
A Multi-Document Summarization Method for Customer Feedback Based on Large Language Models
Bao, S., Guan, X., Cao, B., & Yang, C. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-93251-9_15
Harnessing AI for Business Insight: Key Considerations for Deploying LLMs in Summarization Pipelines
Miller, A. P., Hosanagar, K., & Krishnan, R. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198940272.013.0045
Risk-Aware Automatic Text Summarization with Pre-Identification of Risk Categories and Emphasis
Huang, H., Liu, H., Wu, X., & Cai, Y. (2025) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew67478.2025.00034
“Sub-sequent studies evaluating the use of LLMs in ATS have shown that LLMs can surpass human performance in certain aspects, such as speed and accuracy [2]–[4].”
Abstractive Summarization for Urdu Video Description Generation
Faheem, A., Ullah, F., Ayub, M. S., & Karim, A. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3762992 · Cited by 1
“Human evaluation assesses subjective qualities like fluency, coherence, and relevance, ensuring the summaries meet linguistic and contextual standards [70].”
The Irrational LLM: Implementing Cognitive Agents with Weighted Retrieval-Augmented Generation
Sun, D., Lyu, Y., Li, J., Liu, X., Kara, D., Lebiere, C. J., & Abdelzaher, T. (2025) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn65249.2025.11134012 · Cited by 1
Zhang, L., Zhou, N., Qiu, Z., Ming, M., Cui, D., & Qin, Y. (2025) · Asia-Pacific Conference on Communications Technology and Computer Science (ACCTCS) · doi:10.1109/acctcs66275.2025.00045
“Compared with the traditional methods based on rules or statistics, the output text shows obvious advantages in logical coherence, semantic accuracy and creative expression[6].”
Mesnage, C. S., Wang, X., Dong, H., & Aishwaryaprajna, N. A. (2025) · Intelligenza Artificiale · doi:10.1177/17248035251363882
“Pre-trained large language models (LLMs) are approaching or have exceeded human-level performance in various tasks including abstractive summarization, question answering in some domains, some coding tasks, etc. Bommarito & Katz, 2022; OpenAI, 2023; Zhang et al., 2024.”
Dyslexia-Friendly Summarized Document Enhancement for Improved Readability
Kulkarni, S., Thakur, S., Vyass, T., Yasaswini, V., & Kamat, P. V. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-7511-1_2
Hierarchical Attention-Based GAN with Transformer for Abstractive Text Summarisation
Swetha, G., & Phanikumar, S. (2025) · Journal of Information & Knowledge Management · doi:10.1142/s0219649225500984
Refine-and-Answer: Enhancing Inference Answers with Pseudo-Triples
Yufang, W., Tang, X., & Hao, K. (2025) · Chinese Control Conference (CCC) · doi:10.23919/ccc64809.2025.11178764
“Additionally, Zhang et al. found that news summaries generated by LLMs perform comparably to human-written summaries in evaluations[26].”
Leveraging Large Language Models for Dataset Discovery
Chen, T., Schott, K., Mathiak, B., & Kern, D. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1242
“…Model (LLM)- based conversational agents (CAs) hold the promise of reducing users’ cognitive load during information searches, such as by providing text summarization (Zhang et al., 2024; Deroy et al., 2023), while also enabling follow-up que stions based on the CA’s responses (Yu et al., 2024).”
Fang, S., Cheng, K., Zhang, J., Qin, Z., & Zhang, M. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3537-8_22
APIDocBooster: An Extract-Then-Abstract Framework for Augmenting API Documentation
Yang, C., Liu, J., Xu, B., Treude, C., Lyu, Y., He, J., Li, M., & Lo, D. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00014 · Cited by 15
“Lately, large language models (LLMs) have shown promise in abstractive summarization tasks [19], [20], [21], and the latest model GPT-4 can produce particularly concise and coherent summaries [20], [21].”
Improving LLM-Based Document-Level MT with Multi-Knowledge Fusion
Liu, B., Lyu, X., Li, J., Wei, D., Zhang, M., Tao, S., & Yang, H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3349-7_14 · Cited by 1
Aspect-Based Summarization with Self-Aspect Retrieval Enhanced Generation
Feng, Y., Zhao, S., Li, Y., Xiao, L., Wu, X., & Luu, A. T. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228609 · Cited by 4
“Research shows that LLM-generated summaries are often preferred by humans over those produced by earlier techniques [12].”
Generative AI: Empowering Marathi NLP with Fine-Tuned Transformer Models
Sable, M., Panicker, A., Ansh, A., Sharma, A., Patil, S., & Kotecha, K. V. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-05545-3_12
Cyber Threat Intelligence Report Summarization with Named Entity Recognition
Mimoto, T., Kita, K., Gempei, Y., Isohara, T., Kiyomoto, S., & Tanaka, T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4674-9_24 · Cited by 1
Ad Hoc LLM-to-LLM Evaluation Framework
Almaslukh, A. (2025) · International Conference on Electrical, Communication, and Computer Engineering (ICECCE) · doi:10.1109/icecce67514.2025.11257953
“A recent work [9] benchmarks Large Language Models (LLMs) for news summarization.”
AWMT: Automatic jailbreaking attack framework utilizing working-memory trees
Zhang, Z., Xu, J., Li, B., Sun, Y., Mo, H. M., & Chen, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130643 · Cited by 7
Triple-based Factual Correctness Evaluation of AI-Generated Summaries
Latipov, I.-A., Holenderski, M., & Meratnia, N. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.12.107
Harmonizing Hindi Horizons: Assessing Transformer Variants for News Translation and Abstraction
Khade, S., Biramane, O., Bothara, O., Bhagade, V., Patil, R. R., Bobhate, G., & Kaur, G. (2025) · International Conference on Computing Communication Control and automation · doi:10.1109/iccubea65967.2025.11283657
LLM-Rank: An Unsupervised Keyword Extraction Method Using Local Large Language Models
Jia, X., Roller, C., & Wang, C. (2025) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri66576.2025.00021 · Cited by 1
“Their ability to model context and long-range dependencies enables better semantic comprehension [8]–[11].”
BLUEX Revisited: Enhancing Benchmark Coverage with Automatic Captioning
Santos, J. G. A., Bonás, G. K., & Almeida, T. S. (2025) · doi:10.5753/eniac.2025.14256 · Cited by 2
Gao, C., & Zhong, L. (2025) · International Symposium on Artificial Intelligence for Education (ISAIE) · doi:10.1145/3775073.3775117
“They are now the seed of any media content production [13].”
Exploiting Shared Adversarial Features for Dynamic Attacks in Large Vision-Language Models
Qian, Y., Zhu, X., Bao, Q., Yu, F., Ji, S., Gu, Z., Wang, W., Wang, B., & Lei, Z. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3648873 · Cited by 2
“For image captioning, Zhang et al. [65] propose that traditional evaluation methods perform poorly in the evaluation of large models.”
Kumar, S., Kumar, S., & Singh, S. R. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132522
A Deep User Interface for Exploring LLaMa
Perumal, D. B., & Panda, S. (2025) · doi:10.14236/ewic/bcshci2025.46
LISLLM: Long Context Inference of Large Language Models with Short KV Cache
Liu, T., Li, G., Zhu, Z., Mo, Z., Yao, X., & Cheng, J. (2025) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads67057.2025.11323105
“They have been widely applied in dialogue systems [10], [11], chatbots [12], [13], text summarization [14], code completion [8], [9], healthcare [5]–[7], etc.”
Acknowledging Valued Perspectives! User-Assisted Summarization by Leveraging Opinions and LLMs
Kumar, R., Gupta, T., Kalra, K., & Saha, S. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3654914
“Zhang et al. [33] highlighted instruction tuning’s role in enhancing zero-shot summarization and emphasized high-quality references for reliable evaluation.”
Summarization of News Articles Using Deep Learning Techniques
Prathyakshini, Prathwini, Hegde, U., Bhat, P., Shetty, K., & Dishan (2025) · International Conference on Communication and Electronics Systems (ICCES) · doi:10.1109/icces67310.2025.11337024
“[9] found how can the large language models can summarize the news articles and the aim to see is,can they match human-like writings.”
UniBreak: A Unified Evolutionary Token-Level Jailbreaking Framework for Large Language Models
You, S., Jiang, W., Mei, H., Gong, D., Li, Z., Yu, J., Ji, J., Lin, Q., Li, X., & Wong, K. (2026) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2026.3656951 · Cited by 1
Wu, P. Y., Chou, H.-W., & Chen, K.-Y. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3659424
“Unlike classic models, LLMs leverage reinforcement learning from human feedback (RLHF) [8], fine-tuning their outputs to align more closely with human preferences, further widening the performance gap with classic models [5], [9], [10].”
Arslan, Ş. N., Görmez, Y., & Arslan, H. (2026) · Bilgisayar bilimleri ve mühendisliği dergisi. · doi:10.54525/bbmd.1838196
Leveraging LLMs for Smart Cities Qualitative Data Analysis
Covato, E., Soomro, K., Khan, Z., Bilal, M., Cormack, R., & Green, S. (2026) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70547
Heterogeneous Information-Driven Contrastive Optimization for Abstractive Summarization
Wang, P., Lu, J., & Gong, S. (2026) · Information Systems Frontiers · doi:10.1007/s10796-026-10704-8
“With the powerful text generation abilities of large lan - guage models (LLMs), the majority of current mainstream abstractive summarization models are built upon these foun - dations (Meer et al., 2024 ; Zhang et al., 2024 ).”
Zeng, Z., Chen, Z., & Zhuang, H. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02952-y · Cited by 1
Leveraging Large Language Models for Abstractive Text Summarization
Zergoug, L., & Hantach, R. (2025) · International Conference on Foundation and Large Language Models (FLLM) · doi:10.1109/fllm67465.2025.11390908
“Zhang et al. [18] used GPT-3 with instruction tuning, demonstrating competitive performance, especially with prompt-based control over summary style.”
Negative-Aware Temporal-Causal Summarization of News Texts Using LLMs
Kou, Y., & Zhang, W. (2025) · International Conference on Machine Learning and Natural Language Processing (MLNLP) · doi:10.1109/mlnlp66797.2025.11389034
“Zhang et al. (2023) benchmarked various LLMs, finding that GPT-4 excels while smaller LMs (e.g., Qwen, Zephyr) show promise [43].”
Domain-Specific Contract Analysis: LLMs vs. Engineers in Power Plant TOR Review
Kongurai, K. (2025) · Conference of the Electric Power Supply Industry (CEPSI) · doi:10.1109/cepsi66359.2025.11403579
“These models may produce errors or hallucinations when specialized knowledge is required, potentially compromising the integrity of critical engineering specifications [4].”
Integrating cross-view multi-scale perception and RAG-enabled expert fusion for medical prediction
Wang, C., Liu, Y., Yu, Y., Ouyang, C., & Wan, Y. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.004
Karandikar, S., Merchant, K. A., Atre, M. P., & Suryawanshi, A. (2025) · International Conference on Algorithms, Computing and Artificial Intelligence (ACAI) · doi:10.1109/acai68217.2025.11406499
DiSCo: Making Absence Visible in Intelligent Summarization Interfaces
Fainman, E., Shoshan, H. B., Solomon, A., & Mokryn, O. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789152 · Cited by 1
“Zhang et al. [50] demonstrated that instruction tuning—how the task is framed in the prompt—plays a greater role than model size in determining summarization quality, while Liu et al. [26] showed that even state-of-the-art models still struggle with maintaining factual consistency and controllable coverage across mult…”
RankCut: A Ranking-Based LLM Approach to Extractive Summarization for Transcript-Based Video Editing
Shah, S., Leake, M., Chu, K., Weber, C., Becherer, N., & Wermter, S. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789115 · Cited by 1
Hierarchical BiLSTM with Attention for Text Summarization
Chandel, A. S., Gupta, K., & Kumar, A. (2025) · IEEE International Conference on Recent Advances in Computing and Systems (REACS) · doi:10.1109/reacs67479.2025.11413340
Krithika, S., & Mohan, G. B. (2026) · SN Computer Science · doi:10.1007/s42979-026-04852-6 · Cited by 1
Towards Uncertainty-Aware Low-Bit Quantized LLMs for On-Device Inference
Sparrenberg, L., Schneider, T., Deußer, T., Berger, A., & Sifa, R. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11400739
“Large language models (LLMs), particularly those utilizing Transformer architectures, have significantly advanced natural language understanding and generation, underpinning applications from interactive chatbots to automated summarization [1], [2].”
Dynamic Retrieval Strategy for Summarizing Doctor–Patient Dialogues with RAG
Shi, Y., & Hu, X. T. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402171
Retrospective and Future Views
McKeown, K. R. (2026) · Computational Linguistics · doi:10.1162/coli.a.605
Self-citationAbdulai, H., Manresa-Yee, C., Rexha, H., & Lafond, S. (2026) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ae5442
Khan, R., & Nayak, A. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.01.018
Lei, Y., Zheng, S., Wang, R., Pang, L., He, L., Chen, S., Wang, Y., Shen, H., Cheng, X., & Wang, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_21
The Evolution of Text Summarization: Advancements and Challenges in Low-Resource Languages
Begum, A., & Padmaja, D. L. (2025) · International Conference on Research in Intelligent Computing in Engineering (RICE) · doi:10.1109/rice67503.2025.11439926
“These intrinsic measures provide an effective alternative that correlates strongly with human judgment, proving especially valuable when ideal references are absent, and thereby expanding the toolkit to reliably assess contemporary summarization systems [37] [38] [39].”
Contributions to evaluating and improving the faithfulness for text generation
Zhang, K. (2025) · theses.fr (ABES) · doi:10.70675/28453a20z78f2z4fa2za6edz700065e69e97
Iqbal, S., Ahmadi, M., Ahmadian, D., & Eskander, P. N. (2026) · Cureus · doi:10.7759/cureus.106804
“LLMs are known to make errors in document summarization [17] and clinical tasks [18,19], prompting caution regarding their use in healthcare.”
QuotAnaSum: Interactive News Summary Generation through Quotation AnalysisPreprint
Han, S., Li, G., Wang, S., Cao, P., Bai, Y., & Liu, C. H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6554165
What Stays and What Goes: Auditing the Impact of LLM Summarization on News Partisanship
Savgira, P., Kreiss, E., & Hosseinmardi, H. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799057 · Cited by 1
Song, C., Lee, K., & Lim, H. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798530
“Recent advances in Large Language Models (LLMs) further enhance these capabilities, enabling flexible textual adaptation while largely preserving semantic content [15, 32].”
Aloraini, E., Shaban, K., Hamdi, A., & Elmahjub, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6582144 · Cited by 1
Large Language Models as MT Evaluators: Statistical Evidence from a Low-Resource Setting
Munková, D., Munk, M., Chotar, M., & Hajek, P. (2026) · International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA) · doi:10.1109/acdsa67686.2026.11468166
“These improvements stem from LLMs’ increasing capabilities in contextual understanding, instruction following, and access to large knowledge bases [9] Advanced LLMs, such as text-davinci-003 and ChatGPT have shown the ability to replicate expert human judgments across multiple evaluation…”
Kim, S., Hahn, H., & Kwon, T. (2025) · Journal of Society of Korea Industrial and Systems Engineering · doi:10.11627/jksie.2025.48.4.079
Attention Output Projection Importance Score for Key-Value Eviction
Yuan, J., He, Z., Lin, Z., & Jiang, B. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463309
Wang, Y., Wang, Y., Chen, C., Xue, C. Y., Weng, Q., Chen, Y., Li, Z., Zhu, X., Yang, Y., Chen, Q., et al. (2026) · European Conference on Computer Systems · doi:10.1145/3767295.3803623 · Cited by 1
Multi-task generative dialogue summarization learning framework with topic-based data augmentation
Shan, J., Cao, M., & Wang, J. (2026) · Computer Speech & Language · doi:10.1016/j.csl.2026.101998
LLM-driven Dark Web Intelligence Analysis Pipeline
Jiang, L., Wang, M., Sun, Y., Wang, J., Zhang, H., & Duan, H. (2026) · International Conference on Generative Artificial Intelligence and Information Security (GAIIS) · doi:10.1109/gaiis69281.2026.11519123
Kim, H.-G., Jun, T., & Lee, T. (2026) · Mathematics · doi:10.3390/math14101769
AI-Driven Multi-agent Sustainable Shopping Companion: ECO Wise
Denge, O., Yashwardhan, W., Jadhav, T., Shinde, S., & Kulkarni, S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-27160-0_18
Synonym Knowledge Graph Enhanced Language Model for Inconsistent Hallucination Detection
Yang, J., Zheng, H., Wang, H., & Kim, H. (2026) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3819585
“In recent years, with the emergence and development of large language models ( LLMs ) [41, 51, 63], there has been a steady increase in interest towards leveraging LLMs for the purpose of generating text [61, 66] in practical applications.”
Rubei, R., & Ruscio, D. D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6864772
Chawla, R., Dahiya, S., Singh, G., & Fortino, G. (2026) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2026.3691181 · Cited by 1
“Analyzing such multidocument inputs requires aligning and integrating fragmented or overlapping content, a considerably more complex task than handling isolated texts [13].”
Hsu, F.-Y., Liu, S.-C., Mei, K.-Q., & Chang, Y.-J. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3813044
“Capability studies show not only model size but also instruction tuning drives news summarization quality [41, 55](e.g., zero-shot vs few-shot) [8, 51], with 4.1.1 Three Strategies for Organizing and Synthesizing News-event Digests.”
InfluentialESPOSITO, N., Tricarico, A., Porzio, L., Ardebili, A. A., & Stella, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6955453
Synthesis‐GPT: An Intelligent Assistant for Solid‐State Electrolyte Synthesis
Li, B. R., Li, P. H., Zhong, Y., Yan, Z., Yu, Z., Ning, Z., Ouyang, C., Chen, X., Jin, H., Yang, L., et al. (2026) · Small Structures · doi:10.1002/sstr.70448 · Cited by 1
“Those models lack robust domain grounding and are unreliable for specialized reasoning without expert verification [24, 27, 31].”
Modeling polarization in public opinion through LLM-synthesized arguments and stance trees
Diaz, G. A., Chesñevar, C. I., Patel, R., & Maguitman, A. G. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3906 · Cited by 1
Zanatti, M., Melo, R., Won, M., Ribeiro, R., Pinto, H. S., Borbinha, J. L., Dias, J., & Santos, P. A. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200688
Tuan, C. V., Trang, M. X., Nguyen, T.-C., Ngo, V.-D., & Luong, T. V. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12249-y
“Building upon established LLM summarization techniques [38] and self-improvement concepts [39 ], our innovation introduces an iterative self-refinement loop specifically designed for multi-document sum - marization.”
Ekström, A., Stam, H. R., Neto, F. G. D. O., Gay, G., & Edenlund, S. (2026) · doi:10.1145/3793654.3793748
VideoEvent: Hierarchical and adaptive event modelling for complex video understanding
Zhang, F., Sun, X., Zhao, J., & Shang, Y. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116502
A Fact-Aware Cascaded Framework for Dynamic-Granularity Timeline Summarization
Qin, G., Jia, Y., Zhang, Y., Shen, Y., & Liu, J. T. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-8525-0_19
I/O-Aware PIM Acceleration for Long-Sequence LLM Inference with Hybrid Sparse Attention
Lu, X., Hu, L., Huang, H., Jiang, P., & Sun, X. (2026) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps65963.2026.00046 · Cited by 1
“…significantly advanced the field of artificial intelligence, demonstrating unprecedented performance across diverse applications, including multi-turn dialogue [1], [8], extensive document analysis [16], [56], [59], multi-modal reasoning [13], [28], and automated code generation [10], [30], [38].”
Gao, P., Qin, X., & Zhang, Z. L. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08674-4
“In the news domain, benchmarking studies show that strong language models may still produce factually imperfect summaries [2].”
Tang, W., Gui, T., Ma, H., & Chen, M. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11581585
MQRank: Multi-quality Ranking-Aware Distillation for Abstractive Summarization
Wang, X., Wei, L., Zhang, Z., & Yin, R. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2480-7_6
Promikyridis, R., Patsouras, C. F., Zeginis, D., Tambouris, E., & Tarabanis, K. A. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-24807-7_22 · Cited by 1
SmartSearch: Process Reward-Guided Query Refinement for Search Agents
Wen, T., Dong, G., & Dou, Z. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809700 · Cited by 18
Large model-assisted video summarization via global entity unification and robust importance scoring
Chen, D., Huang, S., Xu, X., Nie, Y., Li, P., & Chen, C. E. (2026) · The Visual Computer · doi:10.1007/s00371-026-04592-x
“Earlier studies find that LLM-based summaries outperform traditional methods in both accuracy and semantic coherence [44, 50].”
Metadata
- Authors
- Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., & Hashimoto, T. B.
- Affiliations
- Stanford University (US)
- Columbia University (US)
- Corresponding author
- Zhang, Tianyi
- Ladhak, Faisal
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 39-57
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391494845
- Cited by
- 415
- References
- 79
- Citations by year
- 2026: 108 · 2025: 165 · 2024: 96
- FWCI
- 54.46
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Open Philanthropy Project
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, news summarization, automatic summarization, human evaluation, instruction tuning, zero-shot summarization, fine-tuning, model size, human-written summaries, paraphrase, freelance writers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.