T5-Based Model for Abstractive Summarization: A Semi-Supervised Learning Approach with Consistency Loss Functions
Open accessAbstract
Text summarization is a prominent task in natural language processing (NLP) that condenses lengthy texts into concise summaries. Despite the success of existing supervised models, they often rely on datasets of well-constructed text pairs, which can be insufficient for languages with limited annotated data, such as Chinese. To address this issue, we propose a semi-supervised learning method for text summarization. Our method is inspired by the cycle-consistent adversarial network (CycleGAN) and considers text summarization as a style transfer task. The model is trained by using a similar procedure and loss function to those of CycleGAN and learns to transfer the style of a document to its summary and vice versa. Our method can be applied to multiple languages, but this paper focuses on its performance on Chinese documents. We trained a T5-based model and evaluated it on two datasets, CSL and LCSTS, and the results demonstrate the effectiveness of the proposed method.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent advancements in NLU prominently feature large transformer-based models like BERT and GPT-3, which have significantly advanced the field by enabling deeper and more complex understanding of language subtleties.
“Recent advancements have prominently featured large transformer-based models like BERT and GPT-3, which have significantly advanced the field by enabling a deeper and more complex understanding of language subtleties [197], [198].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [198]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · see citing works
LexRank: Graph-based Lexical Centrality as Salience in Text Summarization
Erkan, G., & Radev, D. R. (2004) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1523 · Cited by 2,975 · see citing works
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,879 · see citing works
Nallapati, R. M., Zhai, F., & Zhou, B. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10958 · Cited by 1,262 · see citing works
Abstractive Sentence Summarization with Attentive Recurrent Neural Networks
Chopra, S., Auli, M., & Rush, A. M. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1012 · Cited by 896 · see citing works
Neural Summarization by Extracting Sentences and Words
Cheng, J., & Lapata, M. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1046 · Cited by 802 · see citing works
TextRank: Bringing Order into Texts
Mihalcea, R., & Tarau, P. (2004) · University of North Texas Digital Library (University of North Texas) · Cited by 632 · see citing works
Text summarization via hidden Markov models
Conroy, J. M., & O’Leary, D. P. (2001) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/383952.384042 · Cited by 372 · see citing works
CycleGAN-VC: Non-parallel Voice Conversion Using Cycle-Consistent Adversarial Networks
Kaneko, T., & Kameoka, H. (2018) · 2018 26th European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553236 · Cited by 358 · see citing works
LCSTS: A Large Scale Chinese Short Text Summarization Dataset
Hu, B., Chen, Q., & Zhu, F. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1229 · Cited by 325 · see citing works
Assessing sentence scoring techniques for extractive text summarization
Ferreira, R., Cabral, L., Lins, R. D., Silva, G. D. F. P. E., Freitas, F., Cavalcanti, G. D. C., Lima, R. J., Simske, S. J., & Favaro, L. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.04.023 · Cited by 318 · see citing works
CycleGAN-VC2: improved CycleGAN-based non-parallel voice conversion.
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682897 · Cited by 272 · see citing works
Paired cycle‐GAN‐based image correction for quantitative cone‐beam computed tomography
Harms, J. M., Lei, Y., Wang, T., Zhang, R., Zhou, J., Tang, X., Curran, W. J., Liu, T., & Yang, X. (2019) · Medical Physics · doi:10.1002/mp.13656 · Cited by 257 · see citing works
Pretraining-Based Natural Language Generation for Text Summarization
Zhang, H., Cai, J., Xu, J., & Wang, J. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1074 · Cited by 247 · see citing works
Generating Handwritten Chinese Characters Using CycleGAN
Chang, B., Zhang, Q., Pan, S., & Meng, L. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00028 · Cited by 157 · see citing works
Effective deep learning approaches for summarization of legal texts
Anand, D., & Wagh, R. S. (2019) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2019.11.015 · Cited by 146 · see citing works
Neural Summarization by Extracting Sentences and WordsPreprint
Cheng, J., & Lapata, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07252 · Cited by 126 · see citing works
Multiple documents summarization based on evolutionary optimization algorithm
Alguliyev, R. M., Aliguliyev, R. M., & Isazade, N. R. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.09.014 · Cited by 120 · see citing works
GraphSum: Discovering correlations among multiple terms for graph-based summarization
Baralis, E., Cagliero, L., Mahoto, N. A., & Fiori, A. (2013) · Information Sciences · doi:10.1016/j.ins.2013.06.046 · Cited by 109 · see citing works
CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-Spectrogram Conversion
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2280 · Cited by 96 · see citing works
Deep reinforcement learning for extractive document summarization
Yao, K., Zhang, L., Luo, T., & Wu, Y. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.01.020 · Cited by 86 · see citing works
Abstractive Document Summarization via Neural Model with Joint Attention
Hou, L., Hu, P., & Bei, C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-73618-1_28 · Cited by 52 · see citing works
A Machine Learning Approach to Sentence Ordering for Multidocument Summarization and Its Evaluation
Bollegala, D., Okazaki, N., & Ishizuka, M. (2005) · Lecture notes in computer science · doi:10.1007/11562214_55 · Cited by 31 · see citing works
A hierarchical self-attentive neural extractive summarizer via reinforcement learning (HSASRL)
Mohsen, F., Wang, J., & Al‐Sabahi, K. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01669-5 · Cited by 31 · see citing works
Landmark Assisted CycleGAN for Cartoon Face GenerationPreprint
Wu, R., Gu, X., Tao, X., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01424 · Cited by 29 · see citing works
CSL: A Large-scale Chinese Scientific Literature DatasetPreprint
Li, Y., Zhang, Y., Zhao, Z., Shen, L., Liu, W., Mao, W., & Zhang, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.05034 · Cited by 23 · see citing works
Text-to-Image-to-Text Translation using Cycle Consistent Adversarial NetworksPreprint
Gorti, S. K., & Ma, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04538 · Cited by 23 · see citing works
LCSTS: A Large Scale Chinese Short Text Summarization DatasetPreprint
Hu, B., Chen, Q., & Zhu, F. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.05865 · Cited by 23 · see citing works
Neural Diverse Abstractive Sentence Compression Generation
Nayeem, M. T., Fuad, T. A., & Chali, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15719-7_14 · Cited by 8 · see citing works
CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram ConversionPreprint
Kaneko, T., Kameoka, H., Tanaka, K., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11672 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Ban, H. Stylistic Characteristics of English News. In Proceedings of the Japan-Korea Joint Symposium on Emotion & Sensibility, Daejeon, Republic of Korea, 4-5 June 2004.
- Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Nets. In Proceedings of the Neural Information Processing Systems, Montreal, QC, Canada, 8-13 December 2014.
- Bishop, C. Pattern Recognition and Machine Learning; Stat Sci; Springer: Berlin/Heidelberg, Germany, 2006.
- Lin, C.Y. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out; Association for Computational Linguistics: Barcelona, Spain, 2004; pp. 74-81.
- Papineni, K.; Roukos, S.; Ward, T.; Zhu, W.J. Bleu: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA, 7-12 July 2002; pp. 311-318.
Cited by
Sorted by citations
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
“…have focused on developing sophisticated loss functions and regularization techniques, aimed at enhancing the performance and generalization capabilities of supervised learning models, ensuring they remain robust and effective across a wide range of tasks and data types [166], [167], [168].”
BackgroundRobust Multi Model RAG Pipeline For Documents Containing Text, Table & Images
Joshi, P., Gupta, A. K., Kumar, P., & Sisodia, M. S. (2024) · International Conference on Applied Artificial Intelligence and Computing (ICAAIC) · doi:10.1109/icaaic60222.2024.10574972 · Cited by 47 · see citing works
Goel, I., Bhaskar, Y., Kumar, N., Singh, S., Amanullah, M., Dhar, R., & Karmakar, S. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1550407 · Cited by 31 · see citing works
A Hybrid Transformer-Based Framework for Multi-Document Summarization of Turkish Legal Documents
Albayati, M. A. A., & Fındık, O. (2025) · IEEE Access · doi:10.1109/access.2025.3545750 · Cited by 21 · see citing works
A Comparative Study of PEGASUS, BART, and T5 for Text Summarization Across Diverse Datasets
Daraghmi, E. Y., Atwe, L., & Jaber, A. (2025) · Future Internet · doi:10.3390/fi17090389 · Cited by 17 · see citing works
Abstractive text summarization: A comprehensive survey of techniques, systems, and challenges
Almohaimeed, N., & Azmi, A. M. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100762 · Cited by 17 · see citing works
Summarizing Business News: Evaluating BART, T5, and PEGASUS for Effective Information Extraction
Dharrao, D. S., Mishra, M., Kazi, A., Dharrao, M., Pise, P. D., & Bongale, A. M. (2024) · Revue d intelligence artificielle · doi:10.18280/ria.380311 · Cited by 9 · see citing works
Verma, B., Thomas, A., & Verma, R. K. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111478 · Cited by 7 · see citing works
Implementing T5 for Text Summarization: An Algorithmic Approach
Jyoti, D., Srivastava, J., & Mahato, D. P. (2025) · International Conference on Information Networking · doi:10.1109/icoin63865.2025.10992766 · Cited by 6 · see citing works
Towards efficient multi-legal document summarization: An ensemble approach for Turkish law
Albayati, M. A. A., Karaoğlan, K. M., & Fındık, O. (2025) · Engineering Science and Technology an International Journal · doi:10.1016/j.jestch.2025.102138 · Cited by 5 · see citing works
Enhanced Text Summarization through Hybrid Integration of RoBERTa, T5, and Pegasus Models
Myla, S. D., Saini, R., & Kapoor, N. (2024) · International Conference on Advances in Modern Age Technologies for Health and Engineering Science (AMATHE) · doi:10.1109/amathe61652.2024.10582247 · Cited by 5 · see citing works
Vaškevičius, M., Kapočiūtė-Dzikienė, J., & Šlepikas, L. (2023) · Applied Sciences · doi:10.3390/app132413140 · Cited by 5 · see citing works
Domain-Specific Multi-Document Political News Summarization Using BART and ACT-GAN
NIE, Z., Arshad, M., Khurshid, S. K., Javed, A., Waheed, T., Zhang, Q., & Shahzad, M. F. (2025) · IEEE Access · doi:10.1109/access.2025.3598058 · Cited by 5 · see citing works
Hindi Text Summarization: Using BERT
More, M., Yevale, P., Mandwale, A., Agrawal, K., Mahale, O., & Rajput, S. (2024) · International Conference on Intelligent Algorithms for Computational Intelligence Systems (IACIS) · doi:10.1109/iacis61494.2024.10721619 · Cited by 4 · see citing works
Venugopal, N. L. V., Visala, K., Nirmala, S., Varma, C. V., Triparagiri, A., Kumar, A., & Sekhar, C. (2024) · International Journal of experimental research and review · doi:10.52756/ijerr.2024.v46.017 · Cited by 4 · see citing works
ABSTRACTIVE SUMMARIZATION OF INDIAN LEGAL DOCUMENTS USING T5 & QLoRA
Anbarasi, M. S., Aathif Mohammed, A., Deena, R., Manimaran, V., & Mohanraj, S. (2024) · International Education and Research Journal · doi:10.21276/ierj24840819095223 · Cited by 4 · see citing works
Unified Transformer Framework for Automated Cyberbullying Detection
Alikhashashneh, E. A., Alsawan, H., Nahar, K. M. O., Shatnawi, N. M., Almomani, A., Alauthman, M., Bansal, S., & Pan, V. S.-H. (2025) · International Journal of Cloud Applications and Computing · doi:10.4018/ijcac.386166 · Cited by 3 · see citing works
T5-Based Named Entity Recognition for Social Media: A Case Study for Location Extraction
Dahlan, A. F., & Yuangyai, C. (2024) · IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) · doi:10.1109/iaict62357.2024.10617592 · Cited by 3 · see citing works
HyperSumm-RL: A Dialogue Summarization Framework for Modeling Leadership Perception in Social Robots
Das, S. (2025) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3720553.3746685 · Cited by 2 · see citing works
Sentiment-Aware Summary Generation for User Reviews Using Deep Learning Models
Danyal, M. M., Khalid, A., Khan, S. S., Ullah, S., Jan, H.-C., & Khan, D. (2025) · VFAST Transactions on Software Engineering · doi:10.21015/vtse.v13i3.2240 · Cited by 2 · see citing works
Invoice Indexing and Business Expense Management using Generative AI
Devadarshini, M., Karthikeyan, A., & Manimegalai, R. (2025) · International Conference on Next Generation Computing Systems (ICNGCS) · doi:10.1109/icngcs64900.2025.11182987 · Cited by 2 · see citing works
A Text Summarization Approach to Enhance Global and Local Information Awareness of Transformer
Liu, Z., Wang, H., Zhang, C., & Zhang, S. (2024) · IEEE Access · doi:10.1109/access.2024.3449280 · Cited by 3 · see citing works
Word Level LSTM CNN based sign language to sentence translator
Chatterjee, S., & Abinaya, S. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.04.276 · Cited by 1 · see citing works
Advancing Hindi Text Summarization: Named Entity Recognition and Content Augmentation Strategies
Gupta, S., & Pal, S. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3770073 · Cited by 1 · see citing works
Zhou, Y., Wei, J., Sun, Y., & Du, W. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00004-7 · Cited by 3 · see citing works
PFB (2021) · Pesquisa Florestal Brasileira · doi:10.4336/2021.pfb.41e202102240
Elashmawi, W. H., Osman, H., Osama, M., & Nader, N. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-77229-0_6
Mathematic Problems Categorization Using Synthetic- LLM Models
Rahravi, M., Khouri, M., & Ghasemzadeh, M. (2024) · International Conference on Signal Processing and Intelligent Systems (ICSPIS) · doi:10.1109/icspis65223.2024.10931108
EmpowerSpeak: “A Breakthrough Model for Speech Synthesis from Videos for the Differently Abled”
Yeruva, S., Bhavana, A., Sai, V. H., Shakkari, G., & Manish, T. V. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-86290-8_5
Improving Summarization Quality with Citation Networks in Scientific Documents
Nguyen, Q.-A., Doan, T.-T., Ha, Q.-T., & Tran, M.-V. (2024) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse63888.2024.11063598
Abstractive Text Summarization for Hindi Language Using T5 Transformer
Mete, A. M., & Dhore, M. L. (2025) · OPJU International Technology Conference on Smart Computing for Innovation and Advancement in Industry (OTCON) · doi:10.1109/otcon65728.2025.11070753
Text Summarization in the Age of Transformers: A Review of Techniques and Applications
Illangarathne, P., Senanayake, N., & Hewageegana, K. (2025) · International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd64986.2025.11082048
Analysis of Extractive Text Summarization Algorithms on Hindi Legal Document Corpus
Singh, K., Budhwar, P., Mazumder, B., Priya, V., & Upadhyaya, B. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-2329-7_33
Harish, S., Nawar, S., Chaitra, B. N., Amin, R., & Vijayan, A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-4008-9_44
Generating Workplace Insights From Employee Reviews Using Aspect-Based Sentiment Analysis
Ramani, J., Kelkar, T., Shah, D., Maheshwari, D., & Nanade, A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-06691-6_45
AI-Powered Keyword Extraction and Sentiment Profiling of Food Delivery Reviews using T5
Chandra, J. M., Susanto, D. D., Wendhra, Sutedjo, M. P., & Hartono, S. (2025) · International Conference on Information Management and Technology (ICIMTech) · doi:10.1109/icimtech67074.2025.11265599
TextPrep: Automating the Process of Text Data Pre-processing by Leveraging Self-supervised Learning
Anand, A., Sampurn, K., Bajaj, T., Lakshanya, A., & Bharathi, R. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-981-96-9971-1_2
Enhancing Text Summarization Techniques: Comparative Study of BART and T5 Models
Harshitha, G. M., Vasudeva, Ramyashree, Ranika, & Prabhu, R. (2025) · Control Instrumentation System Conference (CISCON) · doi:10.1109/ciscon66933.2025.11337613
Semantic Matching-Based Neural Network for Text Summarization in Knowledge Extraction
Liu, J., Tan, J., & Hua, Z. (2026) · IEEE Access · doi:10.1109/access.2026.3663013
Filbert, K., & Chowanda, A. (2025) · International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) · doi:10.1109/isriti68345.2025.11393389
Small Models, Big Picture! A Language Model Augmentation for Enhanced Reader-Aware Summarization
Kumar, R., Poornash, A. S., & Saha, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_13
Aftiss, A., Lamsiyah, S., Schommer, C., & Alaoui, S. O. E. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01119-4
Automatic text summarisation using a modular pipeline approach with LangChain
Mustafa, K., & Mahbub, K. (2026) · Applied Computing and Informatics · doi:10.1108/aci-10-2025-0437
Kumar, R., & Pathak, H. K. (2026) · International Conference on Artificial Intelligence Systems (AIS) · doi:10.1109/ais69919.2026.11621306
Bilimsel Metinlerde Anlamsal Özetleme: Derin Öğrenme Yaklaşımlarının Performans Analizi
Özcan, S., & Özcan, A. (2026) · Bilişim Teknolojileri Dergisi · doi:10.17671/gazibtd.1863483
What is Attention Mechanism? A Comprehensive Survey of Attention Methods and Transformer Models
Shiri, F. M., Memar, F., & Parhizgar, M. (2026) · Machine Learning · doi:10.1007/s10994-026-07131-w
- Authors
- Wang, M., Xie, P., Du, Y., & Hu, X.
- Affiliations
- Beihang University (CN)
- Chinese Academy of Sciences (CN)
- Corresponding author
- Wang, Mingye
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2023-06-14
- Volume
- 13
- Issue
- 12
- Pages
- 7111
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W4380998171
- Cited by
- 46
- References
- 30
- Citations by year
- 2026: 7 · 2025: 26 · 2024: 11 · 2023: 1 · 2021: 1
- FWCI
- 4.24
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- key R&D project of the Ministry of Science and Technology of the People’s Republic of China (2020-JCJQ-ZD-079-00)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- abstractive summarization, semi-supervised learning, consistency loss, CycleGAN, style transfer, T5, Chinese text summarization, limited labeled data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
