The DOI matches its DataCite record; 60 of the 60 references it cites were found in a registry or an index.
Unified Language Model Pre-training for Natural Language Understanding and Generation
Open accessAbstract
This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirectional, bidirectional, and sequence-to-sequence prediction. The unified modeling is achieved by employing a shared Transformer network and utilizing specific self-attention masks to control what context the prediction conditions on. UniLM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks. Moreover, UniLM achieves new state-of-the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement), the Gigaword abstractive summarization ROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question answering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question generation BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7 document-grounded dialog response generation NIST-4 to 2.67 (human performance is 2.65). The code and pre-trained models are available at https://github.com/microsoft/unilm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“A difference is that UniLM predictions are conditionally independent, whereas BART's are autoregressive.”
From BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2020) · cites this paper as Dong et al., 2019
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This integration leverages the linguistic capabilities of Llama 3, utilizing both its textual output and intermediate embeddings at the token rate (Devlin et al., 2018; Dong et al., 2019; Raffel et al., 2020; Guo et al., 2023) to enhance the prediction of prosody features, thus reducing the lookahead required by the model.”
From The Llama 3 Herd of Models (2024) · cites this paper as Dong et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This approach has recently been used to obtain state-of-the-art results in many of the most common NLP benchmarks (Devlin et al., 2018; Yang et al., 2019; Dong et al., 2019; Liu et al., 2019c; Lan et al., 2019).”
From Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019) · cites this paper as Dong et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
(2004) · Cited by 9,718
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Research works printed but not linked: 3
- [2][2] Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, and Danilo Giampiccolo. The second PASCAL recognising textual entailment challenge. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment , 01 2006. -
- [14][14] Michel Galley, Chris Brockett, Xiang Gao, Jianfeng Gao, and Bill Dolan. Grounded response generation task at dstc7. In AAAI Dialog System Technology Challenges Workshop , 2019. -
- [22][22] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations , San Diego, CA, 2015. -
Cited by
Sorted by citations
Items 901–1,000 of 1,008
SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion
Chen, J., Cao, C., & Jiang, X. (2020) · Language Resources and Evaluation · Cited by 3
A Discussion on Various Methods in Automatic Abstractive Text Summarization
Karnik, M. P., & Kodavade, D. V. (2020) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-49795-8_51 · Cited by 3
Theano: A Greek-speaking conversational agent for COVID-19
Ventoura, N., Palios, K., Vasilakis, Y., Paraskevopoulos, G., Katsamanis, N., & Katsouros, V. (2021) · Workshop on NLP for Positive Impact (NLP4PI) · doi:10.18653/v1/2021.nlp4posimpact-1.5 · Cited by 3
Enhancing Question Generation with Commonsense Knowledge
Jia, X., Wang, H. E., Yin, D., & Wu, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-84186-7_10 · Cited by 6
Extractive-Abstractive Summarization of Judgment Documents Using Multiple Attention Networks
Gao, Y., Liu, Z., Li, J., Guo, F., & Xiao, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89391-0_28 · Cited by 3
Generating Consistent and Diverse QA pairs from Contexts with BN Conditional VAE
Li, J., Qi, P., & Luo, H. (2022) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd54268.2022.9776055 · Cited by 3
Lan, Z., Li, W., Su, J., Xiao, X., Liu, J., Wu, W., & Lyu, Y. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14267 · Cited by 3
Conversational Question Generation in Russian
Makhnytkina, O. V., Matveev, A., Svischev, A., Korobova, P., Zubok, D. A., Mamaev, N., & Tchirkovskii, A. (2020) · Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct49677.2020.9211056 · Cited by 3
Yan, R., & Wu, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i17.17771 · Cited by 4
Chinese Neural Question Generation: Augmenting Knowledge into Multiple Neural Encoders
Liu, M., & Zhang, J. (2022) · Applied Sciences · doi:10.3390/app12031032 · Cited by 6
SAS: Self-Augmentation Strategy for Language Model Pre-training
Xu, Y., Zhang, J., Ru, H., Ge, L., Yang, C., Yang, C. H., & Wu, Y. N. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21412 · Cited by 6
Sequence-to-Sequence Models for Extracting Information from Registration and Legal Documents
Pires, R., de Souza, F. C., Rosa, G. J. M., Lotufo, R., & Nogueira, R. F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06555-2_6 · Cited by 4
Kumar, A., Ghosal, T., Bhattacharjee, S., & Ekbal, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16802-4_17 · Cited by 3
Learning Enhancement Using Question-Answer Generation for e-Book Using Contrastive Fine-Tuned T5
Kumar, S., Chauhan, A., & Kumar, C. P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-24094-2_5 · Cited by 3
Towards Diverse, Relevant and Coherent Open-Domain Dialogue Generation via Hybrid Latent Variables
Sun, B., Li, Y., Mi, F., Wang, W., Li, Y., & Li, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26594 · Cited by 3
Self-citationZhen, Z., Lin, H., Gu, D., Wang, L., Hong, W., & Fang, Y. (2023) · International Journal on Semantic Web and Information Systems · doi:10.4018/ijswis.329965 · Cited by 3
Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine ReadingPreprint
Gao, Y., Wu, C.-S., Joty, S., Xiong, C., Socher, R., King, I., Lyu, M. R.-T., & Hoi, S. C. H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.12484 · Cited by 3
Improving Self-supervised Pre-training via a Fully-Explored Masked Language ModelPreprint
Zheng, M., Shen, D., Shen, Y., Chen, W., & Xiao, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06040 · Cited by 4
Uncertainty-Aware Abstractive Summarization.Preprint
Gidiotis, A., & Tsoumakas, G. (2021) · arXiv (Cornell University) · Cited by 3
Shinoda, K., Sugawara, S., & Aizawa, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03238 · Cited by 3
SAS: Self-Augmented Strategy for Language Model Pre-trainingPreprint
Xu, Y., Zhang, J., He, R., Ge, L., Yang, C., Yang, C. H., & Wu, Y. N. (2021) · arXiv (Cornell University) · Cited by 3
Enhancing Question Generation with Commonsense KnowledgePreprint
Jia, X., Wang, H. E., Yin, D., & Wu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.10454 · Cited by 3
KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense ReasoningPreprint
Li, H., Gong, Y., Jiao, J., Zhang, R.-F., Baldwin, T., & Duan, N. (2021) · arXiv.org · doi:10.48550/arxiv.2109.06704 · Cited by 6
Abstractive Text Summarization via Stacked LSTM
Ireddy, S., Zhan, H., & Sheng, V. S. (2021) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci54926.2021.00143 · Cited by 3
Ni, J. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/168430 · Cited by 3
LexSUS: A Hybrid Lexical-Graph Salience based Text Summarization Technique using PEGASUSPreprint
Ansar, W., Goswami, S., & Chakrabarti, A. (2023) · doi:10.36227/techrxiv.24265702 · Cited by 3
Research on Clarification Question Recognition and Generation in Intelligent Q&A
Hou, J., Du, W., Mu, Q., Zhang, K., & He, Z. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8764-1_14 · Cited by 2
Research and Design of Automatic Questioning System Based on Question Generation
Shi, D., Zhao, S., Liu, Q., & Zhang, Y. (2024) · International Conference on Information and Education Technology (ICIET) · doi:10.1109/iciet60671.2024.10542827 · Cited by 2
Fact-Aware Generative Text Summarization with Dependency Graphs
Chen, R., Li, Y., Jiang, Y., Sun, B., Wang, J., & Li, Z. (2024) · Electronics · doi:10.3390/electronics13163230 · Cited by 2
Lightweight Contextual LLMs for IoT Data Interpretation in Smart Cities
Kannadasan, T. (2025) · International Conference on Visual Analytics and Data Visualization (ICVADV) · doi:10.1109/icvadv63329.2025.10961648 · Cited by 2
A Multi-Layer Attention Knowledge Tracking Method with Self-Supervised Noise Tolerance
Wang, H., Liu, H., Ge, Y., & Yu, Z. (2025) · Applied Sciences · doi:10.3390/app15158717 · Cited by 2
BERT and Its Applications in Natural Language Understanding
Zhu, Z. (2025) · Applied and Computational Engineering · doi:10.54254/2755-2721/2025.ast26090 · Cited by 5
Ghosh, S., Jain, N., & Sunoj, R. B. (2025) · ACS Catalysis · doi:10.1021/acscatal.5c07050 · Cited by 7
Durak, H. Y., Eğin, F., & Onan, A. (2025) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2025.101944 · Cited by 4
AutoEQA: Auto-Encoding Questions for Extractive Question Answering
Varanasi, S., Amin, S., & Neumann, G. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.403 · Cited by 3
A Quantum Language-Inspired Tree Structural Text Representation for Semantic Analysis
Yan, Y., Qiu, D., & Yan, R. (2022) · Mathematics · doi:10.3390/math10060914 · Cited by 2
A Hierarchical Memory Model for Task-Oriented Dialogue System
Ya, Z. E. N. G., Li, W. A. N., Luo, Q., & Mao, C. H. E. N. (2022) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2022edp7001 · Cited by 2
Employing Internal and External Knowledge to Factuality-Oriented Abstractive Summarization
Gao, Z., Jiang, F., Chu, X., & Li, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17120-8_61 · Cited by 2
An Optimized Method for Large-Scale Pre-Training in Symbolic Music
Liu, S., Xu, H., & Xu, K. (2022) · IEEE International Conference on Anti-counterfeiting, Security, and Identification (ASID) · doi:10.1109/asid56930.2022.9995766 · Cited by 2
Ensemble Summarization Models to Leverage Performance
Umejiaku, A. P., Zhou, F., & Sheng, V. S. (2022) · International Conference on Computing and Big Data (ICCBD) · doi:10.1109/iccbd56965.2022.10080715 · Cited by 2
EQGTL: An Ensemble Model for Relevant Question Generation using Transfer Learning
Godslove, J. F., & Nayak, A. K. (2022) · International Conference on Machine Learning, Computer Systems and Security (MLCSS) · doi:10.1109/mlcss57186.2022.00054 · Cited by 2
Zhu, C., Zheng, X. W., & Feng, W. (2023) · IEEE International Conference on Power, Electronics and Computer Applications (ICPECA) · doi:10.1109/icpeca56706.2023.10076029 · Cited by 2
Research on Classification Method of Bank Statement in Low Resource and Cross-domain Scenarios
Wang, C., Hu, Z., Chen, W., Liu, Y., Wang, L., & Jiang, A. (2022) · Euro-Asia Conference on Frontiers of Computer Science and Information Technology (FCSIT) · doi:10.1109/fcsit57414.2022.00015 · Cited by 3
Baykara, B., & Güngör, T. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35320-8_14 · Cited by 2
HSM-QA: Question Answering System Based on Hierarchical Semantic Matching
Zhang, J., He, J., Zhou, Y., Sun, X., & Yu, X. (2023) · IEEE Access · doi:10.1109/access.2023.3296850 · Cited by 2
A Simple Semantics and Topic-aware Method to Enhance Abstractive Summarization
Du, J., Fu, X., Li, J., Hou, C., Zhou, Q., & Zheng, H.-T. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191441 · Cited by 2
Multilingual Pre-training Model-Assisted Contrastive Learning Neural Machine Translation
Sun, S., Hou, H., Yang, Z., & Wang, Y. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191766 · Cited by 2
Self-citationDMLM: Descriptive Masked Language Modeling
Barba, E., Campolungo, N., & Roberto Navigli, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.808 · Cited by 3
Towards Zero-Shot Persona Dialogue Generation with In-Context Learning
Xu, X., Lei, Z., Wu, W., Niu, Z.-Y., Wu, H. R., & Wang, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.90 · Cited by 2
A Joint Learning Framework for Restaurant Survival Prediction and Explanation
Li, X., Zhang, X., JiaHao, P., Mao, R., Zhou, M., Xie, X., & Liao, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.216 · Cited by 2
Self-citationDeepGen: Diverse Search Ad Generation and Real-Time Customization
Golobokov, K., Chai, J., Dong, V. Y., Gu, M., Chi, B., Cao, J., Yan, Y., & Liu, Y. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.19 · Cited by 2
A Framework for Automatic Generation of Spoken Question-Answering Data
Menevşe, M. Ü., Manav, Y., Arısoy, E., & Özgür, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.342 · Cited by 2
Instance Regularization for Discriminative Language Model Pre-training
Zhang, Z., Zhao, H. Y., & Zhou, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.773 · Cited by 2
Self-citationYang, X., Jin, J., Yang, Q., Shen, X., & Chen, X. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107475 · Cited by 2
SUMOPE: Enhanced Hierarchical Summarization Model for Long Texts
Chang, C., Zhou, J., Zeng, X., & Tang, Y. Z. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46664-9_21 · Cited by 2
Abstractive Multi-document Summarization with Cross-Documents Discourse Relations
Han, M., Wang, Z., Wang, H., Bao, X., & Niu, K. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8145-8_36 · Cited by 2
A Comparative Study of Transformer Based Pretrained AI Models for Content Summarization
Rasheed, A. S. A., Masud, M. M., & Abduljabbar, M. (2023) · International Conference on Innovations in Information Technology · doi:10.1109/iit59782.2023.10366411 · Cited by 3
Integrating Extractive and Abstractive Models for Code Comment Generation
Sun, W., Hu, Y., Xu, Y., Chen, Y., & Fang, C. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00027 · Cited by 2
Grammar Error Correction Using Natural Language Processing
Geetha, M., Vijaya, K., M, S. K., Kanna, P. R., Rajeshkumar, G., & Karthika, S. (2023) · International Conference on Sustainable Communication Networks and Application (ICSCNA) · doi:10.1109/icscna58489.2023.10370283 · Cited by 3
HGNN-T5 PEGASUS: A Hybrid Approach for Chinese Long Text Summarization
Luo, X., Li, J., & Chen, Z. (2024) · Communications in computer and information science · doi:10.1007/978-981-99-9637-7_1 · Cited by 2
Liu, L., Li, X., & Yang, F. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29830 · Cited by 3
Improving Vision-and-Language Reasoning via Spatial Relations Modeling
Yang, C., Xu, R., Guo, Y., Huang, P., Chen, Y., Ding, W., Wang, Z., & Zhou, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00082 · Cited by 4
Zhang, M., You, D., & Wang, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0302104 · Cited by 2
Towards Test Time Domain Adaptation via Negative Label Smoothing
Yang, H., Zuo, H., Zhou, R. H., Wang, M., & Zhou, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128182 · Cited by 2
Hamzaoui, Bachiri, Ouassam, E., Mouncif, H., & Bouikhalene, B. (2024) · International Conference on Networking, Information Systems & Security (NISS) · doi:10.1145/3659677.3659823 · Cited by 4
PTE: Prompt tuning with ensemble verbalizers
Liang, L., Wang, G., Lin, C., & Feng, Z. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125600 · Cited by 2
UniBin: Assembly semantic-enhanced binary vulnerability detection without disassembly
Liu, L., Wang, S., & Jiang, X. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121605 · Cited by 2
Fact-Aware Abstractive Summarization Based on Prompt Information and Re-Ranking
Cai, T., Yang, Y., Wan, Y., Mao, X., & Ju, S. (2024) · International Conference on Machine Learning and Natural Language Processing (MLNLP) · doi:10.1109/mlnlp63328.2024.10800044 · Cited by 2
Wu, H., Zhang, L., Zhu, H., Liu, Q., Chen, E., & Xiong, H. (2025) · ACM Transactions on Management Information Systems · doi:10.1145/3712309 · Cited by 4
Zhao, X., Bai, Z., Zhou, M., Ren, X., Wang, Y., & Wang, L. (2024) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc61627.2024.10837308 · Cited by 2
Self-citationBibliometric Analysis of Large Language Model Artificial Intelligence Based on Knowledge Graph
Weng, X., Wang, Y., Weng, S., Xiong, J., & Weng, L. (2024) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc63484.2024.00064 · Cited by 2
Self-citationTowards Reflexive AI: A Comprehensive Exploration of Enhancing Social Science Research Through NLP
Shan, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-84460-7_49 · Cited by 4
Deep Learning-Based Automatic Summarization of Chinese Maritime Judgment Documents
Zhang, L., Li, Y., & Zhang, H. (2025) · Applied Sciences · doi:10.3390/app15105434 · Cited by 3
Corrêa, A. J. P., Silva, C., Xu, L., Brintrup, A. M., & Moniz, S. (2026) · Computers & Operations Research · doi:10.1016/j.cor.2026.107433 · Cited by 2
Strong natural language query generation
Liu, B., Lu, X., & Culpepper, J. S. (2021) · Information Retrieval · doi:10.1007/s10791-021-09395-3 · Cited by 2
Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching
Chopra, P., Rallabandi, S. K., Black, A. W., & Chandu, K. R. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.373 · Cited by 5
Which Discriminator for Cooperative Text Generation?
Chaffin, A., Scialom, T., Lamprier, S., Staiano, J., Piwowarski, B., Ewa Kijak, C., & Claveau, V. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531858 · Cited by 4
Hung, C.-C., Green, T., Litschko, R., Tsereteli, T., Takeshita, S., Bombieri, M., Glavaš, G., & Ponzetto, S. P. (2022) · doi:10.18653/v1/2022.mia-1.8 · Cited by 2
Neural Ranking with Weak Supervision for Open-Domain Question Answering : A Survey
Shen, X., Vakulenko, S., Tredici, M. D., Barlacchi, G., Byrne, B., & de Gispert, A. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.129 · Cited by 2
A Hybrid Approach for Detecting Prerequisite Relations in Multi-Modal Food Recipes
Pan, L., Chen, J., Liu, S., Ngo, C., Kan, M., & Chua, T. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3042706 · Cited by 2
Utilizing Multi-modal Emotion Information in Dialogue Strategy Classification
Jang, J. Y., Kim, J.-E., Jung, M., Jung, H., & Shin, S. (2020) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc49870.2020.9289355 · Cited by 2
Suglia, A., Bisk, Y., Konstas, I., Vergari, A., Bastianelli, E., Vanzo, A., & Lemon, O. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.183 · Cited by 4
Mao, Y., Ma, W., Lei, D., Han, J., & Ren, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.413 · Cited by 2
An Answer Driven Model For Paragraph-level Question Generation
Xu, C., & Xu, J. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534455 · Cited by 2
Transformer with Memory Replay
Liu, R., & Mozafari, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20722 · Cited by 3
Self-supervised Product Title Rewrite for Product Listing Ads
Zhao, X., Liu, D., Ding, J., Yao, L., Yan, M., Wang, H., & Yao, W. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-industry.10 · Cited by 2
Research and Application of Automatic Text Summarization Technology Based on Deep Learning
Sun, Z., Meng, X., Zheng, P., Zhu, X., & Yang, L. (2022) · 2022 11th International Conference of Information and Communication Technology (ICTech)) · doi:10.1109/ictech55460.2022.00052 · Cited by 2
Liao, J., Shi, Y., & Xu, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3219838 · Cited by 5
Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models
Wang, L., He, J., Xu, X., Liu, N., & Liu, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25357 · Cited by 3
DuNST: Dual Noisy Self Training for Semi-Supervised Controllable Text Generation
Feng, Y., Yi, X., Wang, X., Lakshmanan, V. S. L., & Xie, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.488 · Cited by 5
Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk
Li, J., Wu, X., Liu, X., Xie, Q., Tiwari, P., & Wang, B. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.419 · Cited by 4
Self-citationOn Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization
Palaskar, S., Bhagia, A., Bisk, Y., Metze, F., Black, A. W., & Marasović, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.194 · Cited by 4
CSDNet: Contrastive Similarity Distillation Network for Multi-lingual Image-Text Retrieval
Lu, S., Guo, L., He, X., Zhu, X., Liu, J., & Liu, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46311-2_32 · Cited by 2
Ren, Z., Guo, J., Xu, Y., & Xiao, B. J. (2023) · IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC) · doi:10.1109/ic-nidc59918.2023.10390776 · Cited by 2
Label-Aware Auxiliary Learning for Dialogue State Tracking
Liu, Y., Chen, L., & Yu, K. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447189 · Cited by 2
Enhancing Code Representation Learning for Code Search with Abstract Code Semantics
Zhang, S., Ding, Y., Hu, E., Yu, Y., & Zhang, Y. L. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650119 · Cited by 3
Label-Free Topic-Focused Summarization Using Query Augmentation
Mu, W., & Lim, K. H. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650321 · Cited by 2
JBNU at SemEval-2020 Task 4: BERT and UniLM for Commonsense Validation and Explanation
Na, S., & Lee, J. (2020) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2020.semeval-1.65 · Cited by 2
Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data
Yu, D., Sun, K., Yu, D., & Cardie, C. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.6 · Cited by 5
Modeling Endorsement for Multi-Document Abstractive Summarization
Lebanoff, L., Wang, B., Feng, Z., & Liu, F. (2021) · Workshop on New Frontiers in Summarization (NewSum) · doi:10.18653/v1/2021.newsum-1.13 · Cited by 2
Metadata
- Authors
- Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W.
- Affiliations
- Microsoft (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1905.03197
- OpenAlex
- W2945260553
- Cited by
- 1,225
- References
- 48
- Citations by year
- 2026: 1 · 2025: 61 · 2024: 146 · 2023: 253 · 2022: 206 · 2021: 289 · 2020: 240 · 2019: 28
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- pre-training, natural language understanding, natural language generation, sequence-to-sequence prediction, abstractive summarization, question answering, question generation, GLUE benchmark, SQuAD
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
