The DOI matches its DataCite record; 60 of the 60 references it cites were found in a registry or an index.
Unified Language Model Pre-training for Natural Language Understanding and Generation
Open accessAbstract
This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks. The model is pre-trained using three types of language modeling tasks: unidirectional, bidirectional, and sequence-to-sequence prediction. The unified modeling is achieved by employing a shared Transformer network and utilizing specific self-attention masks to control what context the prediction conditions on. UniLM compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks. Moreover, UniLM achieves new state-of-the-art results on five natural language generation datasets, including improving the CNN/DailyMail abstractive summarization ROUGE-L to 40.51 (2.04 absolute improvement), the Gigaword abstractive summarization ROUGE-L to 35.75 (0.86 absolute improvement), the CoQA generative question answering F1 score to 82.5 (37.1 absolute improvement), the SQuAD question generation BLEU-4 to 22.12 (3.75 absolute improvement), and the DSTC7 document-grounded dialog response generation NIST-4 to 2.67 (human performance is 2.65). The code and pre-trained models are available at https://github.com/microsoft/unilm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“A difference is that UniLM predictions are conditionally independent, whereas BART's are autoregressive.”
From BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2020) · cites this paper as Dong et al., 2019
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This integration leverages the linguistic capabilities of Llama 3, utilizing both its textual output and intermediate embeddings at the token rate (Devlin et al., 2018; Dong et al., 2019; Raffel et al., 2020; Guo et al., 2023) to enhance the prediction of prosody features, thus reducing the lookahead required by the model.”
From The Llama 3 Herd of Models (2024) · cites this paper as Dong et al., 2019
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This approach has recently been used to obtain state-of-the-art results in many of the most common NLP benchmarks (Devlin et al., 2018; Yang et al., 2019; Dong et al., 2019; Liu et al., 2019c; Lan et al., 2019).”
From Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2019) · cites this paper as Dong et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
(2004) · Cited by 9,718
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208
Research works printed but not linked: 3
- [2][2] Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, and Danilo Giampiccolo. The second PASCAL recognising textual entailment challenge. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment , 01 2006. -
- [14][14] Michel Galley, Chris Brockett, Xiang Gao, Jianfeng Gao, and Bill Dolan. Grounded response generation task at dstc7. In AAAI Dialog System Technology Challenges Workshop , 2019. -
- [22][22] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations , San Diego, CA, 2015. -
Cited by
Sorted by citations
Items 401–500 of 1,008
Magassouba, A., Sugiura, K., & Kawai, H. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3092686 · Cited by 24
Neural Language Generation: Formulation, Methods, and EvaluationPreprint
Gârbacea, C., & Mei, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.15780 · Cited by 45
VideoMAC: Video Masked Autoencoders Meet ConvNets
Pei, G., Chen, T., Jiang, X., Liu, H., Sun, Z., & Yao, Y. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02145 · Cited by 23
“The success of masked language modeling in NLP [3, 10, 12, 13, 47] has catalyzed a rising interest in masked image modeling (MIM) for visual pre-training.”
BackgroundDiaformer: Automatic Diagnosis via Symptoms Sequence Generation
Chen, J., Li, D., Chen, Q., Zhou, W., & Liu, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20365 · Cited by 23
Self-citationText Summarization using NLP Technique
Balaji, N., Kumari, D., Bhavatarini, N., Megha, N., Sunil Kumar, P., & Shikah Rai, A. (2022) · IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER) · doi:10.1109/discover55800.2022.9974823 · Cited by 24
IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languages
Kumar, A., Shrotriya, H., Sahu, P., Mishra, A., Dabre, R., Puduppully, R., Kunchukuttan, A., Khapra, M. M., & Kumar, P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.360 · Cited by 47
Maity, S., Deroy, A., & Sarkar, S. (2023) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3632754.3632755 · Cited by 23
Exploring the Capability of ChatGPT in Test Generation
Yi, G., Chen, Z., Chen, Z., Wong, W. E., & Chau, N. (2023) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c60940.2023.00013 · Cited by 27
SEAL: Segment-wise Extractive-Abstractive Long-form Text SummarizationPreprint
Zhao, Y., Saleh, M., & Liu, P. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10213 · Cited by 27
Multilingual Speech Translation with Efficient Finetuning of Pretrained ModelsPreprint
Li, X., Wang, C., Tang, Y., Tran, C., Tang, Y., Pino, J., Baevski, A., Conneau, A., & Auli, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12829 · Cited by 60
Explaining Documents' Relevance to Search QueriesPreprint
Rahimi, R., Kim, Y., Zamani, H., & Allan, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.01314 · Cited by 37
Proceedings of the 2nd Workshop on New Frontiers in Summarization
(2019) · doi:10.18653/v1/d19-54 · Cited by 22
Proof Artifact Co-training for Theorem Proving with Language Models
Han, J. M., Rute, J., Wu, Y., Ayers, E. W., & Polu, S. (2021) · arXiv · doi:10.48550/arxiv.2102.06203 · Cited by 21
A Survey on Neural Data-to-Text Generation
Lin, Y., Ruan, T., Liu, J., & Wang, H. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3304385 · Cited by 25
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions
Qin, L., Pan, W., Chen, Q., Liao, L., Zhou, Y., Zhang, Y., Che, W., & Li, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.363 · Cited by 39
Jung, H. S., Lee, H., Woo, Y. S., Baek, S. Y., & Kim, J. H. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304680 · Cited by 21
Can large language models meet the challenge of generating school-level questions?
Maity, S., Deroy, A., & Sarkar, S. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100370 · Cited by 26
Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance
Su, D., Yu, T., & Fung, P. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.275 · Cited by 26
Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems
Mi, F., Zhou, W., Kong, L., Cai, F., Huang, M., & Faltings, B. V. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.142 · Cited by 29
End-to-end neural event coreference resolution
Lu, Y., Lin, H., Tang, J., Han, X., & Sun, L. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103632 · Cited by 31
Self-Training Vision Language BERTs With a Unified Conditional Model
Yang, X., Lv, F., Liu, F., & Lin, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3235704 · Cited by 21
Hu, J. S., Zeng, W., Jia, N.-X., Arif, M., Yu, D., & Zhang, G. (2023) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c00943 · Cited by 22
From statistical methods to deep learning, automatic keyphrase prediction: A survey
Xie, B., Song, J., Shao, L., Wu, S., Wei, X., Yang, B., Lin, H., Xie, J., & Su, J. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103382 · Cited by 24
Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation
Tang, C., Zhang, H., Loakman, T., Lin, C., & Guérin, F. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.253 · Cited by 22
A Convolutional-Transformer-Based Approach for Dynamic Gesture Recognition of Data Gloves
Tang, Y., Pan, M., Li, H., & Cao, X. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3400361 · Cited by 24
Analyzing and Mitigating Interference in Neural Architecture SearchPreprint
Xu, J., Tan, X., Song, K., Luo, R., Leng, Y., Qin, T., Liu, T., & Li, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12821 · Cited by 22
Generative AI: Overview, Economic Impact, and Applications in Asset ManagementPreprint
Luk, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4574814 · Cited by 24
An Enhanced Knowledge Injection Model for Commonsense Generation
Fan, Z., Gong, Y., Wei, Z., Wang, S., Huang, Y., Jiao, J., Huang, X., Duan, N., & Zhang, R. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.182 · Cited by 24
Reinforcing Pretrained Models for Generating Attractive Text Advertisements
Wang, X., Gu, X., Cao, J., Zhao, Z., Yan, Y., Middha, B., & Xie, X. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467105 · Cited by 20
BDKG at MEDIQA 2021: System Report for the Radiology Report Summarization Task
Dai, S., Wang, Q., Lyu, Y., & Zhu, Y. T. (2021) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2021.bionlp-1.11 · Cited by 21
Fang, J., Jiang, H., Zhang, S.-Q., Sun, L., Hu, X.-D., Liu, J., Gong, M., Liu, H.-X., & Fu, Y.-Y. (2023) · Frontiers in Plant Science · doi:10.3389/fpls.2023.1123410 · Cited by 19
Sequence-to-Sequence Learning for Indonesian Automatic Question Generator
Muis, F. J., & Purwarianti, A. (2020) · International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA) · doi:10.1109/icaicta49861.2020.9429032 · Cited by 25
Maria: A Visual Experience Powered Conversational Agent
Liang, Z., Hu, H., Xu, C., Tao, C., Geng, X., Chen, Y., Liang, F., & Jiang, D. (2021) · doi:10.18653/v1/2021.acl-long.435 · Cited by 23
Visualizing and Understanding the Effectiveness of BERTPreprint
Hao, Y., Dong, L., Wei, F., & Xu, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05620 · Cited by 20
Self-citationWhat is this Article about? Extreme Summarization with Topic-aware Convolutional Neural Networks
Narayan, S., Cohen, S. B., & Lapata, M. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11315 · Cited by 19
DMPP: Differentiable multi-pruner and predictor for neural network pruning
Li, J., Zhao, B., & Liu, D. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.12.020 · Cited by 19
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
Kasai, J., Sakaguchi, K., Bras, R. J. L., Dunagan, L., Morrison, J., Fabbri, A. R., Choi, Y., & Smith, N. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.259 · Cited by 21
A High-Precision Two-Stage Legal Judgment Summarization
Huang, Y., Sun, L., Han, C., & Guo, J. B. (2023) · Mathematics · doi:10.3390/math11061320 · Cited by 21
Wang, J., Huang, C., Yan, S., Xie, W., & He, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2352919 · Cited by 22
VLAB: Enhancing Video Language Pretraining by Feature Adapting and Blending
He, X., Chen, S., Ma, F., Huang, Z., Jin, X., Liu, Z., Fu, D., Yang, Y., Liu, J., & Feng, J. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521729 · Cited by 30
Learning Semantic Concepts and Temporal Alignment for Narrated Video Procedural Captioning
Shi, B., Ji, L., Niu, Z., Duan, N., Zhou, M., & Chen, X. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413498 · Cited by 19
Self-citationZero-Shot Paraphrase Generation with Multilingual Language ModelsPreprint
Guo, Y., Liao, Y., Jiang, X., Zhang, Q., Zhang, Y., & Liu, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03597 · Cited by 24
Li, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04195 · Cited by 25
Data Augmentation for Spoken Language Understanding via Pretrained ModelsPreprint
Peng, B., Zhu, C., Zeng, M., & Gao, J. (2020) · arXiv (Cornell University) · Cited by 18
Self-citationRobust Conversational AI with Grounded Text GenerationPreprint
Gao, J., Peng, B., Li, C., Li, J., Shayandeh, S., Lidén, L., & Shum, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.03457 · Cited by 28
Self-citationBilateral Adaptation for Human-Object Interaction Detection with Occlusion-Robustness
Wang, G., Guo, Y., Xu, Z., & Kankanhalli, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02642 · Cited by 24
Mixed-Lingual Pre-training for Cross-lingual Summarization
Xu, R., Zhu, C., Shi, Y., Zeng, M., & Huang, X. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.53 · Cited by 24
Fact-level Extractive Summarization with Hierarchical Graph Mask on BERT
Yuan, R., Wang, Z., & Li, W. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.493 · Cited by 18
ENPAR:Enhancing Entity and Entity Pair Representations for Joint Entity Relation Extraction
Wang, Y., Sun, C., Wu, Y., Zhou, H., Li, L., & Yan, J. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.251 · Cited by 18
Self-citationFew-Shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems
Kulshreshtha, D., Shayan, M., Belfer, R., Reddy, S., Serban, I. V., & Kochmar, E. (2022) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia220062 · Cited by 18
Self-Supervised learning for Conversational Recommendation
Li, S., Xie, R., Zhu, Y., Zhuang, F., Tang, Z., Zhao, W. X., & He, Q. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103067 · Cited by 19
Sparsing and Smoothing for the seq2seq Models
Zhao, S., Liang, Z., Wen, J., & Chen, J. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3207982 · Cited by 20
Cross-View Language Modeling: Towards Unified Cross-Lingual Cross-Modal Pre-training
Zeng, Y., Zhou, W., Luo, A., Cheng, Z., & Zhang, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.315 · Cited by 34
ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples
Zhao, Y., Nan, L., Qi, Z., Zhang, R., & Radev, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.615 · Cited by 31
EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generation
Ge, T., Chen, S., & Wei, F. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.741 · Cited by 21
Self-citationMo, C., Hu, K., Long, C., & Wang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01335 · Cited by 24
Ask to Learn: A Study on Curiosity-driven Question Generation
Scialom, T., & Staiano, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.202 · Cited by 24
MetaGen: An academic Meta-review Generation system
Bhatia, C., Pradhan, T., & Pal, S. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401190 · Cited by 20
Autoregressive Knowledge Distillation through Imitation Learning
Lin, A., Wohlwend, J., Chen, H., & Lei, T. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.494 · Cited by 70
UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction
Wu, H., Liu, W., Li, L., Nie, D., Chen, T., Zhang, F., & Wang, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.73 · Cited by 18
A Knowledge-Enhanced Pretraining Model for Commonsense Story GenerationPreprint
Guan, J., Huang, F., Zhao, Z., Zhu, X., & Huang, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05139 · Cited by 18
Focus your attention: A focal attention for multimodal learning
Liu, C., Mao, Z., Zhang, T., Liu, A.-A., Wang, B., & Zhang, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3046855 · Cited by 17
Bidirectional Language Modeling: A Systematic Literature Review
Jahan, M. S., Khan, H. U., Akbar, S., Farooq, M. U., Gul, S., & Amjad, A. (2021) · Scientific Programming · doi:10.1155/2021/6641832 · Cited by 18
DialSummEval: Revisiting Summarization Evaluation for Dialogues
Gao, M., & Wan, X. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.418 · Cited by 20
Wen, C., & Zhu, L. (2022) · IEEE Access · doi:10.1109/access.2022.3220878 · Cited by 17
ReqGen: Keywords-Driven Software Requirements Generation
Zhao, Z., Zhang, L., Lian, X., Gao, X., Lv, H., & Shi, L. (2023) · Mathematics · doi:10.3390/math11020332 · Cited by 17
Li, F., Zhao, S., Qing, Y., Chen, X., Guan, X., Wang, S., Zhang, G., & Cui, H. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3247883 · Cited by 20
Saeed, A., Khan, H. U., Shankar, A., Imran, T., Khan, D., Kamran, M., & Khan, M. A. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3626318 · Cited by 17
Relevance-Promoting Language Model for Short-Text Conversation
Li, X., Li, P., Bi, W., Liu, X., & Lam, W. P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6340 · Cited by 16
Self-citationRemember the Facts? Investigating Answer-Aware Neural Question Generation for Text Comprehension
Steuer, T., Filighera, A., & Rensing, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-52237-7_41 · Cited by 17
Injecting Entity Types into Entity-Guided Text Generation
Dong, X., Yu, W., Zhu, C., & Jiang, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.56 · Cited by 21
Cross-Thought for Sentence Encoder Pre-training
Wang, S., Fang, Y., Sun, S., Gan, Z., Cheng, Y., Liu, J., & Jiang, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.30 · Cited by 18
Play the Shannon Game with Language Models: A Human-Free Approach to Summary Evaluation
Egan, N., Vasilyev, O., & Bohannon, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21304 · Cited by 22
Xu, S., Li, H., Yuan, P., Wang, Y., Wu, Y., He, X., Liu, Y., & Zhou, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.1 · Cited by 19
Self-citationDiscovering drug–target interaction knowledge from biomedical literature
Hou, Y., Xia, Y., Wu, L., Xie, S., Yang, F., Zhu, J., Qin, T., & Liu, T. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac648 · Cited by 20
SEASum: Syntax-Enriched Abstractive Summarization
Liu, S., Yang, L., & Cai, X. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116819 · Cited by 17
Multi-modal Summarization for Video-containing DocumentsPreprint
Fu, X., Wang, J., & Yang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.08018 · Cited by 21
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter PretrainingPreprint
Lin, J., An, Y., Bai, J., Zhou, C., Jiang, L., Jia, X., Wang, A., Zhang, J., Li, Y., Lin, W., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03888 · Cited by 23
Alhusban, M. I., Khatatbeh, I. N., & Alshurafat, H. (2025) · Competitiveness Review An International Business Journal incorporating Journal of Global Competitiveness · doi:10.1108/cr-06-2024-0121 · Cited by 24
How well do you know your summarization datasets?
Tejaswin, P., Naik, D., & Liu, P. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.303 · Cited by 25
Cai, P.-X., Fan, Y.-C., & Leu, F. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-90072-4_23 · Cited by 16
An Interactive Neural Network Approach to Keyphrase Extraction in Talent Recruitment
Yao, K.-C., Qin, C., Zhu, H.-S., Ma, C., Zhang, J., Du, Y., & Xiong, H. (2021) · International Conference on Information and Knowledge Management · doi:10.1145/3459637.3482319 · Cited by 16
GATSum: Graph-Based Topic-Aware Abstract Text Summarization
Jiang, M., Zou, Y., Xu, J., & Zhang, M. (2022) · Information Technology And Control · doi:10.5755/j01.itc.51.2.30796 · Cited by 17
PERCEIVER-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention
Tang, Z., Cho, J., Lei, J., & Bansal, M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00439 · Cited by 17
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
Zhao, Z., Li, Y., Hou, C., Zhao, J., Tian, R., Liu, W., Chen, Y., Sun, N., Liu, H., Mao, W., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.20 · Cited by 16
Improving Factual Consistency in Summarization with Compression-Based Post-Editing
Fabbri, A. R., Choubey, P. K., Vig, J., Wu, C.-S., & Xiong, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.623 · Cited by 21
Dong, Y., Wieting, J., & Verga, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.76 · Cited by 16
Structure-Unified M-Tree Coding Solver for Math Word Problem
Wang, B., Ju, J., Yang, F., Dai, X., Huang, S., & Chen, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.556 · Cited by 17
Exploring Scope Detection for Aspect-Based Sentiment Analysis
Jiang, X., You, P., Chen, C., Wang, Z., & Zhou, G. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3323136 · Cited by 16
Unifying Structured Data as Graph for Data-to-Text Pre-Training
Li, S., Li, L., Geng, R., Yang, M., Li, B., Yuan, G., He, W., Yuan, S. F., Ma, C., Huang, F., et al. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00641 · Cited by 18
Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
Gao, Y., Wu, C.-S., Joty, S., Xiong, C., Socher, R., King, I., Lyu, M. R.-T., & Hoi, S. C. H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.88 · Cited by 18
TeaForN: Teacher-Forcing with N-grams
Goodman, S., Ding, N., & Soricut, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.702 · Cited by 18
Learning to Fuse Sentences with Transformers for Summarization
Lebanoff, L., Dernoncourt, F., Kim, D. S., Wang, L., Chang, W. H., & Liu, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.338 · Cited by 15
Towards Zero-Shot Conditional Summarization with Adaptive Multi-Task Fine-Tuning
Goodwin, T. R., Savery, M. E., & Demner‐Fushman, D. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.289 · Cited by 20
Inference Time Style Control for Summarization
Cao, S., & Wang, L. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.476 · Cited by 17
Neural Machine Translation for Harmonized System Codes prediction
Chen, X., Bromuri, S., & van Eekelen, M. (2021) · International Conference on Machine Learning Technologies (ICMLT) · doi:10.1145/3468891.3468915 · Cited by 23
Discriminative Adversarial Search for Abstractive SummarizationPreprint
Scialom, T., Dray, P.-A., Lamprier, S., Piwowarski, B., & Staiano, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10375 · Cited by 18
Better Fine-Tuning with Extracted Important Sentences for Abstractive Summarization
Sahu, A., & Sanjeevi, S. G. (2021) · International Conference on Communication, Control and Information Sciences (ICCISc) · doi:10.1109/iccisc52257.2021.9484944 · Cited by 15
Metadata
- Authors
- Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W.
- Affiliations
- Microsoft (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1905.03197
- OpenAlex
- W2945260553
- Cited by
- 1,225
- References
- 48
- Citations by year
- 2026: 1 · 2025: 61 · 2024: 146 · 2023: 253 · 2022: 206 · 2021: 289 · 2020: 240 · 2019: 28
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- TIB Data Manager
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- pre-training, natural language understanding, natural language generation, sequence-to-sequence prediction, abstractive summarization, question answering, question generation, GLUE benchmark, SQuAD
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
