The DOI matches its DataCite record; 63 of the 72 references it cites were found in a registry or an index.
Learned in Translation: Contextualized Word Vectors
Open accessAbstract
Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typically sees initialization of only the lowest layer of deep models with pretrained word vectors. In this paper, we use a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation (MT) to contextualize word vectors. We show that adding these context vectors (CoVe) improves performance over using only unsupervised word and character vectors on a wide variety of common NLP tasks: sentiment analysis (SST, IMDb), question classification (TREC), entailment (SNLI), and question answering (SQuAD). For fine-grained sentiment analysis and entailment, CoVe improves performance of our baseline models to the state of the art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The increase of 4.7% with ELMo is also significantly larger then the 1.8% improvement from adding CoVe to a baseline model (McCann et al., 2017).”
From Deep Contextualized Word Representations (2018) · cites this paper as McCann et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For tasks where direct comparisons are possible, ELMo outperforms CoVe (McCann et al., 2017), which computes contextualized representations using a neural machine translation encoder.”
From Deep Contextualized Word Representations (2018) · cites this paper as McCann et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“[2016] propose to combine text representations with visual representations extracted by a pretrained residual network [He et al., 2016].”
BackgroundMethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“Qi et al. [2016] propose a CNN-based object tracking framework, which uses hierarchical features from a pretrained CNN (VGG-19 by Simonyan and Zisserman [2014]).”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“In computer vision, convolutional neural networks (CNNs) pretrained on ImageNet [Krizhevsky et al., 2012, Deng et al., 2009] have become the de facto initialization for more complex and deeper models.”
MethodSelf-citationGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 14×
“The MT-LSTM is used to compute a sequence of hidden states h = MT-LSTM ( GloVe ( w x )) .”
BackgroundMethodInfluentialSelf-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · cited 1×
“Girshick et al. [2014] show that using a pretrained CNN to extract features from region proposals improves object detection and semantic segmentation models.”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“…representation for all inputs. x pool = max( X | y ); mean( X | y ); min( X | y ); x self (15) We feed this joined representation through a three-layer, batch-normalized [Ioffe and Szegedy, 2015] maxout network [Goodfellow et al., 2013] to produce a probability distribution over possible classes.”
MethodEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“Sutskever et al. [2014] investigate sequence-to-sequence models that consist of a neural network encoder and decoder for machine translation.”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“Bahdanau et al. [2015] propose the augmenting sequence to sequence models with an attention mechanism that gives the decoder access to the encoder representations of the input sequence at each step of sequence generation.”
Background
Research works printed but not linked: 6
- Q. Chen, X.-D. Zhu, Z.-H. Ling, S. Wei, and H. Jiang. Enhancing and combining sequential and tree LSTM for natural language inference. CoRR , abs/1609.06038, 2016. · arXiv
- A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , pages 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P11-1015 .
- H. T. Madabushi and M. Lee. High accuracy rule-based question classification using question syntax and semantics. In COLING , 2016.
- V. Nair and G. E. Hinton. Rectified linear units improve restricted Boltzmann machines. In ICML , 2010.
- L. Sha, B. Chang, Z. Sui, and S. Li. Reading and thinking: Re-read LSTM unit for textual entailment recognition. In COLING , 2016.
- Y. Yu, W. Zhang, K. Hasan, M. Yu, B. Xiang, and B. Zhou. End-to-end reading comprehension with dynamic answer chunk ranking. ICLR , 2017.
Cited by
Sorted by citations
Items 201–300 of 571
Text Adversarial Attacks and Defenses: Issues, Taxonomy, and Perspectives
Han, X., Zhang, Y., Wang, W., & Wang, B. (2022) · Security and Communication Networks · doi:10.1155/2022/6458488 · Cited by 32
Denoising based Sequence-to-Sequence Pre-training for Text Generation
Wang, L., Zhao, W., Jia, R., Li, S., & Liu, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1412 · Cited by 35
Sentiment Classification towards Question-Answering with Hierarchical Matching Network
Shen, C., Sun, C., Wang, J., et al. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1401 · Cited by 25
Word Representation Learning Based on Bidirectional GRUs With Drop Loss for Sentiment Classification
Sun, X., Gao, Y., Sutcliffe, R. F. E., Guo, S., Wang, X., & Feng, J. H. (2019) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2019.2940097 · Cited by 26
Transfer Learning in Multilingual Neural Machine Translation with Dynamic VocabularyPreprint
Lakew, S. M., Erofeeva, A., Negri, M., Federico, M., & Turchi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01137 · Cited by 33
Efficient Strategies for Hierarchical Text Classification: External Knowledge and Auxiliary Tasks
Rojas, K. R., Bustamante, G., Oncevay, A., & Cabezudo, M. A. S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.205 · Cited by 31
Contrastive Cross-Modal Pre-Training: A General Strategy for Small Sample Medical Imaging
Liang, G., Greenwell, C., Zhang, Y., Xing, X., Wang, X., Kavuluru, R., & Jacobs, N. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3110805 · Cited by 26
Peng, W., Hu, Y., Yu, J., Xing, L., & Xie, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107364 · Cited by 28
DAFS: a domain aware few shot generative model for event detection
Xia, N., Yu, H., Wang, Y., Xuan, J., & Luo, X. (2022) · Machine Learning · doi:10.1007/s10994-022-06198-5 · Cited by 24
Improving Sentiment Analysis in Social Media by Handling Lengthened Words
Kukkar, A., Mohana, R., Sharma, A., Nayyar, A., & Shah, M. A. (2023) · IEEE Access · doi:10.1109/access.2023.3238366 · Cited by 32
Text classification with improved word embedding and adaptive segmentation
Sun, G., Cheng, Y., Zhang, Z., Tong, X., & Chai, T. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121852 · Cited by 28
Central bank mandates and monetary policy stances: Through the lens of Federal Reserve speeches
Bertsch, C., Hull, I., Lumsdaine, R. L., & Zhang, X. (2025) · Journal of Econometrics · doi:10.1016/j.jeconom.2025.105948 · Cited by 27
Pre-train, Interact, Fine-tune: a novel interaction representation for text classification
Zheng, J., Cai, F., Chen, H., & de Rijke, M. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102215 · Cited by 25
Improved Danmaku Emotion Analysis and Its Application Based on Bi-LSTM Model
Wang, S., Chen, Y., Ming, H., Huang, H., Mi, L., & Shi, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3001046 · Cited by 24
Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
Augenstein, I., Gella, S., Ruder, S., Kann, K., Can, B., Conneau, A., Welbl, J., Ren, X., & Rei, M. (2019) · doi:10.18653/v1/w19-43 · Cited by 24
SE 3 M: A model for software effort estimation using pre-trained embedding models
Fávero, E. M. D. B., Casanova, D., & Pimentel, A. R. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.106886 · Cited by 41
Moirangthem, D. S., & Lee, M. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113898 · Cited by 24
Czert – Czech BERT-like Model for Language Representation
Sido, J., Pražák, O., Přibáň, P., Pašek, J., Seják, M., & Konopík, M. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_149 · Cited by 37
MVP: Multi-task Supervised Pre-training for Natural Language Generation
Tang, T., Li, J., Zhao, W. X., & Wen, J.-R. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.558 · Cited by 24
Richer Document Embeddings for Author Profiling tasks based on a heuristic search
López, R., Montes-y-Gómez, M., González, L. C., Ramírez-Alonso, G. M. D. J., & Prieto-Ordaz, O. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102227 · Cited by 25
Proceedings of The 5th Workshop on BioNLP Open Shared Tasks
Kim, J.-D., Nédellec, C., Bossy, R., & Deléger, L. (2019) · doi:10.18653/v1/d19-57 · Cited by 23
Experimental study on short-text clustering using transformer-based semantic similarity measure
Abdalgader, K., Matroud, A. A., & Hossin, K. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2078 · Cited by 32
SemEval-2018 Task 12: The Argument Reasoning Comprehension Task
Habernal, I., Wachsmuth, H., Gurevych, I., & Stein, B. (2018) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s18-1121 · Cited by 24
Transfer learning for language model adaptation
Saiful, B. M. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/169892 · Cited by 22
NILC at CWI 2018: Exploring Feature Engineering and Feature Learning
Hartmann, N. S., & dos Santos, L. B. (2018) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w18-0540 · Cited by 20
Retrofitting Contextualized Word Embeddings with Paraphrases
Shi, W., Chen, M., Zhou, P., & Chang, K.-W. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1113 · Cited by 22
Exploiting Stacked Autoencoders for Improved Sentiment Analysis
Ahmed, K., Nadeem, M. I., Li, D., Zheng, Z., Ghadi, Y. Y., Assam, M., & Mohamed, H. G. (2022) · Applied Sciences · doi:10.3390/app122312380 · Cited by 21
Dialogue State Tracking with Incremental Reasoning
Liao, L.-Z., Long, L., Ma, Y.-S., Lei, W., & Chua, T.-S. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00384 · Cited by 30
A Sequential Graph Neural Network for Short Text Classification
Zhao, K., Huang, L., Song, R., Shen, Q., & Xu, H. (2021) · Algorithms · doi:10.3390/a14120352 · Cited by 25
A Survey on Neural Data-to-Text Generation
Lin, Y., Ruan, T., Liu, J., & Wang, H. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3304385 · Cited by 25
Dynamic Compositionality in Recursive Neural Networks with Structure-Aware Tag Representations
Kim, T., Choi, J., Edmiston, D., Bae, S., & Lee, S.-G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016594 · Cited by 21
A comparative study of language transformers for video question answering
Yang, Z., García, N., Chu, C., Otani, M., Nakashima, Y., & Takemura, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.02.092 · Cited by 21
A Survey on Deep Learning for Website Fingerprinting Attacks and Defenses
Liu, P., He, L., & Li, Z. (2023) · IEEE Access · doi:10.1109/access.2023.3253559 · Cited by 21
Document Gated Reader for Open-Domain Question Answering
Wang, B., Yao, T., Zhang, Q., Xu, J., Tian, Z., Liu, K., & Zhao, J. (2019) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3331184.3331190 · Cited by 19
SentiLR: Linguistic Knowledge Enhanced Language Representation for Sentiment Analysis.Preprint
Ke, P., Ji, H., Liu, S., Zhu, X., & Huang, M. (2019) · arXiv (Cornell University) · Cited by 23
Detecting technological recombination using semantic analysis and dynamic network analysis
Cao, X., Chen, X., Huang, L., Deng, L., Cai, Y., & Ren, H. (2023) · Scientometrics · doi:10.1007/s11192-023-04812-4 · Cited by 19
Transfer learning for software vulnerability prediction using Transformer models
Kalouptsoglou, I., Siavvas, M., Ampatzoglou, A., Kehagias, D. D., & Chatzigeorgiou, A. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112448 · Cited by 21
Learning to compose over tree structures via POS tags for sentence representation
Shen, G., Deng, Z., Huang, T., & Chen, X. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112917 · Cited by 20
Multi-task Active Learning for Pre-trained Transformer-based Models
Rotman, G., & Reichart, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00515 · Cited by 22
A benchmark for evaluating Arabic contextualized word embedding models
Elnagar, A., Yagi, S., Mansour, Y., Lulu, L., & Fareh, S. I. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103452 · Cited by 19
Ezukwoke, K., Hoayek, A., Batton‐Hubert, M., Boucher, X., Gounet, P., & Adrian, J. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-024-02346-x · Cited by 23
Pal, K. K., Bajaj, A. P., Banerjee, P., Dutcher, A., Nakamura, M., Basque, Z. L., Gupta, H., Sawant, S. A., Anantheswaran, U., Shoshitaishvili, Y., et al. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00152 · Cited by 24
Multi-task and multi-view training for end-to-end relation extraction
Zhang, J., Zhang, Y., Ji, D., & Liu, M. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.087 · Cited by 19
Lessons from Natural Language Inference in the Clinical DomainPreprint
Romanov, A., & Shivade, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.06752 · Cited by 73
Semi-Supervised Sequence Modeling with Cross-View TrainingPreprint
Clark, K., Luong, M.-T., Manning, C. D., & Le, Q. V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.08370 · Cited by 77
Proceedings of The Third Workshop on Representation Learning for NLP
(2018) · doi:10.18653/v1/w18-30 · Cited by 20
Joint Learning of Token Context and Span Feature for Span-Based Nested NER
Sun, L. J., Sun, Y., Ji, F., & Wang, C. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3024944 · Cited by 18
Sparsity Makes Sense: Word Sense Disambiguation Using Sparse Contextualized Word Representations
Berend, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.683 · Cited by 19
Zheng, X., Du, H., Luo, X., Tong, F., Song, W., & Zhao, D. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-05051-9 · Cited by 21
Do Pretrained Language Models Indeed Understand Software Engineering Tasks?
Li, Y., Zhang, T., Luo, X., Cai, H., Fang, S., & Yuan, D. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3308952 · Cited by 25
Integrating Deep Learning and Machine Translation for Understanding Unrefined Languages
Ji, H., Oh, S., Kim, J., Choi, S. W., & Park, E. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.019521 · Cited by 18
Zero-Shot Multilingual Sentiment Analysis using Hierarchical Attentive Network and BERT
Sarkar, A., Reddy, S., & Iyengar, R. S. (2019) · International Conference on Natural Language Processing and Information Retrieval (NLPIR) · doi:10.1145/3342827.3342850 · Cited by 18
Is Language All We Need? A Query Into Architectural Semantics Using a Multimodal Generative Workflow
Bolojan, D., Vermisso, E., & Yousif, S. (2022) · Proceedings of the International Conference on Computer-Aided Architectural Design Research in Asia · doi:10.52842/conf.caadria.2022.1.353 · Cited by 20
InferLite: Simple Universal Sentence Representations from Natural Language Inference Data
Kiros, J., & Chan, W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1524 · Cited by 17
Robust Word Vectors: Context-Informed Embeddings for Noisy Texts
Malykh, V., Logacheva, V., & Khakhulin, T. (2018) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w18-6108 · Cited by 17
Learned in Speech Recognition: Contextual Acoustic Word Embeddings
Palaskar, S., Raunak, V., & Metze, F. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683868 · Cited by 17
DCNN-BiGRU Text Classification Model Based on BERT Embedding
Huang, H., Jing, X., Wu, F., Yao, Y., Zhang, X., & Dong, X. (2019) · International Conference on Ubiquitous Computing and Communications (IUCC) · doi:10.1109/iucc/dsci/smartcns.2019.00132 · Cited by 17
Creative AI: From Expressive Mimicry to Critical Inquiry
Forbes, A. G. (2020) · Artnodes · doi:10.7238/a.v0i26.3370 · Cited by 24
Applying BERT Embeddings to Predict Legal Textual Entailment
Wehnert, S., Dureja, S., Kutty, L., Sudhi, V., & Luca, E. W. D. (2022) · The Review of Socionetwork Strategies · doi:10.1007/s12626-022-00101-3 · Cited by 19
Protein sequence profile prediction using ProtAlbert transformer
Behjati, A., Zare‐Mirakabad, F., Arab, S. S., & Nowzari-Dalini, A. (2022) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2022.107717 · Cited by 22
A Survey of Text Representation Methods and Their Genealogy
Siebers, P., Janiesch, C., & Zschech, P. (2022) · IEEE Access · doi:10.1109/access.2022.3205719 · Cited by 21
Applying BERT for Early-Stage Recognition of Persistence in Chat-Based Social Engineering Attacks
Tsinganos, N., Fouliras, P., & Mavridis, I. K. (2022) · Applied Sciences · doi:10.3390/app122312353 · Cited by 18
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities
Zhao, Z., Li, Y., Hou, C., Zhao, J., Tian, R., Liu, W., Chen, Y., Sun, N., Liu, H., Mao, W., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.20 · Cited by 16
(2020) · doi:10.18653/v1/2020.conll-1.17 · Cited by 41
Aldana-Bobadilla, E., Molina-Villegas, A., Montelongo-Padilla, Y., López-Arévalo, I., & Siordia, O. S. (2021) · Applied Sciences · doi:10.3390/app112110467 · Cited by 17
Enhancing text classification with attention matrices based on BERT
Yu, Z., Li, H., & Feng, J. (2023) · Expert Systems · doi:10.1111/exsy.13512 · Cited by 16
Attentive User-Engaged Adversarial Neural Network for Community Question Answering
Xie, Y., Shen, Y., Li, Y., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6472 · Cited by 15
A GA-Based Approach to Fine-Tuning BERT for Hate Speech Detection
Madukwe, K. J., Gao, X., & Xue, B. (2020) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308419 · Cited by 17
Sentiment Analysis of Online Product Reviews Based On SenBERT-CNN
Wu, F., Shi, Z., Dong, Z., Pang, C., & Zhang, B. (2020) · International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc51923.2020.9469551 · Cited by 17
An Extensive Study on Pretrained Models for Natural Language Processing Based on Transformers
Ramprasath, M., Dhanasekaran, K., Karthick, T., Velumani, R., & Sudhakaran, P. (2022) · International Conference on Electronics and Renewable Systems (ICEARS) · doi:10.1109/icears53579.2022.9752241 · Cited by 19
Yasuda, Y., & Toda, T. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3190672 · Cited by 16
Translations as Additional Contexts for Sentence Classification
Amplayo, R. K., Lee, K., Yeo, J., et al. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/550 · Cited by 14
EntEval: A Holistic Evaluation Benchmark for Entity Representations
Chen, M., Chu, Z., Chen, Y., Stratos, K., & Gimpel, K. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1040 · Cited by 12
Value-Wise ConvNet for Transformer Models: An Infinite Time-Aware Recommender System
Saaki, M., Hosseini, S., Rahmani, S., Kangavari, M. R., Hua, W., & Zhou, X. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3219231 · Cited by 14
Combination of ELMo Representation and CNN Approaches to Enhance Service Discovery
Huang, Z. H., & Zhao, W. (2020) · IEEE Access · doi:10.1109/access.2020.3009393 · Cited by 14
Classification for Crisis-Related Tweets Leveraging Word Embeddings and Data Augmentation.
Wang, C., & Lillis, D. (2019) · Research Repository UCD (University College Dublin) · Cited by 13
Quantifying the Contextualization of Word Representations with Semantic Class Probing
Zhao, M., Dufter, P., Yaghoobzadeh, Y., & Schütze, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.109 · Cited by 14
Audio Caption in a Car Setting with a Sentence-Level Loss
Xu, X., Dinkel, H., Wu, M., & Yu, K. (2021) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp49672.2021.9362117 · Cited by 16
Generating knowledge aware explanation for natural language inference
Yang, Z., Xu, Y., Hu, J., & Dong, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103245 · Cited by 15
Wang, J., Yan, J., Gu, A., & Rush, A. M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.5 · Cited by 57
Dissecting Contextual Word Embeddings: Architecture and RepresentationPreprint
Peters, M. E., Neumann, M., Zettlemoyer, L., & Yih, W.-T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.08949 · Cited by 90
Large Scale Language Modeling: Converging on 40GB of Text in Four HoursPreprint
Puri, R., Kirby, R. M., Yakovenko, N., & Catanzaro, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.01371 · Cited by 14
Gehrmann, S., Strobelt, H., Rush, A. M., et al. (2019) · doi:10.18653/v1/p19-3 · Cited by 14
Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation
Siddhant, A., Johnson, M., Tsai, H., Ari, N., Riesa, J., Bapna, A., Fırat, O., & Raman, K. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i05.6414 · Cited by 64
A comprehensive analysis of static word embeddings for Turkish
Sarıtaş, K., Öz, C. A., & Güngör, T. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124123 · Cited by 15
BetaAlign: a deep learning approach for multiple sequence alignment
Dotan, E., Belinkov, Y., Ecker, N., Wygoda, E., Ecker, N., Alburquerque, M., Keren, O., Loewenthal, G., & Pupko, T. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btaf009 · Cited by 14
Large Scale Language Modeling: Converging on 40GB of Text in Four Hours
Puri, R., Kirby, R. M., Yakovenko, N., & Catanzaro, B. (2018) · International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD) · doi:10.1109/cahpc.2018.8645935 · Cited by 13
Better Fine-Tuning via Instance Weighting for Text Classification
Wang, Z., Bi, W., Wang, Y., & Liu, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017241 · Cited by 11
Deep Contextualized Word Embeddings for Universal Dependency Parsing
Liu, Y., Che, W., Wang, Y., Zheng, B., Qin, B., & Liu, T. (2019) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3326497 · Cited by 13
Generation-Distillation for Efficient Natural Language Understanding in Low-Data Settings
Melas-Kyriazi, L., Han, G., & Liang, C. (2019) · Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo) · doi:10.18653/v1/d19-6114 · Cited by 14
Are the Multilingual Models Better? Improving Czech Sentiment with Transformers
Přibáň, P., & Steinberger, J. (2021) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-072-4_128 · Cited by 15
Biesialska, M., Biesialska, K., & Rybiński, H. (2021) · Journal of Intelligent Information Systems · doi:10.1007/s10844-021-00664-7 · Cited by 16
Transfer learning based on lexical constraint mechanism in low-resource machine translation
Jiang, H., Zhang, C., Xin, Z., Huang, X., Li, C., & Tai, Y. (2022) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.107856 · Cited by 12
Inductive Zero-Shot Image Annotation via Embedding Graph
Wang, F., Liu, J., Zhang, S., Zhang, G., Li, Y., & Yuan, F. (2019) · IEEE Access · doi:10.1109/access.2019.2925383 · Cited by 12
Sanjanasri, J. P., Menon, V. K., Soman, K. P., et al. (2021) · Electronics · doi:10.3390/electronics10121372 · Cited by 12
Learning Lexical Subspaces in a Distributional Vector Space
Arora, K., Chakraborty, A., & Cheung, J. C. K. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00316 · Cited by 14
MultiMix: A Robust Data Augmentation Framework for Cross-Lingual NLPPreprint
Bari, M., Mohiuddin, T., & Joty, S. (2020) · arXiv (Cornell University) · Cited by 11
Multi-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension
Tay, Y., Luu, A. T., & Hui, S. C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1238 · Cited by 10
Metadata
- Authors
- McCann, B., Bradbury, J., Xiong, C., & Socher, R.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- Volume
- 30
- Pages
- 6297-6308
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1708.00107
- OpenAlex
- W2740711318
- Semantic Scholar
- bc8fa64625d9189f5801837e7b133e7fe3c581f7
- Cited by
- 572
- References
- 72
- Citations by year
- 2025: 14 · 2024: 39 · 2023: 65 · 2022: 57 · 2021: 98 · 2020: 94 · 2019: 122 · 2018: 80 · 2017: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- coves, pre-trained word embeddings, machine translation, LSTM encoder, sentiment analysis, logical consequence, question answering, question classification, transfer learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
