The DOI matches its DataCite record; 63 of the 72 references it cites were found in a registry or an index.
Learned in Translation: Contextualized Word Vectors
Open accessAbstract
Computer vision has benefited from initializing multiple deep layers with weights pretrained on large supervised training sets like ImageNet. Natural language processing (NLP) typically sees initialization of only the lowest layer of deep models with pretrained word vectors. In this paper, we use a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation (MT) to contextualize word vectors. We show that adding these context vectors (CoVe) improves performance over using only unsupervised word and character vectors on a wide variety of common NLP tasks: sentiment analysis (SST, IMDb), question classification (TREC), entailment (SNLI), and question answering (SQuAD). For fine-grained sentiment analysis and entailment, CoVe improves performance of our baseline models to the state of the art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The increase of 4.7% with ELMo is also significantly larger then the 1.8% improvement from adding CoVe to a baseline model (McCann et al., 2017).”
From Deep Contextualized Word Representations (2018) · cites this paper as McCann et al., 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For tasks where direct comparisons are possible, ELMo outperforms CoVe (McCann et al., 2017), which computes contextualized representations using a neural machine translation encoder.”
From Deep Contextualized Word Representations (2018) · cites this paper as McCann et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“[2016] propose to combine text representations with visual representations extracted by a pretrained residual network [He et al., 2016].”
BackgroundMethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“Qi et al. [2016] propose a CNN-based object tracking framework, which uses hierarchical features from a pretrained CNN (VGG-19 by Simonyan and Zisserman [2014]).”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“In computer vision, convolutional neural networks (CNNs) pretrained on ImageNet [Krizhevsky et al., 2012, Deng et al., 2009] have become the de facto initialization for more complex and deeper models.”
MethodSelf-citationGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 14×
“The MT-LSTM is used to compute a sequence of hidden states h = MT-LSTM ( GloVe ( w x )) .”
BackgroundMethodInfluentialSelf-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · cited 1×
“Girshick et al. [2014] show that using a pretrained CNN to extract features from region proposals improves object detection and semantic segmentation models.”
MethodBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 1×
“…representation for all inputs. x pool = max( X | y ); mean( X | y ); min( X | y ); x self (15) We feed this joined representation through a three-layer, batch-normalized [Ioffe and Szegedy, 2015] maxout network [Goodfellow et al., 2013] to produce a probability distribution over possible classes.”
MethodEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1×
“Sutskever et al. [2014] investigate sequence-to-sequence models that consist of a neural network encoder and decoder for machine translation.”
BackgroundNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 1×
“Bahdanau et al. [2015] propose the augmenting sequence to sequence models with an attention mechanism that gives the decoder access to the encoder representations of the input sequence at each step of sequence generation.”
Background
Research works printed but not linked: 6
- Q. Chen, X.-D. Zhu, Z.-H. Ling, S. Wei, and H. Jiang. Enhancing and combining sequential and tree LSTM for natural language inference. CoRR , abs/1609.06038, 2016. · arXiv
- A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , pages 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P11-1015 .
- H. T. Madabushi and M. Lee. High accuracy rule-based question classification using question syntax and semantics. In COLING , 2016.
- V. Nair and G. E. Hinton. Rectified linear units improve restricted Boltzmann machines. In ICML , 2010.
- L. Sha, B. Chang, Z. Sui, and S. Li. Reading and thinking: Re-read LSTM unit for textual entailment recognition. In COLING , 2016.
- Y. Yu, W. Zhang, K. Hasan, M. Yu, B. Xiang, and B. Zhou. End-to-end reading comprehension with dynamic answer chunk ranking. ICLR , 2017.
Cited by
Sorted by citations
Items 101–200 of 571
Analyzing Redundancy in Pretrained Transformer Models
Dalvi, F., Sajjad, H., Durrani, N., & Belinkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.398 · Cited by 97
Constructing and analyzing domain-specific language model for financial text mining
Suzuki, M., Sakaji, H., Hirano, M., & Izumi, K. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103194 · Cited by 67
Improving AMR Parsing with Sequence-to-Sequence Pre-training
Xu, D., Li, J., Zhu, M., Zhang, M., & Zhou, G. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.196 · Cited by 64
Chen, X., Lowerison, M. R., Dong, Z., Sekaran, N. V. C., Huang, C., Chen, S., Fan, T. M., Llano, D. A., & Song, P. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3251197 · Cited by 64
Are Pretrained Convolutions Better than Pretrained Transformers?
Tay, Y., Dehghani, M., Gupta, J. P., Aribandi, V., Bahri, D., Qin, Z., & Metzler, D. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.335 · Cited by 99
Imposing Label-Relational Inductive Bias for Extremely Fine-Grained Entity Typing
Xiong, W., Wu, J., Lei, D., et al. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1084 · Cited by 57
Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research
Gruetzemacher, R., & Paradice, D. (2022) · ACM Computing Surveys · doi:10.1145/3505245 · Cited by 68
SMIT: Stochastic Multi-Label Image-to-Image Translation
Romero, A., Arbeláez, P., Gool, L. V., & Timofte, R. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00410 · Cited by 59
Özçift, A., Akarsu, K., Yumuk, F., & Söylemez, C. (2021) · Automatika · doi:10.1080/00051144.2021.1922150 · Cited by 68
Fusing Label Embedding into BERT: An Efficient Improvement for Text Classification
Xiong, Y., Feng, Y., Wu, H., Kamigaito, H., & Okumura, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.152 · Cited by 60
AgriBERT: Knowledge-Infused Agricultural Language Models for Matching Food and Nutrition
Rezayi, S., Liu, Z., Wu, Z., Dhakal, C., Ge, B., Zhen, C., Liu, T., & Li, S. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/715 · Cited by 65
Data science opportunities of large language models for neuroscience and biomedicine
Bzdok, D., Thieme, A., Levkovskyy, O., Wren, P. B., Ray, T. S., & Reddy, S. (2024) · Neuron · doi:10.1016/j.neuron.2024.01.016 · Cited by 65
Adversarial Domain Adaptation for Machine Reading Comprehension
Wang, H., Gan, Z., Liu, X., Liu, J., Gao, J., & Wang, H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1254 · Cited by 65
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models
Huang, P.-Y., Patrick, M., Hu, J., Neubig, G., Metze, F., & Hauptmann, A. G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.195 · Cited by 68
Dabre, R., Fujita, A., & Chu, C. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1146 · Cited by 56
Label-Agnostic Sequence Labeling by Copying Nearest Neighbors
Wiseman, S., & Stratos, K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1533 · Cited by 56
Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark
Nangia, N., & Bowman, S. R. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1449 · Cited by 68
Neural Machine Translation with BERT for Post-OCR Error Detection and Correction
Nguyen, T. T. H., Jatowt, A., Nguyen, N.-V., Coustaty, M., & Doucet, A. (2020) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3383583.3398605 · Cited by 57
Enhancing Aspect-Based Sentiment Analysis With Capsule Network
Su, J., Yu, S., & Luo, D. (2020) · IEEE Access · doi:10.1109/access.2020.2997675 · Cited by 50
Sentiment Analysis Methods for HPV Vaccines Related Tweets Based on Transfer Learning
Zhang, L., Fan, H., Peng, C., Rao, G., & Cong, Q. (2020) · Healthcare · doi:10.3390/healthcare8030307 · Cited by 53
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search
Kiros, J., Chan, W., & Hinton, G. E. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1085 · Cited by 50
Wang, W., Caswell, I. R., & Chelba, C. I. (2019) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p19-1123 · Cited by 49
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
Yang, Z., Zhu, C., Gmyr, R., Zeng, M., Huang, X., & Darve, E. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.168 · Cited by 51
A survey on training and evaluation of word embeddings
Torregrossa, F., Allesiardo, R., Claveau, V., Kooli, N., & Gravier, G. (2021) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00242-8 · Cited by 53
Das, S., & Kolya, A. K. (2021) · Evolutionary Intelligence · doi:10.1007/s12065-021-00598-7 · Cited by 50
srBERT: automatic article classification model for systematic review using BERT
Aum, S., & Choe, S. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01763-w · Cited by 52
Multi-task Learning with Sample Re-weighting for Machine Reading Comprehension
Xu, Y., Liu, X., Shen, Y., Liu, J., & Gao, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1271 · Cited by 60
Trouble on the Horizon: Forecasting the Derailment of Online Conversations as they Develop
Chang, J. P., & Danescu-Niculescu-Mizil, C. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1481 · Cited by 53
Biden vs Trump: Modeling US General Elections Using BERT Language Model
Chandra, R., & Saini, R. (2021) · IEEE Access · doi:10.1109/access.2021.3111035 · Cited by 52
Which *BERT? A Survey Organizing Contextualized Encoders
Xia, P., Wu, S., & Durme, B. V. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.608 · Cited by 57
Decoding Word Embeddings with Brain-Based Semantic Features
Chersoni, E., Santus, E., Huang, C., & Lenci, A. (2021) · Computational Linguistics · doi:10.1162/coli_a_00412 · Cited by 57
Vector based sentiment and emotion analysis from text: A survey
Uymaz, H. A., & Metın, S. K. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104922 · Cited by 54
Association Analysis: Basic Concepts and Algorithms
Chen, Q. (2024) · Springer eBooks · doi:10.1007/978-981-99-8251-6_2 · Cited by 46
Dynamic Scene Graph Generation via Anticipatory Pre-training
Li, Y., Yang, X., & Xu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01350 · Cited by 49
Zhang, K. W., & Bowman, S. R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10040 · Cited by 45
Explicit Utilization of General Knowledge in Machine Reading Comprehension
Wang, C., & Jiang, H. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1219 · Cited by 47
Low-Rank and Locality Constrained Self-Attention for Sequence Modeling
Guo, Q., Qiu, X., Xue, X., & Zhang, Z. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2944078 · Cited by 47
Sun, C. H., & Yang, Z. (2019) · Workshop on BioNLP Open Shared Tasks (BioNLP-OST) · doi:10.18653/v1/d19-5715 · Cited by 45
Survey of Neural Text Representation Models
Babić, K., Martinčić-Ipšić, S., & Meštrović, A. (2020) · Information · doi:10.3390/info11110511 · Cited by 48
Multimodal Relational Tensor Network for Sentiment and Emotion Classification
Sahay, S., Kumar, S. H., Xia, R., Huang, J., & Nachman, L. (2018) · doi:10.18653/v1/w18-3303 · Cited by 46
WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse
Faruqui, M., Pavlick, E., Tenney, I., & Das, D. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1028 · Cited by 43
Silva, D. D., Mills, N., El-Ayoubi, M., Manic, M., & Alahakoon, D. (2023) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit58465.2023.10143123 · Cited by 52
Rezayi, S., Liu, Z., Wu, Z., Dhakal, C., Ge, B., Dai, H., Mai, G., Liu, N., Zhen, C., Liu, T., et al. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3442542 · Cited by 65
Joint Contextual Modeling for ASR Correction and Language Understanding
Weng, Y., Miryala, S. S., Khatri, C., Wang, R., Zheng, H., Molino, P., Namazifar, M., Papangelis, A., Williams, H. E., Bell, F., et al. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053213 · Cited by 45
A Novel Inference Algorithm for Large Sparse Neural Network using Task Graph Parallelism
Lin, D.-L., & Huang, T. (2020) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec43674.2020.9286218 · Cited by 42
Explicit Alignment Objectives for Multilingual Bidirectional Encoders
Hu, J., Johnson, M., Fırat, O., Siddhant, A., & Neubig, G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.284 · Cited by 57
A comparative study of cross-lingual sentiment analysis
Přibáň, P., Šmíd, J., Steinberger, J., & Mištera, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123247 · Cited by 49
Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference
Pan, B., Yang, Y., Zhao, Z., Zhuang, Y., Cai, D., & He, X. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1091 · Cited by 42
Transfer Learning for Punctuation Prediction
Makhija, K., Ho, T.-N., & Chng, E. S. (2019) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc47483.2019.9023200 · Cited by 41
Comprehensive analysis of embeddings and pre-training in NLP
Tripathy, J. K., Sethuraman, S. C., Cruz, M. V., Namburu, A., Mangalraj, P., Nandha Kumar, R., Sudhakar Ilango, S., & Vijayakumar, V. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100433 · Cited by 46
Tree Communication Models for Sentiment Analysis
Zhang, Y., & Zhang, Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1342 · Cited by 40
Adaptive cross-contextual word embedding for word polysemy with unsupervised topic modeling
Li, S., Pan, R., Luo, H., Liu, X., & Zhao, G. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106827 · Cited by 46
Inline Detection of Domain Generation Algorithms with Context-Sensitive Word Embeddings
Koh, J. J., & Rhodes, B. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622066 · Cited by 43
Drug-Drug Interaction Extraction from Biomedical Text Using Relation BioBERT with BLSTM
KafiKang, M., & Hendawi, A. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5020036 · Cited by 43
Cold-Start Aware User and Product Attention for Sentiment Classification
Amplayo, R. K., Kim, J., Sung, S., et al. (2018) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p18-1236 · Cited by 38
Narayanan, S., Mannam, K., Achan, P., Ramesh, M. V., Rangan, P. V., & Rajan, S. P. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2021.103960 · Cited by 40
Efficient Adaptation of Pretrained Transformers for Abstractive SummarizationPreprint
Hoang, A., Bosselut, A., Çelikyılmaz, A., & Choi, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00138 · Cited by 54
Tay, Y., Luu, A. T., & Hui, S. C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1479 · Cited by 36
Fully Transformer Networks for Semantic Image SegmentationPreprint
Wu, S., Wu, T., Lin, F., Tian, S., & Guo, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04108 · Cited by 64
Body, T., Tao, X., Li, Y., Li, L., & Zhong, N. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115033 · Cited by 41
Cooperative Learning of Disjoint Syntax and Semantics
Havrylov, S., Kruszewski, G., & Joulin, A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1115 · Cited by 43
Sentiment aware word embeddings using refinement and senti-contextualized learning approach
Naderalvojoud, B., & Sezer, E. A. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.094 · Cited by 35
Contextualized Word Representations for Reading Comprehension
Salant, S., & Berant, J. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2088 · Cited by 34
Poliak, A., Belinkov, Y., Glass, J. R., & Durme, B. V. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2082 · Cited by 39
Fusion Learning using Semantics and Graph Convolutional Network for Visual Food Recognition
Heng, Z., Yap, K., & Kot, A. C. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00175 · Cited by 34
Identifying wildlife observations on twitter
Edwards, T., Jones, C. B., & Corcoran, P. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101500 · Cited by 34
An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese
Bataa, E., & Wu, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1458 · Cited by 35
Effective Subword Segmentation for Text Comprehension
Zhang, Z., Zhao, H., Ling, K., Li, J., Li, Z., He, S., & Fu, G. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2922537 · Cited by 31
Luo, Y., Zhang, H., Wen, Y., & Zhang, X. (2019) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3357384.3357899 · Cited by 35
The Impact of Domain-Specific Pre-Training on Named Entity Recognition Tasks in Materials SciencePreprint
Walker, N., Trewartha, A., Huo, H., Lee, S.-H., Cruse, K., Dagdelen, J. M., Dunn, A., Persson, K. A., Ceder, G., & Jain, A. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3950755 · Cited by 36
A Simple and Effective Approach to Automatic Post-Editing with Transfer Learning
Correia, G. M., & Martins, A. F. T. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1292 · Cited by 39
Simple and Effective Paraphrastic Similarity from Parallel Translations
Wieting, J., Gimpel, K., Neubig, G., & Berg-Kirkpatrick, T. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1453 · Cited by 35
Atalaya at SemEval 2019 Task 5: Robust Embeddings for Tweet Classification
Pérez, J. M., & Luque, F. M. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2008 · Cited by 31
Incorporating Scenario Knowledge into A Unified Fine-tuning Architecture for Event Representation
Zheng, J., Cai, F., & Chen, H. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401173 · Cited by 31
QuASE: Question-Answer Driven Sentence Encoding
He, H., Qiang, N., & Roth, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.772 · Cited by 31
Nishida, K., Saito, I., Otsuka, A., Asano, H., & Tomita, J. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3271702 · Cited by 45
Polysemy—Evidence from Linguistics, Behavioral Science, and Contextualized Language Models
Haber, J., & Poesio, M. (2023) · Computational Linguistics · doi:10.1162/coli_a_00500 · Cited by 57
Hierarchical Inter-Attention Network for Document Classification with Multi-Task Learning
Tian, B., Zhang, Y., Wang, J., et al. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/495 · Cited by 30
CRIME: Input-Dependent Collaborative Inference for Recurrent Neural Networks
Pagliari, D. J., Chiaro, R., Macii, E., & Poncino, M. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.3021199 · Cited by 30
A Novel Joint Framework for Multiple Chinese Events Extraction
Xu, N., Xie, H., & Zhao, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63031-7_13 · Cited by 30
Exploring the fine-grained analysis and automatic detection of irony on Twitter
Hee, C. V., Lefever, E., & Hoste, V. (2018) · Language Resources and Evaluation · doi:10.1007/s10579-018-9414-2 · Cited by 31
Improving the Performance of Sentiment Classification on Imbalanced Datasets With Transfer Learning
Xiao, Z., Wang, L., & Du, J. (2019) · IEEE Access · doi:10.1109/access.2019.2892094 · Cited by 30
(2019) · doi:10.18653/v1/d19-1432 · Cited by 88
“Large-scale language modeling plays a central role in both text generation (Sordoni et al., 2015; Nal-lapati et al., 2016) and unsupervised pre-training (Vaswani et al., 2013; Dai and Le, 2015; McCann et al., 2017; Peters et al., 2018; Devlin et al., 2018; Radford et al., 2018).”
BackgroundDeep Representation Learning: Fundamentals, Technologies, Applications, and Open Challenges
Payandeh, A., Baghaei, K. T., Fayyazsanavi, P., Ramezani, S. B., Chen, Z., & Rahimi, S. (2023) · IEEE Access · doi:10.1109/access.2023.3335196 · Cited by 49
Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations
Chen, M., Chu, Z., & Gimpel, K. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1060 · Cited by 29
DICE: Deep Intelligent Contextual Embedding for Twitter Sentiment Analysis
Naseem, U., & Musiał, K. (2019) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2019.00157 · Cited by 29
Introduction to Neural Transfer Learning With Transformers for Social Science Text Analysis
Wankmüller, S. (2022) · Sociological Methods & Research · doi:10.1177/00491241221134527 · Cited by 45
Li, K., Xian, X., Wang, J., & Yu, N. (2018) · Applied Surface Science · doi:10.1016/j.apsusc.2018.11.214 · Cited by 30
A survey on knowledge-enhanced multimodal learning
Lymperaiou, M., & Stamou, G. B. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10825-z · Cited by 45
Fine-Grained Entity Typing in Hyperbolic Space
López, F., Heinzerling, B., & Strube, M. (2019) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w19-4319 · Cited by 32
Faster Depth-Adaptive Transformers
Liu, Y., Meng, F., Zhou, J., Chen, Y., & Xu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17584 · Cited by 29
Dense Information Flow for Neural Machine Translation
Shen, Y., Tan, X., He, D., Qin, T., & Liu, T.-Y. (2018) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n18-1117 · Cited by 36
Improving Latent Alignment in Text Summarization by Generalizing the Pointer Generator
Shen, X., Zhao, Y., Su, H., & Klakow, D. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1390 · Cited by 28
Learning sentiment sentence representation with multiview attention model
Zhang, Y., Wang, J., & Zhang, X. (2021) · Information Sciences · doi:10.1016/j.ins.2021.05.044 · Cited by 29
Socially Enhanced Situation Awareness from Microblogs Using Artificial Intelligence: A Survey
Lamsal, R., Harwood, A., & Rodriguez, M. A. (2022) · ACM Computing Surveys · doi:10.1145/3524498 · Cited by 31
A deep learning-based network for the simulation of airborne electromagnetic responses
Wu, S., Huang, Q., & Zhao, L. (2022) · Geophysical Journal International · doi:10.1093/gji/ggac463 · Cited by 28
Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing---and BackPreprint
Meyerson, E., & Miikkulainen, R. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04062 · Cited by 32
Retrieve-and-Read: Multi-task Learning of Information Retrieval and Reading ComprehensionPreprint
Nishida, K., Saito, I., Otsuka, A., Asano, H., & Tomita, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.10628 · Cited by 27
Neural Multi-Task Learning for Stance Prediction
Fang, W., Nadeem, M., Mohtarami, M., & Glass, J. R. (2019) · Fact Extraction and VERification Workshop (FEVER) · doi:10.18653/v1/d19-6603 · Cited by 26
Accelerating Large Sparse Neural Network Inference using GPU Task Graph Parallelism
Lin, D.-L., & Huang, T. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3138856 · Cited by 26
Metadata
- Authors
- McCann, B., Bradbury, J., Xiong, C., & Socher, R.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- Volume
- 30
- Pages
- 6297-6308
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1708.00107
- OpenAlex
- W2740711318
- Semantic Scholar
- bc8fa64625d9189f5801837e7b133e7fe3c581f7
- Cited by
- 572
- References
- 72
- Citations by year
- 2025: 14 · 2024: 39 · 2023: 65 · 2022: 57 · 2021: 98 · 2020: 94 · 2019: 122 · 2018: 80 · 2017: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- coves, pre-trained word embeddings, machine translation, LSTM encoder, sentiment analysis, logical consequence, question answering, question classification, transfer learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
