The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Bidirectional LSTM with attention mechanism and convolutional layer for text classification
Abstract
Abstract Neural network models have been widely used in the field of natural language processing (NLP). Recurrent neural networks (RNNs), which have the ability to process sequences of arbitrary length, are common methods for sequence modeling tasks. Long short-term memory (LSTM) is one kind of RNNs and has achieved remarkable performance in text classification. However, due to the high dimensionality and sparsity of text data, and to the complex semantics of the natural language, text classification presents difficult challenges. In order to solve the above problems, a novel and unified architecture which contains a bidirectional LSTM (BiLSTM), attention mechanism and the convolutional layer is proposed in this paper. The proposed architecture is called attention-based bidirectional long short-term memory with convolution layer (AC-BiLSTM). In AC-BiLSTM, the convolutional layer extracts the higher-level phrase representations from the word embedding vectors and BiLSTM is used to access both the preceding and succeeding context representations. Attention mechanism is employed to give different focus to the information outputted from the hidden layers of BiLSTM. Finally, the softmax classifier is used to classify the processed context information. AC-BiLSTM is able to capture both the local feature of phrases as well as global sentence semantics. Experimental verifications are conducted on six sentiment classification datasets and a question classification dataset, including detailed analysis for AC-BiLSTM. The results clearly show that AC-BiLSTM outperforms other state-of-the-art text classification methods in terms of the classification accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 6,961
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
Proceedings of the 19th International Joint Conference on Artificial Intelligence
Pujol, J. M., Delgado, J., Ramon Sangüesa, A., & Flache, A. (2005) · International Joint Conference on Artificial Intelligence · Cited by 5,771
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Peter, P., & Turney (2002) · Cited by 3,650
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts
Pang, B., & Lee, L. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1218990 · Cited by 3,301
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02145-9 · Cited by 3,281
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
Recurrent Convolutional Neural Networks for Text Classification
Lai, S., Xu, L., Liu, K., & Zhao, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9513 · Cited by 2,280
A Structured Self-attentive Sentence Embedding
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.03130 · Cited by 1,743 · cited 1×
“Lin et al. [39] proposed a self-attention mechanism.”
BackgroundProceedings of the International Joint Conference on Neural Networks, 2003.
(2003) · doi:10.1109/ijcnn.2003 · Cited by 1,674
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification
Tang, D., Qin, B., & Liu, T. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1167 · Cited by 1,519
Bidirectional Attention Flow for Machine Comprehension
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,445
Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
Socher, R., Pennington, J., Huang, E., Ng, A. Y., & Manning, C. D. (2011) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 1,431
A Decomposable Attention Model for Natural Language Inference
Parikh, A., Täckström, O., Das, D., & Uszkoreit, J. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1244 · Cited by 1,296
Semantic Compositionality through Recursive Matrix-Vector Spaces
Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012) · Cited by 1,259
Birch, A., & Osborne, M. (2011) · The Association for Computational Linguistics · Cited by 1,200
A Deep Reinforced Model for Abstractive Summarization
Paulus, R., Xiong, C., & Socher, R. (2017) · arXiv · doi:10.48550/arxiv.1705.04304 · Cited by 1,182
Baselines and Bigrams: Simple, Good Sentiment and Topic Classification
Wang, S. I., & Manning, C. D. (2012) · Cited by 1,006
Approximation of dynamical systems by continuous time recurrent neural networks
Funahashi, K.-I., & Nakamura, Y. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80125-x · Cited by 980
Recurrent Neural Network for Text Classification with Multi-Task Learning
Liu, P., Qiu, X., & Huang, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.05101 · Cited by 972
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics
Gimpel, K., Schneider, N., O’Connor, B., Das, D., Mills, D. Q., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2011) · The Association for Computational Linguistics · Cited by 890
Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN
Chen, T., Xu, R., He, Y., & Wang, X. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.10.065 · Cited by 730
Pang, B., & Lee, L. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0506075 · Cited by 678
Sentiment Analysis: Mining Opinions, Sentiments, and Emotions
Liu, B. (2015) · Cited by 656
Effective Approaches to Attention-based Neural Machine Translation
Luong, M.-T., Pham, H., & Manning, C. D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.04025 · Cited by 655
A C-LSTM Neural Network for Text Classification
Zhou, C., Sun, C., Liu, Z., & Lau, F. C. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.08630 · Cited by 641
Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model
Wang, J., Yu, L.-C., Lai, K. R., & Zhang, X. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2037 · Cited by 584
Sentiment Analysis: Mining Opinions, Sentiments, and Emotions
Zhao, J., Liu, K., & Xu, L. (2016) · Computational Linguistics · doi:10.1162/coli_r_00259 · Cited by 552
Silberer, C., & Lapata, M. (2012) · The Association for Computational Linguistics · Cited by 520
Núñez, J. C., Cabido, R., Pantrigo, J. J., Montemayor, A. S., & Vélez, J. F. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.033 · Cited by 419
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.2188 · Cited by 407
Deep Recursive Neural Networks for Compositionality in Language
İrsoy, O., & Cardie, C. (2014) · neural information processing systems · Cited by 316
CNN for situations understanding based on sentiment analysis of twitter data
Liao, S., Wang, J., Yu, R., Sato, K., & Cheng, Z. (2017) · Procedia Computer Science · doi:10.1016/j.procs.2017.06.037 · Cited by 241
Liu, Y., Bi, J.-W., & Fan, Z.-P. (2017) · Information Sciences · doi:10.1016/j.ins.2017.02.016 · Cited by 214
Attention Based LSTM for Target Dependent Sentiment Classification
Yang, M., Tu, W., Wang, J., Xu, F., & Chen, X. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11061 · Cited by 207
From pixels to sentiment: Fine-tuning CNNs for visual sentiment prediction
Campos, V., Jou, B., & Giró-i-Nieto, X. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.01.011 · Cited by 197
From symbolic to sub-symbolic information in question classification
Silva, J., Coheur, L., Mendes, A. C., & Wichert, A. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-010-9188-4 · Cited by 186
Recurrent neural networks for classifying relations in clinical notes
Luo, Y. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.07.006 · Cited by 177
FusionNet: Fusing via Fully-Aware Attention with Application to Machine ComprehensionPreprint
Huang, H.-Y., Zhu, C., Shen, Y., & Chen, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07341 · Cited by 167
Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence ModelingPreprint
Shen, T., Zhou, T., Long, G., Jiang, J. B., & Chengqi Zhang, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00857 · Cited by 159
LSTM Recurrent Neural Networks for Short Text and Sentiment Classification
Nowak, J., Taspinar, A., & Scherer, R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59060-8_50 · Cited by 156
Convolutional LSTM Networks for Subcellular Localization of Proteins
Sønderby, S. K., Sønderby, C. K., Nielsen, H., & Winther, O. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-21233-3_6 · Cited by 144
Encoding Syntactic Knowledge in Neural Networks for Sentiment Classification
Huang, M., Qian, Q., & Zhu, X. (2017) · ACM Transactions on Information Systems · doi:10.1145/3052770 · Cited by 129
Zhang, Z., Zou, Y., & Gan, C. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.09.080 · Cited by 126
Wang, L., Wang, Z., & Liu, S. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.08.055 · Cited by 123
Text normalization and semantic indexing to enhance Instant Messaging and SMS spam filtering
Almeida, T. A., Silva, T. P., Santos, I., & Hidalgo, J. M. G. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.05.001 · Cited by 109
A multi-view recurrent neural network for 3D mesh segmentation
Le, T., Bui, G. T. T., & Duan, Y. (2017) · Computers & Graphics · doi:10.1016/j.cag.2017.05.011 · Cited by 108
A Convolution-LSTM-Based Deep Neural Network for Cross-Domain MOOC Forum Post Classification
Wei, X., Lin, H., Yang, L., & Yu, Y. (2017) · Information · doi:10.3390/info8030092 · Cited by 103
Sentiment classification using Comprehensive Attention Recurrent models
Zhang, Y., Meng, J. E., Venkatesan, R., Wang, N., & Pratama, M. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727384 · Cited by 61
Deep Belief Neural Networks and Bidirectional Long-Short Term Memory Hybrid for Speech Recognition
Brocki, Ł., & Marasek, K. P. (2015) · Archives of Acoustics · doi:10.1515/aoa-2015-0021 · Cited by 45
Emphasizing Essential Words for Sentiment Classification Based on Recurrent Neural Networks
Hu, F., Li, L., Zhang, Z., Wang, J., & Xu, X. (2017) · Journal of Computer Science and Technology · doi:10.1007/s11390-017-1759-2 · Cited by 41
Lee, L. H., Isa, D., Choo, W. O., & Chue, W. Y. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.07.116 · Cited by 35
Wu, D., & Chi, M. (2017) · IEEE Access · doi:10.1109/access.2016.2647384 · Cited by 31
A P-LSTM Neural Network for Sentiment Classification
Lu, C., Huang, H., Jian, P., Wang, D., & Guo, Y. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-57454-7_41 · Cited by 20
Bi-Directional LSTM with Quantum Attention Mechanism for Sentence Modeling
Niu, X., Hou, Y., & Wang, P. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70096-0_19 · Cited by 20
Proceedings of the 9th international conference on Artificial Intelligence and Soft Computing
Rutkowski, L., Tadeusiewicz, R., Zadeh, L. A., et al. (2006) · Cited by 19
Textual Inference with Tree-Structured LSTM
John, A. K., Di, L., Robaldo, L., & Guido Boella, A. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-67468-1_2 · Cited by 11
Cited by
A review on the attention mechanism of deep learning
Niu, Z., Zhong, G., & Yu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.03.091 · Cited by 3,379
ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis
Basiri, M. E., Nemati, S., Abdar, M., Cambria, E., & Acharya, U. R. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.08.005 · Cited by 745
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
“The recurrent neural models could not capture distant dependencies in data sequences, for example, information occurring at the text beginning or end [7].”
BackgroundJang, B., Kim, M., Harerimana, G., Kang, S.-U., & Kim, J. W. (2020) · Applied Sciences · doi:10.3390/app10175841 · Cited by 442
“1D CNN processes text as a one-dimensional image and a 1D CNN is used to capture the latent associations between neighboring words, in contrast with LSTMs, which process each word in a sequential pattern [8].”
MethodCNN-based transfer learning–BiLSTM network: A novel approach for COVID-19 infection detection
Aslan, M. F., Unlersen, M. F., Sabanci, K., & Durdu, A. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106912 · Cited by 354
Metadata
- Authors
- Liu, G., & Guo, J.
- Venue
- Neurocomputing
- Publisher
- Elsevier BV
- Published
- 2019-04
- Volume
- 337
- Pages
- 325-338
- ISSN
- 0925-2312 (ISSN-L)
- OpenAlex
- W2914767245
- Cited by
- 1,127
- References
- 81
- FWCI
- 51.57
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.