The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Bidirectional LSTM with attention mechanism and convolutional layer for text classification
Abstract
Abstract Neural network models have been widely used in the field of natural language processing (NLP). Recurrent neural networks (RNNs), which have the ability to process sequences of arbitrary length, are common methods for sequence modeling tasks. Long short-term memory (LSTM) is one kind of RNNs and has achieved remarkable performance in text classification. However, due to the high dimensionality and sparsity of text data, and to the complex semantics of the natural language, text classification presents difficult challenges. In order to solve the above problems, a novel and unified architecture which contains a bidirectional LSTM (BiLSTM), attention mechanism and the convolutional layer is proposed in this paper. The proposed architecture is called attention-based bidirectional long short-term memory with convolution layer (AC-BiLSTM). In AC-BiLSTM, the convolutional layer extracts the higher-level phrase representations from the word embedding vectors and BiLSTM is used to access both the preceding and succeeding context representations. Attention mechanism is employed to give different focus to the information outputted from the hidden layers of BiLSTM. Finally, the softmax classifier is used to classify the processed context information. AC-BiLSTM is able to capture both the local feature of phrases as well as global sentence semantics. Experimental verifications are conducted on six sentiment classification datasets and a question classification dataset, including detailed analysis for AC-BiLSTM. The results clearly show that AC-BiLSTM outperforms other state-of-the-art text classification methods in terms of the classification accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 6,961
Cited by
A review on the attention mechanism of deep learning
Niu, Z., Zhong, G., & Yu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.03.091 · Cited by 3,379
ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis
Basiri, M. E., Nemati, S., Abdar, M., Cambria, E., & Acharya, U. R. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.08.005 · Cited by 745
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758
“The recurrent neural models could not capture distant dependencies in data sequences, for example, information occurring at the text beginning or end [7].”
BackgroundJang, B., Kim, M., Harerimana, G., Kang, S.-U., & Kim, J. W. (2020) · Applied Sciences · doi:10.3390/app10175841 · Cited by 442
“1D CNN processes text as a one-dimensional image and a 1D CNN is used to capture the latent associations between neighboring words, in contrast with LSTMs, which process each word in a sequential pattern [8].”
MethodCNN-based transfer learning–BiLSTM network: A novel approach for COVID-19 infection detection
Aslan, M. F., Unlersen, M. F., Sabanci, K., & Durdu, A. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106912 · Cited by 354
Evaluation of Sentiment Analysis in Finance: From Lexicons to Transformers
Mishev, K., Gjorgjevikj, A., Vodenska, I., Chitkushev, L. T., & Trajanov, D. (2020) · IEEE Access · doi:10.1109/access.2020.3009626 · Cited by 345
Li, W., Qi, F., Tang, M., & Yu, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.006 · Cited by 318
Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals
Zhang, Y., Chen, Y., Wang, J., & Pan, Z. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3102110 · Cited by 312
Attention, please! A survey of neural attention models in deep learning
de Santana Correia, A., & Colombini, E. L. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10148-x · Cited by 298
Onan, A. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.02.025 · Cited by 289
CNN-Based Transformer Model for Fault Detection in Power System Networks
Thomas, J. B., Chaudhari, S. G., K. V., S., & Verma, N. K. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3238059 · Cited by 275
Lv, H., Chen, J., Pan, T., Zhang, T., Feng, Y., & Liu, S. (2022) · Measurement · doi:10.1016/j.measurement.2022.111594 · Cited by 261
Fuzzy logic applied to opinion mining: A review
Serrano-Guerrero, J., Romero, F. P., & Olivas, J. A. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107018 · Cited by 141
A Mathematical Investigation of Hallucination and Creativity in GPT Models
Lee, M. (2023) · Mathematics · doi:10.3390/math11102320 · Cited by 105
Cost-sensitive PLM-based Approach for Arabic and English Cyberbullying Classification
Harbaoui, A., Benaissa, A. R., & ghezala, H. B. (2023) · doi:10.21203/rs.3.rs-2524732/v1 · Cited by 3
Metadata
- Authors
- Liu, G., & Guo, J.
- Venue
- Neurocomputing
- Publisher
- Elsevier BV
- Published
- 2019-04
- Volume
- 337
- Pages
- 325-338
- ISSN
- 0925-2312 (ISSN-L)
- OpenAlex
- W2914767245
- Cited by
- 1,127
- References
- 81
- FWCI
- 51.57
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.