The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Bi-LSTM Model to Increase Accuracy in Text Classification: Combining Word2vec CNN and Attention Mechanism
Open accessAbstract
There is a need to extract meaningful information from big data, classify it into different categories, and predict end-user behavior or emotions. Large amounts of data are generated from various sources such as social media and websites. Text classification is a representative research topic in the field of natural-language processing that categorizes unstructured text data into meaningful categorical classes. The long short-term memory (LSTM) model and the convolutional neural network for sentence classification produce accurate results and have been recently used in various natural-language processing (NLP) tasks. Convolutional neural network (CNN) models use convolutional layers and maximum pooling or max-overtime pooling layers to extract higher-level features, while LSTM models can capture long-term dependencies between word sequences hence are better used for text classification. However, even with the hybrid approach that leverages the powers of these two deep-learning models, the number of features to remember for classification remains huge, hence hindering the training process. In this study, we propose an attention-based Bi-LSTM+CNN hybrid model that capitalize on the advantages of LSTM and CNN with an additional attention mechanism. We trained the model using the Internet Movie Database (IMDB) movie review data to evaluate the performance of the proposed model, and the test results showed that the proposed hybrid attention Bi-LSTM+CNN model produces more accurate classification results, as well as higher recall and F1 scores, than individual multi-layer perceptron (MLP), CNN or LSTM models as well as the hybrid models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the other hand, long-term dependencies between word sequences can be captured by LSTM models, which is a vital requirement in the context of cyberbullying detection [125].”
From A Review on Deep-Learning-Based Cyberbullying Detection (2023) · cites this paper as [125]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“The model leverages the novelty of BERT (bidirectional encoder representation from transformers) [31] for text extraction.”
MethodConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 2×
“To further increase the sentiment classification accuracy of unstructured text, Yoon Kim [7] proposed a 1D CNN approach for text classification.”
MethodFramewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 6,152
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601 · cited 1×
“With the 1D CNN, the convolution layer uses a 1D cross-correlation operation that involves a sliding convolution window with variable-size kernels over the input text starting from left to right [13].”
MethodRecurrent Convolutional Neural Networks for Text Classification
Lai, S., Xu, L., Liu, K., & Zhao, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9513 · Cited by 2,384
Deep learning for sentiment analysis: A survey
Zhang, L., Wang, S., & Liu, B. (2018) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1253 · Cited by 2,182 · cited 1×
“Natural-language-based sentiment classification has a wide range of applications, such as movie review classification, subjective and objective sentence classification, and text classification technology [1].”
BackgroundComparative Study of CNN and RNN for Natural Language ProcessingPreprint
Yin, W., Kann, K., Yu, M., & Schütze, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01923 · Cited by 1,264 · cited 1×
“Figure 2 shows the basic architecture of the 1D CNN [14].”
BackgroundBidirectional LSTM with attention mechanism and convolutional layer for text classification
Liu, G., & Guo, J. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.01.078 · Cited by 1,166 · cited 1×
“1D CNN processes text as a one-dimensional image and a 1D CNN is used to capture the latent associations between neighboring words, in contrast with LSTMs, which process each word in a sequential pattern [8].”
MethodA Convolutional Neural Network for Modelling SentencesPreprint
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.2188 · Cited by 967 · cited 1×
“With the 1D CNN, the convolution layer uses a 1D cross-correlation operation that involves a sliding convolution window with variable-size kernels over the input text starting from left to right [13].”
Method
Research works printed but not linked: 2
- Pouyanfar, 2018, A survey on deep learning: Algorithms, techniques, and applications, ACM Comput. Surv. (CSUR), 51, 1
- Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv.
Cited by
Sorted by citations
A Survey of Sentiment Analysis: Approaches, Datasets, and Future Research
Tan, K. L., Lee, C. P., & Lim, K. M. (2023) · Applied Sciences · doi:10.3390/app13074550 · Cited by 307
“Jang et al. (2020) [34] improved on the hybrid CNN and BiLSTM model by incorporating an attention mechanism.”
Shaik, T., Tao, X., Li, Y., Dann, C. E., McDonald, J., Redmond, P., & Galligan, L. (2022) · IEEE Access · doi:10.1109/access.2022.3177752 · Cited by 362
“…[172]–[174] Text Evaluation Text Summarization LSTM-NN, CNN, Bi-GRU, TextRank, LSA, BERT, LSTM-CNN [57], [92]–[109] LSTM, CNN, BLSTM-C [19], [110]–[114] Entity Extraction CNN [115]–[117] Knowledge Graphs Multidimensional knowledge graphs [118], [119] CNN, LSTM [3], [120]–[129] implementation…”
MethodImpact of word embedding models on text analytics in deep learning environment: a review
Asudani, D. S., Nagwani, N. K., & Singh, P. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10419-1 · Cited by 274
“The construction of text classification and document classification techniques includes extracting features, dimension minimization, classifier selection, and assessments (Jang et al. 2020).”
MethodWearable Sensor-Based Human Activity Recognition with Transformer Model
Dirgová Luptáková, I., Kubovčík, M., & Pospíchal, J. (2022) · Sensors · doi:10.3390/s22051911 · Cited by 270
“Before the transformer model was introduced, the cutting-edge systems for complex time series problems such as natural language processing were based on gated recurrent neural networks with a superimposed attention mechanism [42].”
MethodBaviskar, D., Ahirrao, S. N., Potdar, V. M., & Kotecha, K. V. (2021) · IEEE Access · doi:10.1109/access.2021.3072900 · Cited by 175
“We found few other studies also using LSTM and GRU for information extraction task [29], [50], [57], [68], [110].”
MethodResult
Metadata
- Authors
- Jang, B., Kim, M., Harerimana, G., Kang, S.-U., & Kim, J. W.
- Affiliations
- Sangmyung University (KR)
- Corresponding author
- Kim, Jong Wook
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2020-08-24
- Volume
- 10
- Issue
- 17
- Pages
- 5841
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3080359515
- Cited by
- 442
- References
- 33
- Citations by year
- 2026: 25 · 2025: 93 · 2024: 97 · 2023: 101 · 2022: 73 · 2021: 50 · 2020: 3
- FWCI
- 22.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Research Foundation of Korea (NRF-2019R1F1A1058058)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Keywords
- text classification, bidirectional LSTM, convolutional neural networks, attention mechanism, Word2Vec, IMDB movie reviews, sentiment analysis, feature extraction, long short-term memory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
