An Enhanced Neural Word Embedding Model for Transfer Learning
Open accessAbstract
Due to the expansion of data generation, more and more natural language processing (NLP) tasks are needing to be solved. For this, word representation plays a vital role. Computation-based word embedding in various high languages is very useful. However, until now, low-resource languages such as Bangla have had very limited resources available in terms of models, toolkits, and datasets. Considering this fact, in this paper, an enhanced BanglaFastText word embedding model is developed using Python and two large pre-trained Bangla models of FastText (Skip-gram and cbow). These pre-trained models were trained on a collected large Bangla corpus (around 20 million points of text data, in which every paragraph of text is considered as a data point). BanglaFastText outperformed Facebook’s FastText by a significant margin. To evaluate and analyze the performance of these pre-trained models, the proposed work accomplished text classification based on three popular textual Bangla datasets, and developed models using various machine learning classical approaches, as well as a deep neural network. The evaluations showed a superior performance over existing word embedding techniques and the Facebook Bangla FastText pre-trained model for Bangla NLP. In addition, the performance in the original work concerning these textual datasets provides excellent results. A Python toolkit is proposed, which is convenient for accessing the models and using the models for word embedding, obtaining semantic relationships word-by-word or sentence-by-sentence; sentence embedding for classical machine learning approaches; and also the unsupervised finetuning of any Bangla linguistic dataset.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pre-trained word embeddings can predict word contexts in an unsupervised manner.
“Pre-trained word embeddings can predict word contexts in an unsupervised manner, assuming that words in close proximity within a sentence have similar meanings (Badri et al. 2022; Kowsher et al. 2022).”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Kowsher et al. 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Hosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302 · see citing works
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936 · see citing works
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · see citing works
Hearst, M. A., Dumais, S. T., Osuna, E., Platt, J., & Scholkopf, B. (1998) · IEEE Intelligent Systems and their Applications · doi:10.1109/5254.708428 · Cited by 7,243 · see citing works
Convolutional neural networks: an overview and application in radiology
Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0639-9 · Cited by 4,682 · see citing works
Neural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401 · see citing works
Software Framework for Topic Modelling with Large Corpora
Rek, R. R. U., & Sojka, P. (2010) · doi:10.13140/2.1.2393.1847 · Cited by 3,766 · see citing works
A Kernel Method for Multi-Labelled Classification
Elisseeff, A., & Weston, J. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0092 · Cited by 1,388 · see citing works
Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification
Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., & Qin, B. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1146 · Cited by 1,181 · see citing works
Parameter selection in SVM with RBF kernel function
Han, S., Qubo, C., & Han, M. (2012) · Cited by 149 · see citing works
BanFakeNews: A Dataset for Detecting Fake News in BanglaPreprint
Hossain, Z., Rahman, A., Islam, M. S., & Kar, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.08789 · Cited by 103 · see citing works
A New Minkowski Distance Based on Induced Aggregation Operators
Merigó, J. M., & Casanovas, M. (2011) · International Journal of Computational Intelligence Systems · doi:10.1080/18756891.2011.9727769 · Cited by 90 · see citing works
Sentiment analysis of Bengali comments with Word2Vec and sentiment information of words
Al-Amin, M., Islam, M. S., & Uzzal, S. D. (2017) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecace.2017.7912903 · Cited by 77 · see citing works
Named Entity Recognition with Word Embeddings and Wikipedia Categories for a Low-Resource Language
Das, A., Ganguly, D., & Garain, U. (2017) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3015467 · Cited by 55 · see citing works
Cross-lingual sentiment classification in low-resource Bengali language
Sazzed, S. (2020) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2020.wnut-1.8 · Cited by 53 · see citing works
LSTM-ANN & BiLSTM-ANN: Hybrid deep learning models for enhanced classification accuracy
Kowsher, M., Tahabilder, A., Sanjid, M. Z. I., Prottasha, N. J., Uddin, M. S., Hossain, M. A., & Jilani, M. A. K. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.10.013 · Cited by 52 · see citing works
Self-citationBengali word embeddings and it's application in solving document classification problem
Ahmad, A., & Amin, M. R. (2016) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccitechn.2016.7860236 · Cited by 38 · see citing works
Performance Analysis of Different Word Embedding Models on Bangla Language
Ritu, Z. S., Nowshin, N., Nahid, M. M. H., & Ismail, S. (2018) · International Conference on Bangla Speech and Language Processing (ICBSLP) · doi:10.1109/icbslp.2018.8554681 · Cited by 27 · see citing works
Detecting Third Umpire Decisions & Automated Scoring System of Cricket
Kowsher, M., Alam, M. A., Uddin, M. J., Ahmed, F., Ullah, M. W., & Islam, M. R. (2019) · International Conference on Computer, Communication, Chemical, Materials and Electronic Engineering (IC4ME2) · doi:10.1109/ic4me247184.2019.9036705 · Cited by 26 · see citing works
Self-citationChowdhury, H. A., Imon, M. A. H., & Islam, M. S. (2018) · doi:10.1109/iccitechn.2018.8631977 · Cited by 24 · see citing works
BenLem (A Bengali Lemmatizer) and Its Role in WSD
Chakrabarty, A., & Garain, U. (2016) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/2835494 · Cited by 23 · see citing works
Data Set For Sentiment Analysis On Bengali News Comments And Its Baseline Evaluation
Ashik, M. A.-U.-Z., Shovon, S., & Haque, S. (2019) · doi:10.1109/icbslp47725.2019.201497 · Cited by 23 · see citing works
Machine Learning and Deep Learning based Information Extraction from Bangla Names
Kowsher, M., Sanjid, M. Z. I., Das, A., Ahmed, M., & Sarker, M. M. H. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.11.024 · Cited by 10 · see citing works
Self-citationVector Representation of Bengali Word Using Various Word Embedding Model
Rafat, A. A. A., Salehin, M., Khan, F. R., Hossain, S. A., & Abujar, S. (2019) · International Conference on System Modeling & Advancement in Research Trends (SMART) · doi:10.1109/smart46866.2019.9117386 · Cited by 7 · see citing works
Authorship Attribution in Bengali Literature Using fastText's Hierarchical Classifier
Chowdhury, H. A., Imon, M. A. H., & Islam, M. S. (2018) · International Conference on Electrical Engineering and Information Communication Technology · doi:10.1109/ceeict.2018.8628109 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Efficient Estimation of Word Representations in Vector Space. arXiv 2013, arXiv:1301.3781. · arXiv
- Baeza-Yates, R.; Ribeiro-Neto, B. Modern Information Retrieval; ACM Press: New York, NY, USA, 1999; Volume 463.
- Collobert, R.; Weston, J.; Bottou, L.; Karlen, M.; Kavukcuoglu, K.; Kuksa, P. Natural Language Processing (almost) from Scratch. arXiv 2011, arXiv:1103.0398. · arXiv
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention Is All You Need. arXiv 2017, arXiv:1706.03762. · arXiv
- Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understand- ing. arXiv 2019, arXiv:1810.04805. · arXiv
- Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G.; Dean, J. Distributed Representations of Words and Phrases and Their Compositionality. arXiv 2013, arXiv:1310.4546. · arXiv
- Kunchukuttan, A.; Kakwani, D.; Golla, S.; Gokul, N.C.; Bhattacharyya, A.; Khapra, M.M.; Kumar, P. AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages. arXiv 2020, arXiv:2005.00085. · arXiv
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2017, arXiv:1412.6980. · arXiv
Cited by
Sorted by citations
Large language models (LLMs): survey, technical frameworks, and future challenges
Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395 · see citing works
Transfer Learning for Sentiment Analysis Using BERT Based Supervised Fine-Tuning
Prottasha, N. J., Sami, A. A., Kowsher, M., Murad, S. A., Bairagi, A. K., Masud, M., & Baz, M. (2022) · Sensors · doi:10.3390/s22114157 · Cited by 268 · see citing works
Self-citationEnd-to-End Transformer-Based Models in Textual-Based NLP
Rahali, A., & Akhloufi, M. A. (2023) · AI · doi:10.3390/ai4010004 · Cited by 187 · see citing works
Bangla-BERT: Transformer-Based Efficient Model for Transfer Learning and Language Understanding
Kowsher, M., Sami, A. A., Prottasha, N. J., Arefin, M. S., Dhar, P. K., & Koshiba, T. (2022) · IEEE Access · doi:10.1109/access.2022.3197662 · Cited by 102 · see citing works
Self-citationUsing deep learning and word embeddings for predicting human agreeableness behavior
Alsini, R. A. I., Naz, A., Khan, H. U., Bukhari, A. A., Daud, A. Y., & Ramzan, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-81506-8 · Cited by 33 · see citing works
Singla, S., priyanshu, P., Thakur, A., Swami, A., Sawarn, U., & Singla, P. (2024) · International Conference on Advanced Computing & Communication Technologies (ICACCTech) · doi:10.1109/icacctech65084.2024.00068 · Cited by 14 · see citing works
An Analytical Review of Preprocessing Techniques in Bengali Natural Language Processing
Chakraborty, S., Das, P., Dipto, S. M., Pramanik, M. A., & Noor, J. (2025) · IEEE Access · doi:10.1109/access.2025.3574234 · Cited by 13 · see citing works
A hybrid model for low-resource language text classification and comparative analysis
Salleh, A., Osman, M. H., Hassan, S., Said, M. Y., Sharif, K. Y., & Wei, K. T. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114068 · Cited by 9 · see citing works
Developing a Large Benchmark Corpus for Urdu Semantic Word Similarity
Muneer, I., Fatima, G., Khan, M. S., Nawab, R. M. A., & Saeed, A. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3566124 · Cited by 9 · see citing works
Do, T., Gurung, B. D. S., Aryal, S., Khanal, A., Chataut, S., Gadhamshetty, V. R., Lushbough, C., & Gnimpiéba, E. Z. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385544 · Cited by 9 · see citing works
Facial Image Generation from Bangla Textual Description using DCGAN and Bangla FastText
Arnob, N. M. K., Rahman, N. N., Mahmud, S., Uddin, M. N., Rahman, R., & Saha, A. K. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.01406134 · Cited by 8 · see citing works
Arabic Paraphrase Generation Using Transformer-Based Approaches
Al-Shameri, N. A., & Al-Khalifa, H. S. (2024) · IEEE Access · doi:10.1109/access.2024.3450931 · Cited by 7 · see citing works
Amer, A. Y. A., Siddiqui, T., & Athamena, B. (2022) · Research Square · doi:10.21203/rs.3.rs-2184218/v1 · Cited by 7 · see citing works
Emotion Classification using Generative Pre-trained Embedding and Machine Learning
Pattun, G., & Kumar, P. (2023) · IEEE International Conference on Machine Learning and Applied Network Technologies (ICMLANT) · doi:10.1109/icmlant59547.2023.10372980 · Cited by 6 · see citing works
Arabic paraphrased parallel synthetic dataset
Al-Shameri, N. A., & Al-Khalifa, H. S. (2024) · Data in Brief · doi:10.1016/j.dib.2024.111004 · Cited by 5 · see citing works
Feature Engineering Trends in Text-Based Affective Computing: Rules to Advance Deep Learning Models
Pattun, G., & Kumar, P. (2025) · International Research Journal of Multidisciplinary Technovation · doi:10.54392/irjmt2528 · Cited by 4 · see citing works
Attention-Oriented Deep Multi-Task Hash Learning
Wang, L., Meng, Z., Dong, F., Yang, X., Xi, X., & Nie, X. (2023) · Electronics · doi:10.3390/electronics12051226 · Cited by 3 · see citing works
Hybrid Optimization Based BERT Model for Drug Detection in NLP
Kruthika, C. G., Devi, N. N. N., Madlur, V. V., Asha, K., & Aruna, T. M. (2023) · IEEE Mysore Sub Section International Conference (MysuruCon) · doi:10.1109/mysurucon59703.2023.10396892 · Cited by 2 · see citing works
Tariquzzaman, M., Kader, M. W., Anam, A. N., Haque, N., Kabir, M., Mahmud, H., & Hasan, M. K. (2023) · Workshop on Bangla Language Processing (BLP) · doi:10.18653/v1/2023.banglalp-1.26 · Cited by 2 · see citing works
Moon, J.-H., Posada-Quintero, H. F., & Chon, K. H. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.05.015 · Cited by 2 · see citing works
Xia, Z., Liang, S., Wu, D., & Lv, S. (2026) · Applied Sciences · doi:10.3390/app16094158 · Cited by 2 · see citing works
Maliha, F., Nil, S. G., Tabassum, S., Rahman, M., Rahman, A., & Habib, M. T. (2026) · Engineering Reports · doi:10.1002/eng2.70853 · Cited by 1 · see citing works
WaveGuard: A flood monitoring and rescue system
Likhan, A. N., Roy, A., Salim, M. S., & Hossain, S. I. (2025) · SoftwareX · doi:10.1016/j.softx.2025.102361 · Cited by 1 · see citing works
Query Resolution of Literature Knowledge Graphs Using Hybrid Document Embeddings
Muhammad, I., Coenen, F., Gamble, C. L., Kearney, A., & Williamson, P. R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21441-7_7
Fine-Tuned XLNet: A Transfer Learning Model for MOOCs Evaluation Using Forum Posts
Purnachary, M., & Adilakshmi, T. (2023) · Cognitive science and technology · doi:10.1007/978-981-99-2746-3_34
Wasid, M., Amer, A. Y. A., Ahmad, S., & Mustafa, J. (2025) · International Journal of Reliability Quality and Safety Engineering · doi:10.1142/s021853932550038x
Analysis of Multimodal Transfer Learning for Leveraging Multiple Data Resources
Sandhu, A. K., & Agrawal, R. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-4319-6_26
- Authors
- Kowsher, M., Sobuj, M. S. I., Shahriar, M. F., Prottasha, N. J., Arefin, M. S., Dhar, P. K., & Koshiba, T.
- Affiliations
- Stevens Institute of Technology (US)
- Hajee Mohammad Danesh Science and Technology University (BD)
- Daffodil International University (BD)
- Chittagong University of Engineering & Technology (BD)
- Waseda University (JP)
- Corresponding author
- Arefin, Mohammad Shamsul
- Koshiba, Takeshi
- ORCID
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2022-03-10
- Volume
- 12
- Issue
- 6
- Pages
- 2848
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W4220850917
- Cited by
- 27
- References
- 29
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 5 · 2023: 9 · 2022: 5
- FWCI
- 2.06
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- low-resource languages, skip-gram, CBOW, semantic similarity, sentence embeddings, unsupervised fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
