A Comprehensive Survey on Word Representation Models: From Classical to State-of-the-Art Word Representation Language Models
Open accessThe DOI matches its Crossref record; 174 of the 174 references it cites were found in a registry or an index.
Abstract
Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP-related tasks. In the end, this survey briefly discusses the commonly used ML- and DL-based classifiers, evaluation metrics, and the applications of these word embeddings in different NLP tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Doc2vec can be a powerful feature extractor but fails to capture out-of-vocabulary words (Naseem et al., 2021).”
From Active learning-based systematic reviewing using switching classification models: the case of the onset, maintenance, and relapse of depressive disorders (2023) · cites this paper as Naseem et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“• Transformer-based Pre-trained Language Models Transformer [162] has been proven to be more efficient and faster than LSTM or CNN for language modelling, and thus the following advances in this domain will rely on this architecture.”
BackgroundMethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“• Long Short-TermMemory (LSTM): LSTM was presented by Hochreiter and Schmidhuber [51].”
BackgroundLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“directional Encoder Representations from Transformers (BERT) is a direct descendant of GPT: train a huge LM on free text and then fine-tune individual tasks without custom network architectures. BERT [30] is another contextualised word representation LM, where the transformer NN uses parallel attention layers rather t…”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“y used for hierarchical classification in a DL [57]. CNN was built and used for image classification in early days, but over the period, it has shown excellent results for text classification as well [77]. In image classification, an image tensor is convolved with a set of kernels of size �x�. The convolution layers i…”
MethodLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 4×
“Continuous word representation models like Word2Vec , GloVe and FastText [16, 63, 89, 96] etc.”
BackgroundMethodInfluentialQuinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439 · cited 1×
“• Decision Tree (DT) classifier: Decision tree (DT) was presented by Magerman [87] and developed by Quinlan [131].”
MethodRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 1×
“• RoBERTa: RoBERTa: A Robustly Optimized BERT Pre-training Approach was implemented in July 2019 [86]; it is like a light version of BERT, but it has fewer parameters and better performance as it removes the training on the sentence classification task.”
MethodConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 1×
“Qin et al. [128] used pre-trained Word2Vec embeddings and forwarded these word embeddings to CNN. Similarly, Kim [66] utilized pre-trained embeddings of Word2Vec and forwarded to CNN neural network, which increased the classification results.”
Method
Research works printed but not linked: 28
- 10.5555/2535015 · doi:10.5555/2535015
- 10.5555/3235571 · doi:10.5555/3235571
- 10.5555/3157382.3157584 · doi:10.5555/3157382.3157584
- Kevin Clark , Minh-Thang Luong , Quoc V. Le , and Christopher D . Manning . 2020 . Electra : Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 (2020). Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 (2020). · arXiv
- 10.5555/3454287.3455457 · doi:10.5555/3454287.3455457
- 10.5555/2002472.2002489 · doi:10.5555/2002472.2002489
- 10.5555/2891460.2891697 · doi:10.5555/2891460.2891697
- 10.5555/1753126.1753129 · doi:10.5555/1753126.1753129
- 10.5555/3044805.3045025 · doi:10.5555/3044805.3045025
- 10.5555/2832415.2832428 · doi:10.5555/2832415.2832428
- 10.5555/559226 · doi:10.5555/559226
- 10.5555/3295222.3295377 · doi:10.5555/3295222.3295377
- 10.5555/3295222.3295377 · doi:10.5555/3295222.3295377
- 10.18653/v1/K16-1006 · DOI · Paper page
- 10.5555/2999792.2999959 · doi:10.5555/2999792.2999959
- 10.5555/3042817.3043083 · doi:10.5555/3042817.3043083
- 10.1016/S1002-0160(15)60047-9 · DOI
- Matthew E. Peters , Mark Neumann , Mohit Iyyer , Matt Gardner , Christopher Clark , Kenton Lee , and Luke Zettlemoyer . 2018. Deep contextualized word representations. CoRR abs/1802.05365 ( 2018 ). Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. CoRR abs/1802.05365 (2018). · arXiv
- 10.1023/A:1022643204877 · doi:10.1023/a:1022643204877 · Paper page
- Colin Raffel , Noam Shazeer , Adam Roberts , Katherine Lee , Sharan Narang , Michael Matena , Yanqi Zhou , Wei Li , and Peter J . Liu . 2019 . Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). · arXiv
- 10.5555/3015812.3015844 · doi:10.5555/3015812.3015844
- 10.5555/1886436.1886447 · doi:10.5555/1886436.1886447
- 10.5555/106765.106782 · doi:10.5555/106765.106782
- 10.5555/3104482.3104610 · doi:10.5555/3104482.3104610
- 10.1109/TKDE.2015.2489653 · DOI · Paper page
- 10.5555/3295222.3295349 · doi:10.5555/3295222.3295349
- 10.5555/3297863.3297977 · doi:10.5555/3297863.3297977
- 10.5555/3454287.3454804 · doi:10.5555/3454287.3454804
Cited by
Items 101–195 of 195
Using Semantic Similarity to Proxy for Knowledge Structures in Accounting Research
Janvrin, D. J., Raschke, R. L., Schneider, G. P., & Hsieh, S.-F. (2025) · Journal of Information Systems · doi:10.2308/isys-2023-010 · Cited by 4
Rauniyar, K., Shiwakoti, S., Poudel, S., Thapa, S., Naseem, U., & Nasim, M. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.4 · Cited by 4
“Through groundbreaking advancements in NLP, machines are now able to comprehend human language with unprecedented accuracy, unlocking a new realm of possibilities for data analysis and knowledge extraction (Naseem et al., 2021).”
BackgroundSelf-citationXtremeLLMs: Towards Extremely Large Language ModelsPreprint
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Preprints.org · doi:10.20944/preprints202408.1483.v1 · Cited by 3
Unsupervised Insider Threat Detection Using Multi-Head Self-Attention Mechanisms
Bertrand, S., Germain, P., & Tawbi, N. (2024) · Intelligent Cybersecurity Conference (ICSC) · doi:10.1109/icsc63108.2024.10895186 · Cited by 4
“Even though such techniques reduce manual pre-processing requirements, contextual information from the resulting feature vectors is limited [16].”
Wu, H., Li, D., Chen, Z., Tang, X., & Wang, G. (2025) · Trees Forests and People · doi:10.1016/j.tfp.2025.101068 · Cited by 3
A Comparative Study of Word Embedding Techniques in Natural Language Processing
Andrabi, S. A. B., & Wahid, A. (2022) · Advances in intelligent systems and computing · doi:10.1007/978-981-16-9573-5_50 · Cited by 2
Heuristic Masking for Text Representation Pretraining
Zhuang, Y. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095445 · Cited by 2
“The masked language model (MLM) pretraining [1] is a self-supervised method widely applied to natural language processing tasks and has been proved as a state-of-the-art representation learning technique [2, 3].”
MethodFerraria, M. A., Ferraria, V. A., & de Castro, L. N. (2023) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2023.08.006 · Cited by 3
“• SMS Spam Data : a dataset collected by [62] containing 5,572 messages (4,825 ham and 747 spam).”
BackgroundDetection of Online Hate Comments Based on Feature Embedding and Deep Learning
Hamzah, S., Mohd, M., & Zakaria, L. Q. (2023) · International Conference on Artificial Intelligence and Data Sciences (AiDAS) · doi:10.1109/aidas60501.2023.10284606 · Cited by 2
“[10].”
Dusart, A., Pinel-Sauvagnat, K., & Hubert, G. (2023) · Online Social Networks and Media · doi:10.1016/j.osnem.2023.100272 · Cited by 2
Assessing Text Representation Methods on Tag Prediction Task for StackOverflow
Skenderi, E., Laaksonen, S.-M., Stefanidis, K., & Huhtamäki, J. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.075 · Cited by 3
“Nevertheless, their proposed architecture has become popular in many NLP applications since it is more efficient than previous models used for language modeling (Naseem et al., 2020).”
Adopting Pre-trained Large Language Models for Regional Language Tasks: A Case Study
Gaikwad, H. R., Kiwelekar, A. W., Laddha, M. D., & Shahare, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53827-8_2 · Cited by 2
Linguistic Harmony in Diversity: Lemmatizing Rojak Malay for Global Communication
Yong, J. Z. H., Koh, J. Y., Liew, J. X., & Tan, C. W. (2024) · International Conference on Digital Transformation and Applications (ICDXA) · doi:10.1109/icdxa61007.2024.10470819 · Cited by 2
“Various lemmatization methods, including WordNet, TextBlob, spaCy, TreeTagger, Pattern, Gensim, and Stanford CoreNLP, have been developed to handle different languages and contexts [11], [12].”
MethodSilva, F. T. D., de Araújo, F. R., & Bezerra, E. C. (2024) · International Conference on Artificial Intelligence and Computer Engineering (ICAICE) · doi:10.1109/icaice63571.2024.10864116 · Cited by 2
Uslu, N. A., & Onan, A. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13482-z · Cited by 8
“The term "dft" represents the sum of documents containing the term "t" (Naseem et al., 2021).”
Exploring text representation impact on K-means based arabic text documents clustering
Rahhali, D., Khaider, Y., & En-Nahnahi, N. (2022) · International Conference on Intelligent Systems and Computer Vision (ISCV) · doi:10.1109/iscv54655.2022.9806067 · Cited by 2
Artificial Text Detection with Multiple Training Strategies
Li, B., Weng, Y., Song, Q., & Deng, H. (2022) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2022-20-375-381 · Cited by 7
“It is noted that the pre-trained model has a solid ability to differentiate the results generated by different models (Qiu et al., 2020; Naseem et al., 2021), we resort to the state-of-the-art (SOTA) pre-trained language model for better prediction.”
MethodKaur, G., Kaur, A., & Khurana, M. (2023) · International Conference on Data Analytics for Business and Industry (ICDABI) · doi:10.1109/icdabi60145.2023.10629356 · Cited by 2
“The most frequently used algorithms for this task are Porter Snowball, Lancaster, WordNet Lemmatizer, Stanford CoreNLP [17], SpaCy, TextBlob,and NLTK [18] [19] [20].”
MethodGupta, A., Joshi, R. M., Tambe, S., Laban, R. M., Tandlekar, G., & Chitre, V. (2022) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet54531.2022.9824562 · Cited by 2
Unveiling Deception: Cutting-Edge Approaches for Fraud Detection in Financial StatementsPreprint
Chimonaki, C., Papadakis, S., & Lemonakis, C. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4516701 · Cited by 2
Improving misspelled word solving for human trafficking detection in online advertising data
Wiriyakun, C., & Kurutach, W. (2023) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v13i6.pp6558-6567 · Cited by 1
UNDERSTANDING LARGE LANGUAGE MODELS: THE FUTURE OF ARTIFICIAL INTELLIGENCE
Yurchak, I., Khich, A. O., & Оksentyuk, V. (2024) · Computer Design Systems Theory and Practice · doi:10.23939/cds2024.02.051 · Cited by 1
Meng, D., Phuntsho, T., & Gonsalves, T. (2024) · IEEE Access · doi:10.1109/access.2024.3510388 · Cited by 1
“The use of pre-trained language models (PLMs) like Bidirectional Encoder Representations from Transformers (BERT) [8] and their variants [9], [10] have significantly advanced the field of HMC task.”
Shah-Mohammadi, F., & Finkelstein, J. (2024) · IEEE Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS) · doi:10.1109/ecbios61468.2024.10885505 · Cited by 2
“Word embeddings are widely utilized across a variety of NLP tasks, including topic modeling, text classification, sentiment analysis and named entity recognition [11-13].”
Bias Detection in AI Models Using Word Embedding Association Tests (WEAT)
Kamaluddin, K. (2024) · International Journal of Emerging Research in Engineering and Technology · doi:10.63282/3050-922x.ijeret-v5i4p109 · Cited by 1
Amri, M., El-Hassani, F. Z., & Haddouch, K. (2025) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-025-10550-x · Cited by 1
“Word representation models are diverse, ranging from classical models to representation learning models [22].”
InfluentialSemantic Search for Occupation-Aligned Course Recommendation Using SOTA Embedding Models
Khumwichai, P., Praserttitipong, D., Unankard, S., & Pikulkaew, K. (2025) · International Computer Science and Engineering Conference · doi:10.1109/icsec67360.2025.11297992 · Cited by 1
“Traditional models such as TF-IDF [9], Word2Vec [10], and Latent Semantic Analysis (LSA) [11] are computationally efficient but often lack a deep understanding of semantics.”
Benchmarking BoW, TF-IDF and Bert-Based Sentiment Analysis in Low-Resource Minang Language
Permana, I., Fajri, H., Alhadi, Z., Hendriyani, Y., Yuliarti, & Akmal, A. D. (2025) · International Conference on Informatics and Computing (ICIC) · doi:10.1109/icic68054.2025.11309385 · Cited by 1
“Text classification is a fundamental task in natural language processing (NLP), involving a sequence of stages: data acquisition, preprocessing, feature extraction, and classification [9].”
Impact analysis of feature selection techniques on cyberstalking detection
Arvind, K. G., & Bansal, A. (2022) · i-manager’s Journal on Image Processing · doi:10.26634/jip.9.4.19138 · Cited by 1
Li, R. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-2092-1_43 · Cited by 1
Example Applications Beyond Node Classification
Paulheim, H., Ristoski, P., & Portisch, J. (2023) · Synthesis lectures on data, semantics and knowledge · doi:10.1007/978-3-031-30387-6_7 · Cited by 1
Embedding Mental Health Discourse for Community Recommendation
Dang, H., Nguyen, B., Ziems, N., & Jiang, M. (2023) · Workshop on Computational Approaches to Discourse (CODI) · doi:10.18653/v1/2023.codi-1.22 · Cited by 1
Recommendation System for Clinical Concept Mapping
Pal, S., Gaur, M., Chaudhuri, R., Anto, O. B., Kalaivanan, R., Chetan, K. V., & Pradhan, P. (2023) · International Conference on Computational Intelligence, Networks and Security (ICCINS) · doi:10.1109/iccins58907.2023.10450023 · Cited by 2
“Another interesting comprehensive study was presented in [12] where different word representation models and their power of expression were explored.”
BackgroundINCEPT: A Framework for Duplicate Posts Classification with Combined Text Representations
Skenderi, E., Huhtamäki, J., Laaksonen, S.-M., & Stefanidis, K. (2024) · ACM Transactions on the Web · doi:10.1145/3677322 · Cited by 3
“Text representation methods can be divided into two broad categories: classical methods , which generate representations based on statistical and probabilistic measurements of a text, and transformer-based methods , which can capture the semantics and context of an underlying text [26].”
InfluentialText Mining and Fault Risk Identification in Power Transformers Based on Deep Semantic Learning
Lei, S., He, J., Zhao, Z., & Wu, L. (2024) · International Conference on Energy, Power and Electrical Technology (ICEPET) · doi:10.1109/icepet61938.2024.10627201 · Cited by 1
“In the field of text mining, CNN and RNN are two mainstream architectures[15,16].”
BackgroundAddressing Semantic Variability in Clinical Outcome Reporting Using Large Language Models
Shah-Mohammadi, F., & Finkelstein, J. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4040116 · Cited by 4
“Answer: Yes, “mortality” and “death” are semantically the same.”
Chen, S., Naseem, U., & Razzak, I. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.13 · Cited by 4
Self-citationLejeune, A., Robaglia, B.-M., Walter, M., Berrouiguet, S., & Lemey, C. (2022) · doi:10.2196/preprints.36986
The Curriculum of Civil Engineering From The View Of Students. A Case Study
Paz, H. R., & Abdala, N. C. (2022) · IEEE World Engineering Education Conference (EDUNINE) · doi:10.1109/edunine53672.2022.9782359
Zhu, J., Viaud, G., & Hudelot, C. (2022) · Advances in intelligent systems and computing · doi:10.1007/978-981-19-6153-3_7 · Cited by 1
Dehghani, M., & Faili, H. (2023) · Research Square · doi:10.21203/rs.3.rs-3415317/v1
Transferência de estilo textual arbitrário em português
da Costa, P. B., & Paraboni, I. (2023) · Linguamática · doi:10.21814/lm.15.2.410
Ayorinde, A. O., Panneerselvam, J., & Liu, L. (2023) · IEEE Smart World Congress (SWC) · doi:10.1109/swc57546.2023.10449135
“When using pre-trained contextualised word embeddings, such as ELMo[3], BERT[8], XLNet [17], RoBERTa [9], embedding a long document for improving classification performance remains an issue.”
MethodComparing hybrid models for recognising objects in thermal images at nighttime
Bandi, M., & Reeja, S. R. (2024) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v34.i3.pp1823-1831
“…as micro-objects [7] Create the image-wise annotated framework for saliency recognition of objects [8] Detect complex composite objects [ Vehicle detection [10] mAP: 77.4 Recognize vehicles in foggy weather [11] multiple and mixed style LP recognition [12] Driver activity recognition [13] Urban”
BackgroundInfluentialChatParse: A New Application for Persian Bourse Chatbot
Shahedi, A., & Seyedin, S. (2023) · Sharq. · doi:10.61186/itrc.15.2.49 · Cited by 1
“The basic representation of sequence is One-Hot Encoding [4].”
BackgroundMethodInfluentialAkhila, V. H., & Chacko, A. M. (2024) · ACM Symposium on Applied Computing · doi:10.1145/3605098.3636038 · Cited by 2
“BERT can generate embeddings for OOV words that are specific to their context, which can help to improve the accuracy of NLP models [28], [22].”
MethodNeural Network Algorithms in the Construction of Intelligent Financial Sharing Platform
Fu, Y. (2024) · International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE) · doi:10.1109/icdcece60827.2024.10549176
“It converts financial text data into numerical feature representations through a word bag model, and uses a pre trained word vector model to transform the text into a distributed representation of word vectors, capturing semantic relationships between words [14-15].”
MethodAn embedded diachronic sense change model with a case study from ancient Greek
Zafar, S., & Nicholls, G. K. (2024) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2024.108011 · Cited by 3
“Recent surveys discussing modern embedding models, including many variants of the ones mentioned here, are given by Naseem et al. (2021) and Apidianaki (2022).”
BackgroundConSentiment: Análise de Sentimentos baseada em Aspectos a partir de Consenso de Modelos em Grafos
Marcon, G. D. S. (2024) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.55.2024.tde-17062024-085235
Improving the Understanding of Arabic Political Text Through Emotion Analysis and Text Mining
Nassar, A., Tzoreff, M., Kuflik, T., & Alon, I. (2026) · Journal on Computing and Cultural Heritage · doi:10.1145/3815767
Júnior, V. O. D. S., Oliveira, M., Cruz, L. A., & da Silva, T. L. C. (2024) · Natural language processing. · doi:10.1017/nlp.2024.45 · Cited by 1
Construction of Knowledge Graph in Clothing Commodity Field Based on BERT-Lattice-Att-CRF Model
Quan, X., Hou, R., Zhao, X., Song, L., & Liu, Y. (2024) · IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) · doi:10.1109/itnec60942.2024.10732986
“…the intrinsic correlation of all words in the entire sentence, strengthen the semantic representation ability of the model, and solve the problem of poor recognition of ambiguous words by the word2vec word vector model, this article introduces the BERT layer at the head of the Lattice-Att model.”
MethodLeveraging Text Classification to Identify Anti-Patterns in SQL QueriesPreprint
Chatterjee, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4990322
Li, X., Vaghi, E., Pasi, G., Coulson, N., Simoni, A. D., Viviani, M., & AD HOC Group (2025) · Journal of Medical Internet Research · doi:10.2196/56038 · Cited by 2
“BERT is a contextualized word representation pretrained language model [29,30].”
Amalia, N. W. F., & Setiawan, E. B. (2023) · International Conference on Artificial Intelligence, Robotics, Signal and Image Processing (AIRoSIP) · doi:10.1109/airosip58759.2023.10873879 · Cited by 1
“The training uses statistics on the global incidence of terms from the retrieved corpus [20]–[22].”
An Analysis of Different Text Representation Schemes for an Immune Clustering Algorithm
Ferraria, M. A., Balbi, P. P., & de Castro, L. N. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-82073-1_25
Personalized Federated Learning by Model Pruning via Batch Normalization Layers
Wang, A., & Gong, B. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-2681-6_7
Advantages & Impact of Fine Tuning Large Language Models for Ecommerce Search
Samantapudi, R. K. R. (2025) · Journal of Information Systems Engineering & Management · doi:10.52783/jisem.v10i45s.8898 · Cited by 7
“Domain-specific search implementation of large language models (LLMs) needs practitioners to understand both model structure and the operational needs of target sectors (Naseem et al., 2021).”
Architectural Gaps in Generative AI: Quantifying Cognitive Risks for Safety Applications
Wen, H., & Hu, P. (2025) · AI · doi:10.3390/ai6070138 · Cited by 2
Methods to Evaluate and Improve the Quality of Text Summarization
Nguyen, H. (2025) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc2443390 · Cited by 1
Attention-based Binary Question Answering Using Hybrid of Bi-LSTM and Bi-GRU
Mohammed, N. F., & Ali, I. H. (2025) · Baghdad Science Journal · doi:10.21123/2411-7986.5005
Term-Unigram Extractions Using Embedding-Based Filtering
Kalykulova, A., & Alzhanov, A. (2025) · IEEE International Conference on Smart Information Systems and Technologies (SIST) · doi:10.1109/sist61657.2025.11139174
“Moreover, static embeddings suffer from the Out-of-Vocabulary (OOV) problem, where the model encounters new or rare words that were not present in the training sample, and such words do not get representation, making them difficult to process [20, 21].”
Transformative NLP in healthcare through clinical text analysis with spacy
Vedadri, Y. B., Chilupuri, H., Papaiahgari, G., Mathur, J., Lakshmikanth, M., Soujanya, K. L. S., & Husain, S. O. (2025) · AIP conference proceedings · doi:10.1063/5.0298371
Петровна, Б. И. (2025) · Vestnik NSUEM · doi:10.34020/2073-6495-2025-4-142-157
The Role of Computational Parallelism in Artificial Intelligence Learning Models
Sousa, C., Couto, P., Rocha, G. A., Quintas, J. M., & Lopes, S. (2026) · Springer eBooks · doi:10.1007/978-3-031-99147-9_20
A Comparative Study of Bug Triage Classification Approaches from FastText to LLMs
Bezerra, E. C., Dantas, V. D. S., Diniz, P. H. B., & de Souza, D. F. (2025) · International Conference on Artificial Intelligence and Computer Engineering (ICAICE) · doi:10.1109/icaice68195.2025.11382491
“This process is essential for accurate classification [31].”
Value creation and destruction in telehealth services: the role of technology affordances
Quach, S., Thaichon, P., Zhang, Y., Nguyen, T. T., & Nguyen, Q. V. H. (2026) · Marketing Intelligence & Planning · doi:10.1108/mip-02-2025-0098
Analyzing Customer Reviews of XYZ Travel Using Sentiment Analysis
Abdullah, N., Mohamed, H., & Isa, N. B. (2026) · International Conference on Cognitive Systems and Computer Interaction (ICoSCI) · doi:10.1109/icosci66700.2026.11447494
Zhang, A., Zhang, Y., Xie, H., & Zhang, X. (2026) · Discover Applied Sciences · doi:10.1007/s42452-026-08614-0
Analysis and control of online interactions through neural natural language processing
Laugier, L. (2022) · theses.fr (ABES) · doi:10.70675/4743aa15z209az4addz8f29z95150aa230b8 · Cited by 2
Machine Learning Approaches for Sub-surface Geological Heterogeneous Sources
Arman, M. (2023) · theses.fr (ABES) · doi:10.70675/42bb8288za220z4273za3f7zb042954f2cac
Learning Recruitment-Related Representations from Graphs and Sequential Data
Zhu, J. (2023) · theses.fr (ABES) · doi:10.70675/45fbe640z43b8z454bz902azb8501bcf111d · Cited by 2
Emerging Trend Detection in News Articles
Nguyen, N. K. (2023) · theses.fr (ABES) · doi:10.70675/7a92cf20z7488z4551zb81dz255fc375df3b · Cited by 1
Experiential recommendation system for document access and browsing
Khoury, T. E. (2024) · theses.fr (ABES) · doi:10.70675/1752d10az8ec0z4eeaz96b4z582a166e3765
Machine Learning of Emotional Expressions In the Wild from Acoustic Signals and Text
Samir, S. A. (2023) · theses.fr (ABES) · doi:10.70675/d64c4bf2z99b1z4e18zbc76z736e91c6d559
Guermazi, Y. (2023) · theses.fr (ABES) · doi:10.70675/d45bebabzebcdz43b7z9a3fzce9af03766bd
Rudiyono, E., & Miranda, E. (2026) · International Seminar on Intelligent Business and Edge-Computing Research (ISIBER) · doi:10.1109/isiber68248.2026.11470193
Salinas, L. I. F., Castillo, E., Morales, M. A. R., & Hernández, V. A. S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6598804
A Novel Semantic Embedding Using Complex Mapping
Farzi, P., & Farzi, S. (2026) · Neural Processing Letters · doi:10.1007/s11063-026-11842-5
User Story Estimation using Natural Language Processing and Deep Learning: A Comparative Study
Silva, D. D. O., Souza, A. C. C., Souza, F. C. M., & Sanches, S. R. R. (2026) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2026.5860
Unsupervised Graph Learning For Insider Threat Detection
Bertrand, S., Germain, P., & Tawbi, N. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-27993-4_2
Meta-User2Vec: Recommendations Based on Embeddings for Users and Products Using Meta-data
Wassenberg, D., Frăsincar, F., & Robal, T. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7251-9_3
Patel, F. N., & Shah, H. B. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-95-7350-9_20
Lee, S., Hong, S., Shin, J., & Yi, M. Y. (2026) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-026-02913-4
Examine neural network models for protein sequencing based features prediction
Senapati, B., & Das, R. (2026) · Journal of Computer-Aided Molecular Design · doi:10.1007/s10822-026-00921-3
“Numbers should be eliminated as unnecessary characters from the token set [34].”
Alsayed, A. S., Dam, H. K., & Nguyen, C. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7381502
An Intelligent English Grammar Error Correction Method Based on the BERT Pre-trained Language Model
Li, D. (2026) · International Conference on Machine Intelligence and Digital Applications (MIDA) · doi:10.1145/3801438.3806610
“To overcome the above limitations, a promising direction is to exploit contextualized word representations learned from large-scale unlabeled corpora, so that deep semantic and syntactic information can be embedded into downstream correction models [13].”
Word Counting Is Not Enough: A Syntax-Driven Computational Method for Analyzing Participation
Raveau, M. P., Goñi, J., Amigo, P., & Fuentes-Bravo, C. (2026) · Computation · doi:10.3390/computation14090209
An Interpretable and Optimized Attention-Driven LSTM Framework for Email Sentiment Classification
Pradhan, B., Panda, A. R., Rautaray, S. S., & Giri, S. K. (2026) · F1000Research · doi:10.12688/f1000research.186136.1
Music Recommender System Based on Content-Based Filtering with GloVe
Yazdi, F. M., & Baizal, Z. K. A. (2026) · IEEE International Conference on Artificial Intelligence and Mechatronics Systems (AIMS) · doi:10.1109/aims71270.2026.11712613
“Naseem et al. [11] co-occurrence statistics, which makes it suitable for lyric text.”
InfluentialSustainalyze: An LLM-Driven Approach for Transparent Sustainability Benchmarking
Pionzewski, C., Schreynemackers, P., Hoppe, C., Plenne, A., Verburg, K., Vogel, M., Rademacher, R., Coskun, A., Ponikarov, A., Rutinowski, J., et al. (2026) · Sustainable development goals series · doi:10.1007/978-3-032-13396-0_6
Mankar, P. S., & Manwar, A. B. (2026) · EPJ Web of Conferences · doi:10.1051/epjconf/202639101037
Woldaregay, A. Z., Lund, J. A., Ngo, P. D., Tayefi, M., Burman, J., Hansen, S., Sillesen, M., Dalianis, H., Jenssen, R., Lindsetmo, R. O., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/91880
Metadata
- Authors
- Naseem, U., Razzak, I., Khan, S. K., & Prasad, M.
- Affiliations
- The University of Sydney (AU)
- Deakin University (AU)
- RMIT University (AU)
- University of Technology Sydney (AU)
- Venue
- ACM Transactions on Asian and Low-Resource Language Information Processing
- Published
- 2021-06-30
- Volume
- 20
- Issue
- 5
- Pages
- 1-35
- ISSN
- 2375-4699 (ISSN-L)
- 2375-4702
- OpenAlex
- W3095760691
- Cited by
- 194
- References
- 168
- Citations by year
- 2026: 22 · 2025: 31 · 2024: 37 · 2023: 61 · 2022: 32 · 2021: 10
- FWCI
- 12.26
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Asian and Low-Resource Language Information Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- ResearchOnline at James Cook University (James Cook University) (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- RMIT Research Repository (RMIT University Library) (submittedVersion)
- Deakin Research Online (Deakin University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Figshare (submittedVersion)
- OPUS - Open Publications of UTS Scholars (University of Technology Sydney) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- word representation, word embeddings, text representation, evaluation metrics, low-resource languages
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.