A Comprehensive Survey on Word Representation Models: From Classical to State-of-the-Art Word Representation Language Models
Open accessThe DOI matches its Crossref record; 174 of the 174 references it cites were found in a registry or an index.
Abstract
Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP-related tasks. In the end, this survey briefly discusses the commonly used ML- and DL-based classifiers, evaluation metrics, and the applications of these word embeddings in different NLP tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Doc2vec can be a powerful feature extractor but fails to capture out-of-vocabulary words (Naseem et al., 2021).”
From Active learning-based systematic reviewing using switching classification models: the case of the onset, maintenance, and relapse of depressive disorders (2023) · cites this paper as Naseem et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“• Transformer-based Pre-trained Language Models Transformer [162] has been proven to be more efficient and faster than LSTM or CNN for language modelling, and thus the following advances in this domain will rely on this architecture.”
BackgroundMethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“• Long Short-TermMemory (LSTM): LSTM was presented by Hochreiter and Schmidhuber [51].”
BackgroundLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“directional Encoder Representations from Transformers (BERT) is a direct descendant of GPT: train a huge LM on free text and then fine-tune individual tasks without custom network architectures. BERT [30] is another contextualised word representation LM, where the transformer NN uses parallel attention layers rather t…”
BackgroundInfluentialGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“y used for hierarchical classification in a DL [57]. CNN was built and used for image classification in early days, but over the period, it has shown excellent results for text classification as well [77]. In image classification, an image tensor is convolved with a set of kernels of size �x�. The convolution layers i…”
MethodLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 4×
“Continuous word representation models like Word2Vec , GloVe and FastText [16, 63, 89, 96] etc.”
BackgroundMethodInfluentialQuinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439 · cited 1×
“• Decision Tree (DT) classifier: Decision tree (DT) was presented by Magerman [87] and developed by Quinlan [131].”
MethodRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 1×
“• RoBERTa: RoBERTa: A Robustly Optimized BERT Pre-training Approach was implemented in July 2019 [86]; it is like a light version of BERT, but it has fewer parameters and better performance as it removes the training on the sentence classification task.”
MethodConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 1×
“Qin et al. [128] used pre-trained Word2Vec embeddings and forwarded these word embeddings to CNN. Similarly, Kim [66] utilized pre-trained embeddings of Word2Vec and forwarded to CNN neural network, which increased the classification results.”
MethodRepresentation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 1×
“It is very important because the performance of ML models heavily depends on the representations of the input [10].”
BackgroundDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 1×
“[125] proposed Embedding from Language Models (ELMo), which gives deep contextual word representations.”
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 1×
“final output gate. �is bias vector,�denotes weight matrix and � �denotes input to the memory cell at time �. •GatedRecurrentUnit(GRU): GRU is another type of RNNs which are presented by Chung et al. [25] and Cho et al. [24]. GRU is the simplest form of LSTM architecture. However, it includes two gates and does not con…”
MethodEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · cited 1×
“improved results drastically over the past few years in many areas such as image detection, speech recognition, NLP etc. [76]. Continuous word representation models like Word2Vec , GloVe and FastText [16, 63, 89, 96] etc. have drastically improved the classification results and overcome shortcomings of categorical rep…”
BackgroundLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · cited 1×
“• BART:[80]: For pre-training sequence-to-sequence models, BART added additional noise functions beyond MLM.”
MethodLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“ck ’18, June 03–05, 2018, Woodstock, NY The one of the drawback of RNN is that they are sensitive to gradient vanishing problem and exploding gradient when gradient descent’s error is back propagated [12]. •Long Short-Term Memory (LSTM): LSTM was presented by Hochreiter and Schmidhuber [51]. LSTM was presented to addr…”
BackgroundBioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 2×
“Some of the most famous in the biomedical domain are Sci-BERT [9], BioBERT [79] and BioAL-BERT [109].”
BackgroundThe Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037 · cited 3×
“However, the popularity of word representationmethods are due to two famousmodels,Word2vec [96] and GloVe [89].”
BackgroundMethodThumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · cited 1×
“[120] performed that binary classification task on IMDb dataset and employed unigrams, bigram and POS tags as features.”
MethodRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · cited 1×
“to capture the issue of syntactic dependencies like "but" and "negations" can change the complete meaning of a sentence and long dependencies within a sentence [21]. Sochet et al. [147] proposed a recursive neural network which can capture and model long semantic and sentiment dependencies of words…”
MethodThe random subspace method for constructing decision forests
Ho, T. K. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.709601 · Cited by 6,842
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · cited 1×
“• XLNet: XLNet, also known as Generalized Auto-regressive Pre-training for Language Understanding [169] which proposes a new task to predict the bidirectional context instead of the masked Language task in BERT, it is a permutation language in which we make some permutations of each sentence so the…”
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 1×
“formation and high dimensionality etc. restrict their use for efficient text classification tasks. Later, representation learning methods of learning text representation directly using neural network [27] was adopted, which improved classification results. Word embeddings from continuous word representation models suc…”
MethodNatural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962 · cited 1×
“Word2vec has proved to be useful in many NLP-related tasks [28].”
BackgroundDistributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934 · cited 1×
“A paragraph representationmodel solved this issue learns word vectors and does not reply on parsing [75].”
BackgroundDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 1×
“DistilBERT: [141], a distilled version of BERT, reduces the size of a BERT LM by 40% while retaining 97% of its language understanding proficiency and being 60% quicker.”
BackgroundBag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1607.01759 · Cited by 4,542 · cited 2×
“Continuous word representation models such as Word2vec, GloVe, FastText [16, 63, 89, 96] and so on, have drastically improved the classification results and overcome shortcomings of categorical representations.”
BackgroundMethodOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · cited 1×
“It is more effective to overcome the issues of vanishing gradient [122].”
BackgroundALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · cited 2×
“[72].”
MethodUniversal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 2×
“ai and Sebastian Ruder of the NUI Galway Insight Center, Universal Language Model Fine-tuning (ULMFiT) [52] is basically a method to allow transfer learning and achieve excellent performance for any NLP task, without 20”
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K., & Cohan, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1371 · Cited by 3,218 · cited 2×
“Some of the most famous in the biomedical domain are Sci-BERT [9], BioBERT [79] and BioAL-BERT [109].”
BackgroundTwitter as a Corpus for Sentiment Analysis and Opinion Mining
Pak, A., & Paroubek, P. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/47akcj8foo4d · Cited by 2,704 · cited 1×
“The NB classification algorithms are widely used for information retrieval [129] and many text classification tasks [95, 119].”
MethodA neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
Quinlan, J. R. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80053-6 · Cited by 2,553 · cited 1×
“be, can be understood easily by its visualizations whereas the significant disadvantages of DT classifier are; it is sensitive to a minor change in the data [42] and have a probability of overfitting [130], complex computations in case of a large number of class labels and have difficulties with-out-of sample predicti…”
BackgroundAttention-based LSTM for Aspect-level Sentiment Classification
Wang, Y., Huang, M., zhu, X., & Zhao, L. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1058 · Cited by 2,450 · cited 1×
“Similarly, a study conducted by Wang et al. [166], used pre-trained GloVe embeddings as an input to LSTM with attention model for aspect based classification and Liu et al. [85] employed pre-trained word embeddings Word2Vec for recommending idioms in essay writing.”
MethodZhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264 · cited 1×
“BERT is trained in the dataset of Books Corpus [170] and English Wikipedia text passages.”
MethodZhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06724 · Cited by 2,264 · cited 1×
“BERT is trained in the dataset of Books Corpus [170] and English Wikipedia text passages.”
MethodSpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., & Levy, O. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00300 · Cited by 1,827 · cited 1×
“SpanBERT [62] generalized ERNIE to mask random spans, without indicating to external knowledge.”
BackgroundSpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M., Chen, D., Liu, Y., Weld, D. S., Zettlemoyer, L., & Levy, O. (2019) · arXiv · doi:10.48550/arxiv.1907.10529 · Cited by 1,827 · cited 1×
“SpanBERT [62] generalized ERNIE to mask random spans, without indicating to external knowledge.”
BackgroundGenerating Text with Recurrent Neural Networks
Sutskever, I., Martens, J., & Hinton, G. E. (2011) · International Conference on Machine Learning · Cited by 1,783
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
Zhang, Y., & Wallace, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.03820 · Cited by 1,593 · cited 1×
“Similarly, Wallace [163] applied GloVe, and Word2Vec pre-trained word embedding in deep neural network-based algorithms and enhanced the classification results.”
MethodCross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · cited 1×
“t 1.7 times faster. ALBERT establishes new SOTA results while having fewer parameters compared to BERT. •Other Models: Some of the other recently proposed LMs are a cross-lingual LM Pretraining (XLM) [71] from Facebook enhanced BERT for Cross-lingual LM. Two unsupervised training objectives that only include monolingu…”
ResultInfluential(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322533223 · Cited by 1,566
Sentiment Analysis of Twitter Data
Agarwal, A., Xie, B., Vovsha, I., Rambow, O., & Passonneau, R. J. (2011) · Cited by 1,565
Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition
Scherer, D., Müller, A., & Behnke, S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15825-4_10 · Cited by 1,503 · cited 1×
“f dimensionality, CNN uses pooling layer to reduce the size from one layer to the other one. Different pooling methods have been proposed by researchers to decrease the output without losing features [142]. Max pooling is the most common pooling technique where maximum elements in the pooling window are selected. To f…”
MethodText Classification Algorithms: A Survey
Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., & Brown, D. (2019) · Information · doi:10.3390/info10040150 · Cited by 1,312 · cited 3×
“To make this annotation process easy, researchers initially worked on finding a sign of sentiment and emotion within the content of the text like emoticons and hashtags [69, 155, 165].”
MethodProblems in the Analysis of Survey Data, and a Proposal
Morgan, J. N., & Sonquist, J. A. (1963) · Journal of the American Statistical Association · doi:10.1080/01621459.1963.10500855 · Cited by 1,305 · cited 1×
“ented by Magerman [87] and developed by Quinlan [131]. It is one of the earliest classification models for text and data mining and is employed successfully in different areas for classification task [98]. The main intuition behind this idea was to create tree-based attributes for data points, but the major question i…”
BackgroundTwitter Sentiment Analysis: The Good the Bad and the OMG!
Kouloumpis, E., Wilson, T., & Moore, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14185 · Cited by 1,225 · cited 2×
“ation. Researchers have proposed different methods to handle this kind of issue in a text, but the most useful technique is to convert them to an actual word which is easy for a machine to understand [68, 100]. •ReplacingelongatedcharactersSocialmediausers, sometimesintentionallyuseelongated words in which they purpos…”
MethodUnified Language Model Pre-training for Natural Language Understanding and Generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019) · arXiv · doi:10.48550/arxiv.1905.03197 · Cited by 1,225 · cited 1×
“Other prominent LM includes UniLM: [32], which uses three objective functions : (i) language modelling (LM), (ii) masked language modelling (MLM), and (iii) sequence-tosequence language modelling (seq2seq LM), for pre-training a transformer model.”
MethodReading text in the wild with convolutional neural networks
Jaderberg, M., Simonyan, K., Vedaldi, A., & Zisserman, A. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-015-0823-z · Cited by 1,191 · cited 2×
“of vector t, and� ℎdenotes the hyperbolic tangent function. •Convolutional Neural Networks (CNN): Another famous architecture of DL is CNN which is mostly used for hierarchical classification in a DL [57]. CNN was built and used for image classification in early days, but over the period, it has shown excellent result…”
BackgroundMethodLearning Sentiment-Specific Word Embedding for Twitter Sentiment Classification
Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., & Qin, B. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1146 · Cited by 1,181 · cited 1×
“entation models improves performance. To integrate the sentiment information into word embeddings, researchers have proposed different hybrid word representations by changing existing skip-gram model [160]. Tang et al. [159] proposed several hybrid ranking models (HyRank) and developed sentiment embeddings based on C&…”
Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts
dos Santos, C., & de Bayser, M. G. (2014) · International Conference on Computational Linguistics · Cited by 1,162
Joint sentiment/topic model for sentiment analysis
Lin, C., & He, Y. (2009) · Proceedings of the 18th ACM conference on Information and knowledge management · doi:10.1145/1645953.1646003 · Cited by 961
Chen, W., Xie, X., Wang, J., Pradhan, B., Hong, H., Bui, D. T., Duan, Z., & Ma, J. (2017) · CATENA · doi:10.1016/j.catena.2016.11.032 · Cited by 930 · cited 1×
“traditional ML algorithms of text classification such as Logistic Regression, which is used in many data mining areas [23, 123], Naive Bayes, which is computationally not expensive and works well with less amount of memory [73], K-nearest Neighbor, which is non-parametric methods, and Support Vector Machine is a famou…”
MethodMegatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., & Catanzaro, B. (2019) · arXiv · doi:10.48550/arxiv.1909.08053 · Cited by 893 · cited 1×
“MegatronLM [145], a scaled-up transform-based model, 24 times larger than BERT, training multi-billion parameter LMs using model parallelism.”
MethodEffective Use of Word Order for Text Categorization with Convolutional Neural Networks
Johnson, R., & Zhang, T. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1011 · Cited by 864 · cited 1×
“For text, it can be a vast number which makes dimensions very high for text classification [61].”
BackgroundBERTweet: A pre-trained language model for English Tweets
Nguyen, D. Q., Vu, T., & Tuan Nguyen, A. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.2 · Cited by 843
A Survey of Text Classification Algorithms
Aggarwal, C. C., & Zhai, C. (2012) · Mining Text Data · doi:10.1007/978-1-4614-3223-4_6 · Cited by 842 · cited 1×
“Text representation is divided into main two parts: i) Text pre-processing and, ii) Features Extraction and then classify the learned representations using an appropriate classifier [3, 69].”
MethodLarge-Scale Bayesian Logistic Regression for Text Categorization
Genkin, A., Lewis, D. D., & Madigan, D. (2007) · Technometrics · doi:10.1198/004017007000000245 · Cited by 808 · cited 1×
“etc. •Logistic Regression (LR) classifier: Logistic regression (LR) is a statistical model and is one of the earliest techniques used for classification. LR predicts probabilities rather than classes [35, 39] or existence of an event like a win/lose or healthy/sick etc. This can be expanded to model many classes of ev…”
MethodPart-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Gimpel, K., Schneider, N., O'Connor, B., Das, D., Mills, D. P., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473609.v1 · Cited by 800
ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
Sun, Y., Wang, S., Li, Y., Feng, S., Tian, H., Wu, H., & Wang, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6428 · Cited by 766
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Speer, R., Chin, J., & Havasi, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03975 · Cited by 758 · cited 1×
“Some of the models presented are proposed in the following studies which inject external knowledge in existing word embedding and improves the results [36, 99, 118, 143, 151].”
BackgroundData Clustering: Algorithms and Applications
(2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315373515 · Cited by 756 · cited 1×
“Over the years text has been used in various applications such as email filtering [20], Irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mining [109,…”
MethodCTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · cited 1×
“CRTL [64], a Conditional Transformer Language Model for Controllable Generation, is a 1.”
BackgroundMASS: Masked Sequence to Sequence Pre-training for Language Generation
Song, K., Tan, X., Qin, T., Lu, J., & Liu, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.02450 · Cited by 733 · cited 1×
“MASS [149] is another recently proposed LM.”
ERNIE: Enhanced Representation through Knowledge Integration
Sun, Y., Wang, S., Li, Y., Feng, S., Chen, X., Zhang, H., Tian, X., Zhu, D., Tian, H., & Wu, H. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09223 · Cited by 721 · cited 1×
“Another recently proposed model, ERNIE [152], Enhanced representation through knowledge integration, used knowledge masking techniques including entity-level masking and phrase-level masking instead of randomly masking tokens.”
MethodA Survey of Text Mining Techniques and Applications
Gupta, V., & Lehal, G. S. (2009) · Journal of Emerging Technologies in Web Intelligence · doi:10.4304/jetwi.1.1.60-76 · Cited by 699 · cited 1×
“Over the years text has been used in various applications such as email filtering [20], Irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mining [109,…”
MethodTwitter Sentiment Analysis with Deep Convolutional Neural Networks
Severyn, A., & Moschitti, A. (2015) · Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/2766462.2767830 · Cited by 659 · cited 1×
“t classification tasks. Tang et al. [158] proposed sentiment specific word representation model, which are achieved from emoticonslabelledtweetmessageswiththehelpoftheneuralnetwork.SeverynandMoschitti[144] presented another neural network-based model where they used Word2Vec to learn embedding. Tweets are presented as…”
MethodThe impact of preprocessing on text classification
Uysal, A. K., & Gunal, S. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2013.08.006 · Cited by 657 · cited 2×
“[161] to analyze the role of pre-processing on two different languages for sentiment classification was presented.”
BackgroundRecurrent Neural Networks for Prediction: Learning Algorithms,Architectures and Stability
Mandic, D. P., & Chambers, J. A. (2001) · ORCA Online Research @Cardiff (Cardiff University) · Cited by 640
The Role of Text Pre-processing in Sentiment Analysis
Haddi, E., Liu, X., & Shi, Y. (2013) · Procedia Computer Science · doi:10.1016/j.procs.2013.05.005 · Cited by 619 · cited 2×
“[45] also explored the effect of text pre-processing on movie review dataset.”
BackgroundStatistical decision-tree models for parsing
Magerman, D. M. (1995) · Proceedings of the 33rd annual meeting on Association for Computational Linguistics - · doi:10.3115/981658.981695 · Cited by 567 · cited 2×
“• Decision Tree (DT) classifier : Decision tree (DT) was presented by Magerman [87] and developed by Quinlan [131].”
Methodcontext2vec: Learning Generic Context Embedding with Bidirectional LSTM
Melamud, O., Goldberger, J., & Dagan, I. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1006 · Cited by 513
Sentiment analysis of blogs by combining lexical knowledge with text classification
Melville, P., Gryc, W., & Lawrence, R. D. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557156 · Cited by 503 · cited 1×
“The NB classification algorithms are widely used for information retrieval [129] and many text classification tasks [95, 119].”
MethodDeep Convolution Neural Networks for Twitter Sentiment Analysis
Jianqiang, Z., Xiaolin, G., & Xuejun, Z. (2018) · IEEE Access · doi:10.1109/access.2017.2776930 · Cited by 482
NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets
Mohammad, S. M., Kiritchenko, S., & Zhu, X. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.6242 · Cited by 451 · cited 1×
“Detection and Replacement of elongated words have been studied by [97] and [5].”
BackgroundLocate the Hate: Detecting Tweets against Blacks
Kwok, I., & Wang, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8539 · Cited by 446 · cited 1×
“tures. For classification, they used SVM, Maximum entropy and NB classifiers in their study and found out that best results were achieved with unigrams as feature and SVM as classifier. Kwok and Wang [70] used n-grams features along with NB classifier tweets classification. These legacy based word representation metho…”
MethodSenticNet 5: Discovering Conceptual Primitives for Sentiment Analysis by Means of Context Embeddings
Cambria, E., Poria, S., Hazarika, D., & Kwok, K. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11559 · Cited by 411 · cited 1×
“e resource. Rezaeinia et al. [138] proposed improved word vectors (IWV) by combining word embeddings, part of speech (POS) and combination of lexicons for sentiment analysis. Recently, Cambria et al. [19] proposed context embeddings for sentiment analysis by conceptual primitives from text and linked with commonsense…”
On Using Twitter to Monitor Political Sentiment and Predict Election Results
Bermingham, A., & Smeaton, A. F. (2011) · Arrow@dit (Dublin Institute of Technology) · Cited by 392
Comparison Research on Text Pre-processing Methods on Twitter Sentiment Analysis
Jianqiang, Z., & Xiaolin, G. (2017) · IEEE Access · doi:10.1109/access.2017.2672677 · Cited by 389 · cited 1×
“Similarly, Jianqiang and Xiaolin [59] studied the role of six different pre-processing techniques on five datasets in their study, where they used four different classifiers.”
BackgroundEnriching Word Vectors with Subword Information
Bojanowski, P., Grave, É., Joulin, A., & Mikolov, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.04606 · Cited by 384 · cited 1×
“improved results drastically over the past few years in many areas such as image detection, speech recognition, NLP etc. [76]. Continuous word representation models like Word2Vec , GloVe and FastText [16, 63, 89, 96] etc. have drastically improved the classification results and overcome shortcomings of categorical rep…”
BackgroundText Mining: The state of the art and the challenges
Tan, A. (1999) · Cited by 359
COVIDSenti: A Large-Scale Benchmark Twitter Data Set for COVID-19 Sentiment Analysis
Naseem, U., Razzak, I., Khushi, M., Eklund, P. W., & Kim, J. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3051189 · Cited by 343 · cited 1×
“Besides, understanding semantics, syntactical information and context is important for text analysis [104, 111].”
BackgroundSelf-citationBoosting Trees for Anti-Spam Email Filtering
Carreras, X., & Màrquez, L. (2001) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0109015 · Cited by 342 · cited 1×
“opinion and sentiments of users about a product, restaurant and movie reviews etc. to predict the sentiment of users. Over the years text has been used in various applications such as email filtering [20], Irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115],…”
MethodSentiment Embeddings with Applications to Sentiment Analysis
Tang, D., Wei, F., Qin, B., Yang, N., Liu, T., & Zhou, M. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2489653 · Cited by 335
LIBLINEAR: A Library for Large Linear Classification
FanRong-En, ChangKai-Wei, HsiehCho-Jui, WangXiang-Rui, & LinChih-Jen (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1442794 · Cited by 328 · cited 1×
“LR predicts probabilities rather than classes [35, 39] or existence of an event like a win/lose or healthy/sick etc.”
BackgroundFast and Accurate Sentiment Classification Using an Enhanced Naive Bayes Model
Narayanan, V., Arora, I., & Bhatia, A. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-41278-3_24 · Cited by 317 · cited 1×
“s it does not help to capture and classify accurately so handling a negation can be a challenging task in the case of word-level text analysis.Replacingnegationwordswiththeprefix’NEG_’hasbeenstudiedby[103].Similarly, handling negations with antonym has been studied by [124]. 2.2 Related work on text pre-processing met…”
BackgroundInfluentialEfficient phrase-based document indexing for Web document clustering
Hammouda, K. M., & Kamel, M. S. (2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.58 · Cited by 316
Natural Language Processing (Almost) from Scratch
CollobertRonan, WestonJason, BottouLéon, KarlenMichael, KavukcuogluKoray, & KuksaPavel (2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078186 · Cited by 306 · cited 1×
“Word2vec has proved to be useful in many NLP-related tasks [28].”
BackgroundHarnessing Twitter "Big Data" for Automatic Emotion Identification
Wang, W., Chen, L., Thirunarayan, K., & Sheth, A. P. (2012) · 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing · doi:10.1109/socialcom-passat.2012.119 · Cited by 303 · cited 1×
“To make this annotation process easy, researchers initially worked on finding a sign of sentiment and emotion within the content of the text like emoticons and hashtags [69, 155, 165].”
MethodTOM: Twitter opinion mining framework using hybrid classification scheme
Khan, F. H., Bashir, S., & Qamar, U. (2014) · Decision Support Systems · doi:10.1016/j.dss.2013.09.004 · Cited by 299
Transformer based Deep Intelligent Contextual Embedding for Twitter sentiment analysis
Naseem, U., Razzak, I., Musial, K., & Imran, M. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.06.050 · Cited by 296 · cited 1×
“Over the years text has been used in various applications such as email filtering [20], Irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mining [109,…”
MethodSelf-citationSensEmbed: Learning Sense Embeddings for Word and Relational Similarity
Iacobacci, I., Pilehvar, M. T., & Navigli, R. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1010 · Cited by 267 · cited 1×
“n different studies [31, 48, 127] But still these models does not capture the polysemy issues. This issue of words with different meanings (polysemy) is addressed in different models presented by the [55, 116]. In recent days, researchers has presented more robust models to handle OOV words and polysemy issues [83, 91…”
BackgroundAutomated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 263 · cited 1×
“These legacy-based word representation methods such as n-grams, BoW, TF, and TFIDF have been widely used in different studies for various text classification tasks [29, 43, 68, 84].”
MethodOpinion Mining and Information Fusion: A survey
Balazs, J. A., & Velásquez, J. D. (2016) · Information Fusion · doi:10.1016/j.inffus.2015.06.002 · Cited by 253 · cited 1×
“In tokenization, a sequence to text is divided into the words, symbols, phrases or tokens [6].”
BackgroundText Classification and Classifiers:A Survey
Korde, V. (2012) · International Journal of Artificial Intelligence & Applications · doi:10.5121/ijaia.2012.3208 · Cited by 249 · cited 1×
“These methods utilise the extensive amounts of training input data to achieve the high quality of semantically rich text representations which can be given as input to different ML methods which can make better predictions [67, 69].”
MethodRetrofitting Word Vectors to Semantic LexiconsPreprint
Faruqui, M., Dodge, J., Jauhar, S. K., Dyer, C., Hovy, E. H., & Smith, N. A. (2014) · arXiv (Cornell University) · doi:10.1184/r1/6473660 · Cited by 244
Python Text Processing with NLTK 2.0 Cookbook
Perkins, J. (2010) · CERN Document Server (European Organization for Nuclear Research) · Cited by 231
Symeonidis, S., Effrosynidis, D., & Arampatzis, A. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.06.022 · Cited by 229 · cited 1×
“[156] presented the comparative analysis of different techniques on two datasets for Twitter sentiment analysis.”
MethodEvaluation Datasets for Twitter Sentiment Analysis: A survey and a new dataset, the STS-Gold.
Saif, H., Fernández, M., He, Y., & Alani, H. (2013) · Open Research Online - ORO (The Open University) · Cited by 206
Effective Use of Word Order for Text Categorization with Convolutional Neural NetworksPreprint
Johnson, R., & Zhang, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1058 · Cited by 195 · cited 1×
“For text, it can be a vast number which makes dimensions very high for text classification [61].”
BackgroundCamacho-Collados, J., Pilehvar, M. T., & Navigli, R. (2016) · Artificial Intelligence · doi:10.1016/j.artint.2016.07.005 · Cited by 188 · cited 1×
“forwarded these word embeddings to CNN. Similarly, Kim [66] utilized pre-trained embeddings of Word2Vec and forwarded to CNN neural network, which increased the classification results. Camacho et al. [18] for concept representation in their work. Jianqiang and Xiolin [60] have initialized word embeddings using pre-tra…”
MethodComparative study of word embedding methods in topic segmentation
Naili, M., Chaibi, A. H., & Ben Ghezala, H. H. (2017) · Procedia Computer Science · doi:10.1016/j.procs.2017.08.009 · Cited by 187 · cited 3×
“The last phase of this model is the correlation between output and every word of the context to adjust representation based on back-propagation [34, 102].”
BackgroundMethodInfluentialStarSpace: Embed All The Things!
Wu, L., Fisch, A., Chopra, S., Adams, K., Bordes, A., & Weston, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11996 · Cited by 187
Naseem, U., Razzak, I., & Eklund, P. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10082-6 · Cited by 182 · cited 1×
“Over the years text has been used in various applications such as email filtering [20], irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mining [109,…”
MethodSelf-citationText Analytics in Social Media
Hu, X., & Liu, H. (2012) · Mining Text Data · doi:10.1007/978-1-4614-3223-4_12 · Cited by 177 · cited 1×
“hether social media, online forums, published articles or clinical notes for patients and online reviews given online where people express their opinions and sentiments to some products or businesses [53]. Text data is a rich source of getting information and gives more opportunity to explore valuable insights which c…”
BackgroundMimicking Word Embeddings using Subword RNNs
Pinter, Y., Guthrie, R., & Eisenstein, J. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1010 · Cited by 177 · cited 1×
“raining and they are assigned UNK token and same vector for all words and degrades results if the number of OOV is large. Different methods to handle OOV words have been proposed in different studies [31, 48, 127] But still these models does not capture the polysemy issues. This issue of words with different meanings…”
BackgroundRefining Word Embeddings Using Intensity Scores for Sentiment Analysis
Yu, L.-C., Wang, J., Lai, K. R., & Zhang, X. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2788182 · Cited by 175
Posterior Distribution of Percentiles: Bayes' Theorem for Sampling From a Population
Hill, B. M. (1968) · Journal of the American Statistical Association · doi:10.1080/01621459.1968.11009286 · Cited by 171
Mrkšić, N., Vulić, I., Séaghdha, D. Ó., Leviant, I., Reichart, R., Gašić, M., Korhonen, A., & Young, S. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00063 · Cited by 170 · cited 1×
“lly different from the data we use in our tasks. Some of the models presented are proposed in the following studies which inject external knowledge in existing word embedding and improves the results [36, 99, 118, 143, 151]. Word embeddings are beneficial in different areas beyond NLP like link prediction, information…”
Role of Text Pre-processing in Twitter Sentiment Analysis
Singh, T., & Kumari, M. (2016) · Procedia Computer Science · doi:10.1016/j.procs.2016.06.095 · Cited by 164
Distant Supervision for Emotion Classification with Discrete Binary Values
Suttles, J., & Ide, N. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37256-8_11 · Cited by 163 · cited 1×
“To make this annotation process easy, researchers initially worked on finding a sign of sentiment and emotion within the content of the text like emoticons and hashtags [69, 155, 165].”
MethodDeep contextualized word representations for detecting sarcasm and irony
Ilić, S., Marrese-Taylor, E., Balazs, J., & Matsuo, Y. (2018) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/w18-6202 · Cited by 163 · cited 1×
“an input to LSTM with attention model for aspect based classification and Liu et al. [85] employed pre-trained word embeddings Word2Vec for recommending idioms in essay writing. Recently, Ilic et al. [56] have used contextual word embeddings (ELMo) for word representation for the detection of sarcasm and irony and sho…”
MethodContext-Sensitive Twitter Sentiment Classification Using Neural Network
Ren, Y., Zhang, Y., Zhang, M., & Ji, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9974 · Cited by 158 · cited 1×
“[135] also used Word2Vec to generate word representations and proposed a new model for the Twitter classification task.”
MethodCovid-twitter-bert: A natural language processing model to analyse covid-19 content on twitter
Müller, M., Salathé, M., & Kummervold, P. E. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7074654 · Cited by 141
Covid-twitter-bert: A natural language processing model to analyse covid-19 content on twitter
Müller, M., Salathé, M., & Kummervold, P. E. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7074653 · Cited by 136
A Preliminary Investigation into Sentiment Analysis of Informal Political Discourse.
Mullen, T., & Malouf, R. (2006) · Cited by 128
Altszyler, E., Ribeiro, S., Sigman, M., & Fernández Slezak, D. (2017) · Consciousness and Cognition · doi:10.1016/j.concog.2017.09.004 · Cited by 128
The Role of Pre-processing in Twitter Sentiment Analysis
Bao, Y., Quan, C., Wang, L., & Ren, F. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-09339-0_62 · Cited by 122 · cited 2×
“[8] study showed the effect of pre-processing techniques on Twitter analysis task.”
BackgroundMethodCombine HowNet lexicon to train phrase recursive autoencoder for sentence-level sentiment analysis
Fu, X., Liu, W., Xu, Y., & Cui, L. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.079 · Cited by 120 · cited 1×
“l results tell that pretraining step enables for a better initialization of the networks’ weights and therefore, has a positive role in classification results. In another study conducted by Fu et al. [38], Word2Vec was employed to get word representation which was forwarded to the recursive encoder as an input 27 Wood…”
MethodExploring Feature Definition and Selection for Sentiment Classifiers
Mejova, Y., & Srinivasan, P. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14163 · Cited by 111 · cited 1×
“antic meaning of those words is still the same. Stemming is the techniques to replace and remove the suffixes and affixes to get the root, base or stem word. The importance of stemming was studied by [92]. There are several types of stemming algorithms which helps to consolidate different forms of words into the same…”
BackgroundSentiment Analysis in Social Media Texts
Balahur, A. (2013) · Cited by 106
Soheily-Khah, S., Marteau, P.-F., & Béchet, N. (2018) · International Conference on Data Intelligence and Security (ICDIS) · doi:10.1109/icdis.2018.00043 · Cited by 105
High-risk learning: acquiring new word vectors from tiny data
Herbelot, A., & Baroni, M. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1030 · Cited by 104 · cited 1×
“raining and they are assigned UNK token and same vector for all words and degrades results if the number of OOV is large. Different methods to handle OOV words have been proposed in different studies [31, 48, 127] But still these models does not capture the polysemy issues. This issue of words with different meanings…”
BackgroundWillett, P. (1988) · Cited by 102
Improving the Accuracy of Pre-trained Word Embeddings for Sentiment AnalysisPreprint
Rezaeinia, S. M., Ghodsi, A., & Rahmani, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08609 · Cited by 100 · cited 1×
“Rezaeinia et al. [138] proposed improved word vectors (IWV) by combining word embeddings, part of speech (POS) and combination of lexicons for sentiment analysis.”
BackgroundDeep Lattice Networks and Partial Monotonic Functions
You, S., Ding, X. D., Canini, K. R., Pfeifer, J., & Gupta, M. R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06680 · Cited by 95 · cited 1×
“Some of the models presented are proposed in the following studies which inject external knowledge in existing word embedding and improves the results [36, 99, 118, 143, 151].”
BackgroundA Joint Segmentation and Classification Framework for Sentence Level Sentiment Classification
Tang, D., Qin, B., Wei, F., Li, D., Liu, T., & Zhou, M. X. (2015) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2015.2449071 · Cited by 94 · cited 1×
“Similarly, several other models are presented, which considers context and sentiment polarity of words for sentiment analysis [81, 137, 157].”
BackgroundStarSpace: Embed All The Things!Preprint
Wu, L., Fisch, A., Chopra, S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.03856 · Cited by 92
An empirical convolutional neural network approach for semantic relation classification
Qin, P., Xu, W., & Guo, J. H. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.12.091 · Cited by 91 · cited 1×
“[128] used pre-trained Word2Vec embeddings and forwarded these word embeddings to CNN.”
MethodStructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding
Wang, W., Bi, B., Yan, M., Wu, C., Bao, Z., Xia, J., Peng, L., & Si, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04577 · Cited by 91 · cited 1×
“Similarly, StructBERT [164] implemented a structural objective word that randomly permits the reconstruction order of 3-grams and a structural objective sentence that predicts the ordering of two consecutive segments.”
MethodLearning context-sensitive word embeddings with neural tensor skip-gram model
Liu, P., Qiu, X., & Huang, X. (2015) · International Conference on Artificial Intelligence · Cited by 85
A Comparative Analysis of Active Learning for Biomedical Text Mining
Naseem, U., Khushi, M., Khan, S. K., Shaukat, K., & Moni, M. A. (2021) · Applied System Innovation · doi:10.3390/asi4010023 · Cited by 82 · cited 1×
“[121], and both models improved the classification results obtained by using BoW features.”
MethodSelf-citationA :) Is Worth a Thousand Words: How People Attach Sentiment to Emoticons and Words in Tweets
Boia, M., Faltings, B. V., Musat, C.-C., & Pu, P. (2013) · International Conference on Social Computing · doi:10.1109/socialcom.2013.54 · Cited by 79 · cited 1×
“xpanded to their original root /base words. For example, words like how is, I’m, can’t and don’t are the contractions for words how is, I am, cannot and do not respectively. In the study conducted by [14], contractions were replaced with their original words or by the relevant word. If contractions are not replaced, t…”
BackgroundPre-processing Boosting Twitter Sentiment Analysis?
Jianqiang, Z. (2015) · IEEE International Conference on Smart City/SocialCom/SustainCom · doi:10.1109/smartcity.2015.158 · Cited by 76 · cited 1×
“The simple and standard method is to remove them [47, 58].”
MethodFrom News to Comment: Resources and Benchmarks for Parsing the Language of Web 2.0
Foster, J., Çetinoğlu, Ö., Wagner, J., Roux, J. L., Nivre, J., Hogan, D., & van Genabith, J. (2011) · Arrow@dit (Dublin Institute of Technology) · Cited by 71
Naseem, U., Musiał, K., Eklund, P., & Prasad, M. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206808 · Cited by 63 · cited 1×
“n [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mining [109, 112] and many more. Fig. 1. Text Classification Pipeline However, being unstructured content, it adds complexity…”
BackgroundSelf-citationComparative study of LSA vs Word2vec embeddings in small corpora: a case study in dreams database
Altszyler, E., Sigman, M., Ribeiro, S., et al. (2016) · El Servicio de Difusión de la Creación Intelectual (National University of La Plata) · doi:10.48550/arxiv.1610.01520 · Cited by 60 · cited 1×
“A predefined length of the window is moved together with the corpus in both models, and the training is done with words inside the window in each step [4].”
MethodText Mining of Stocktwits Data for Predicting Stock Prices
Jaggi, M., Mandal, P., Narang, S., Naseem, U., & Khushi, M. (2021) · Applied System Innovation · doi:10.3390/asi4010013 · Cited by 59
Self-citationHybrid Words Representation for Airlines Sentiment Analysis
Naseem, U., Khan, S. K., Razzak, I., & Hameed, I. A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-35288-2_31 · Cited by 58 · cited 1×
“of users. Over the years text has been used in various applications such as email filtering [20], Irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mi…”
MethodSelf-citationEnhancing Named Entity Recognition in Twitter Messages Using Entity Linking
Yamada, I., Takeda, H., & Takefuji, Y. (2015) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w15-4320 · Cited by 53 · cited 2×
“[140] and Takeda and Takefuiji [168] in the natural language workshop that focuses on noise user-generated text.”
BackgroundPatriche, C. V., Pîrnău, R. G., Grozavu, A., & Roșca, B. (2016) · Pedosphere · doi:10.1016/s1002-0160(15)60047-9 · Cited by 49
Combining N-gram based Similarity Analysis with Sentiment Analysis in Web Content Classification
Liu, S., & Forss, T. (2014) · International Conference on Knowledge Discovery and Information Retrieval · doi:10.5220/0005170305300537 · Cited by 48 · cited 1×
“These legacy based word representation methods such as n-grams, BoW, TF, and TF-IDF have been widely used in different studies for various text classification tasks [29, 43, 68, 84].”
MethodDeep Context-Aware Embedding for Abusive and Hate Speech detection on Twitter.
Naseem, U., Razzak, I., & Hameed, I. A. (2019) · Cited by 46
Self-citationStatistical Decision-Tree Models for ParsingPreprint
Magerman, D. M. (1995) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9504030 · Cited by 44 · cited 2×
“• Decision Tree (DT) classifier : Decision tree (DT) was presented by Magerman [87] and developed by Quinlan [131].”
MethodA Comparative Study of Word Embeddings for Reading ComprehensionPreprint
Dhingra, B., Liu, H., Salakhutdinov, R., & Cohen, W. W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00993 · Cited by 42 · cited 1×
“raining and they are assigned UNK token and same vector for all words and degrades results if the number of OOV is large. Different methods to handle OOV words have been proposed in different studies [31, 48, 127] But still these models does not capture the polysemy issues. This issue of words with different meanings…”
BackgroundTowards Improved Deep Contextual Embedding for the identification of Irony and Sarcasm
Naseem, U., Razzak, I., Eklund, P., & Musiał, K. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207237 · Cited by 42 · cited 1×
“Over the years text has been used in various applications such as email filtering [20], Irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mining [109,…”
MethodSelf-citationLauren, P., Qu, G., Zhang, F., & Lendasse, A. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.117 · Cited by 38
Mrkšić, N., Vulić, I., Séaghdha, D. Ó., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00374 · Cited by 38 · cited 1×
“Some of the models presented are proposed in the following studies that inject external knowledge in existing word embedding and improves the results [36, 99, 118, 143, 151].”
BackgroundAcquiring a Large Scale Polarity Lexicon Through Unsupervised Distributional Methods
Castellucci, G., Croce, D., & Basili, R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19581-0_6 · Cited by 37
Mrkšić, N., Vulić, I., Ó Séaghdha, D., Leviant, I., Reichart, R., Gašić, M., Korhonen, A., & Young, S. J. (2017) · arXiv (Cornell University) · doi:10.17863/cam.10174 · Cited by 37
DICE: Deep Intelligent Contextual Embedding for Twitter Sentiment Analysis
Naseem, U., & Musiał, K. (2019) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2019.00157 · Cited by 36 · cited 1×
“els Woodstock ’18, June 03–05, 2018, Woodstock, NY applying appropriate pre-processing techniques. Besides, understanding semantics, syntactical information and context is important for text analysis [104, 111]. Much research has been dedicated to addressing each of these concerns individually. However, in this survey…”
BackgroundSelf-citationNONPARAMETRIC BAYESIAN ESTIMATION OF PERIODIC LIGHT CURVES
Wang, Y., Khardon, R., & Protopapas, P. (2012) · The Astrophysical Journal · doi:10.1088/0004-637x/756/1/67 · Cited by 33 · cited 1×
“any two features are independent given the output class, which gives bad results [148, 167].”
BackgroundTowards an aggregator that exploits big data to bid on frequency containment reserve market
Giovanelli, C., Liu, X., Sierla, S., Vyatkin, V. V., & Ichise, R. (2017) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2017.8217316 · Cited by 27 · cited 1×
“The advantages of DT classifier are; the amount of hyper-parameters which require tuning is nearly zero, easy to describe, can be understood easily by its visualizations whereas the significant disadvantages of DT classifier are; it is sensitive to a minor change in the data [42] and have a probability of overfitting…”
BackgroundModelling context with neural networks for recommending idioms in essay writing
Liu, Y., Liu, B., Shan, L., & Wang, X. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.11.005 · Cited by 23 · cited 1×
“[85] employed pre-trained word embeddings Word2Vec for recommending idioms in essay writing.”
MethodToward meaningful notions of similarity in NLP embedding models
Elekes, Á., Englhardt, A., Schäler, M., & Böhm, K. (2018) · International Journal on Digital Libraries · doi:10.1007/s00799-018-0237-y · Cited by 22 · cited 2×
“t and by low dimensional vectors. Nowadays, word embedding is being used in many different applications like semantic analysis, philology, psychiatry, cognitive science, social science and psychology [34]. An automatic feature learning technique in which every token in a vocabulary is indexed into an N dimension vecto…”
BackgroundQu, Z., Song, X., Zheng, S., Wang, X., Song, X., & Zu-quan, L. (2018) · IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp.2018.00124 · Cited by 21 · cited 1×
“The NB classification algorithms are widely used for information retrieval [129] and many text classification tasks [95, 119].”
MethodReading Text in the Wild with Convolutional Neural NetworksPreprint
Jaderberg, M., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1842 · Cited by 21 · cited 2×
“of vector t, and� ℎdenotes the hyperbolic tangent function. •Convolutional Neural Networks (CNN): Another famous architecture of DL is CNN which is mostly used for hierarchical classification in a DL [57]. CNN was built and used for image classification in early days, but over the period, it has shown excellent result…”
BackgroundMethodAbusive Language Detection: A Comprehensive Review
Naseem, U., Khan, S. K., Farasat, M., & Ali, F. (2019) · Indian Journal of Science and Technology · doi:10.17485/ijst/2019/v12i45/146538 · Cited by 18 · cited 1×
“Over the years text has been used in various applications such as email filtering [20], irony and sarcasm detection [113] document organization [46], sentiment and opinion mining prediction [107, 115], hate speech detection [105, 106, 110, 114], question answering [44], content mining [2], biomedical text mining [109,…”
MethodSelf-citationBioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity RecognitionPreprint
Naseem, U., Khushi, M., Reddy, V. L., Rajendran, S., Razzak, I., & Kim, J. (2020) · doi:10.21203/rs.3.rs-90025/v1 · Cited by 18
Self-citationDeep AutoEncoder-Decoder Framework for Semantic Segmentation of Brain Tumor.
Rehman, A., Naz, S., Naseem, U., Razzak, I., & Hameed, I. A. (2019) · Cited by 17
Self-citationReuter, J., Pereira-Martins, J., & Kalita, J. K. (2016) · International Journal on Natural Language Computing · doi:10.5121/ijnlc.2016.5402 · Cited by 14
Sentiment Propagation for Predicting Reputation Polarity
Giachanou, A., Gonzalo, J. A., Mele, I., & Crestani, F. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-56608-5_18 · Cited by 12 · cited 1×
“ymbols, phrases or tokens [6]. The prime objective of tokenization is to find out the words in a sentence. Usually, tokenization is applied as a first and standard pre-processing step in any NLP task.[40] •Removal of Noise, URLs, Hashtag and User-mentions Unwanted strings and Unicode are considered as leftover during…”
BackgroundDiabetic Retinopathy Detection Using Multi-layer Neural Networks and Split Attention with Focal Loss
Naseem, U., Khushi, M., Khan, S. K., Waheed, N., Mir, A., Qazi, A., Alshammari, B. M., & Poon, S. K.-S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63836-8_3 · Cited by 12 · cited 1×
“e more training time and utilize much memory. 4.2 Deep learning based classifiers DL based models are motivated by the working of the human brain. It has attained SOTA results in many different areas [108, 134] including NLP. It requires a large number of training data to achieve a semantically good representation of…”
ResultInfluentialSelf-citationToward Tweets Normalization Using Maximum Entropy
Saloot, M. A., Idris, N. B., Shuib, L. M., Raj, R. G., & Aw, A. T. (2015) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/w15-4303 · Cited by 11 · cited 1×
“[140] and Takeda and Takefuiji [168] in the natural language workshop that focuses on noise user-generated text.”
BackgroundAutomatic text categorisation of racist webpages
Greevy, E. (2004) · Dublin City University Open Access Institutional Repository (Dublin City University) · Cited by 9
Soheily-Khah, S., Marteau, P.-F., & Béchet, N. (2017) · HAL · doi:10.1016/j.jisa.nnnn.nn.nnn · Cited by 9 · cited 1×
“the disadvantages are; NB classifiers make a solid makes a reliable hypothesis on the shape of data distribution, i.e. any two features are independent given the output class, which gives bad results [148, 167]. Another limitation of NB classifiers is due to data scarcity. For any value of the feature, we have to appr…”
MethodSegmenting Hashtags using Automatically Created Training Data
Çelebi, A., & Özgür, A. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2prf4qvnqvzq · Cited by 8 · cited 1×
“ll tweets to associate their tweets with some particular topic. The phrase or keyword starting with # is known as hashtags. Various techniques are presented in the literature for word segmentation in [22, 136]. 5 Woodstock ’18, June 03–05, 2018, Woodstock, NY Naseem U, et al. •Replacing Emoticons and Emojis Twitter us…”
MethodHybrid Words Representation for the classification of low quality text
Naseem, U. (2020) · OPUS - Open Publications of UTS Scholars (University of Technology Sydney) · Cited by 8
Self-citationEfficient Non-parametric Estimation of Multiple Embeddings per Word in Vector SpacePreprint
Neelakantan, A., Shankar, J., Passos, A., & McCallum, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.06654 · Cited by 5 · cited 1×
“n different studies [31, 48, 127] But still these models does not capture the polysemy issues. This issue of words with different meanings (polysemy) is addressed in different models presented by the [55, 116]. In recent days, researchers has presented more robust models to handle OOV words and polysemy issues [83, 91…”
BackgroundLearning Semantic Relatedness From Human Feedback Using Metric LearningPreprint
Niebler, T., Becker, M., Pölitz, C., & Andreas Hotho, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07425 · Cited by 4 · cited 1×
“lly different from the data we use in our tasks. Some of the models presented are proposed in the following studies which inject external knowledge in existing word embedding and improves the results [36, 99, 118, 143, 151]. Word embeddings are beneficial in different areas beyond NLP like link prediction, information…”
Method
Research works printed but not linked: 28
- 10.5555/2535015 · doi:10.5555/2535015
- 10.5555/3235571 · doi:10.5555/3235571
- 10.5555/3157382.3157584 · doi:10.5555/3157382.3157584
- Kevin Clark , Minh-Thang Luong , Quoc V. Le , and Christopher D . Manning . 2020 . Electra : Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 (2020). Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 (2020). · arXiv
- 10.5555/3454287.3455457 · doi:10.5555/3454287.3455457
- 10.5555/2002472.2002489 · doi:10.5555/2002472.2002489
- 10.5555/2891460.2891697 · doi:10.5555/2891460.2891697
- 10.5555/1753126.1753129 · doi:10.5555/1753126.1753129
- 10.5555/3044805.3045025 · doi:10.5555/3044805.3045025
- 10.5555/2832415.2832428 · doi:10.5555/2832415.2832428
- 10.5555/559226 · doi:10.5555/559226
- 10.5555/3295222.3295377 · doi:10.5555/3295222.3295377
- 10.5555/3295222.3295377 · doi:10.5555/3295222.3295377
- 10.18653/v1/K16-1006 · DOI · Paper page
- 10.5555/2999792.2999959 · doi:10.5555/2999792.2999959
- 10.5555/3042817.3043083 · doi:10.5555/3042817.3043083
- 10.1016/S1002-0160(15)60047-9 · DOI
- Matthew E. Peters , Mark Neumann , Mohit Iyyer , Matt Gardner , Christopher Clark , Kenton Lee , and Luke Zettlemoyer . 2018. Deep contextualized word representations. CoRR abs/1802.05365 ( 2018 ). Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. CoRR abs/1802.05365 (2018). · arXiv
- 10.1023/A:1022643204877 · doi:10.1023/a:1022643204877 · Paper page
- Colin Raffel , Noam Shazeer , Adam Roberts , Katherine Lee , Sharan Narang , Michael Matena , Yanqi Zhou , Wei Li , and Peter J . Liu . 2019 . Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). · arXiv
- 10.5555/3015812.3015844 · doi:10.5555/3015812.3015844
- 10.5555/1886436.1886447 · doi:10.5555/1886436.1886447
- 10.5555/106765.106782 · doi:10.5555/106765.106782
- 10.5555/3104482.3104610 · doi:10.5555/3104482.3104610
- 10.1109/TKDE.2015.2489653 · DOI · Paper page
- 10.5555/3295222.3295349 · doi:10.5555/3295222.3295349
- 10.5555/3297863.3297977 · doi:10.5555/3297863.3297977
- 10.5555/3454287.3454804 · doi:10.5555/3454287.3454804
Cited by
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2023) · arXiv · doi:10.48550/arxiv.2307.06435 · Cited by 1,259
“We refer the readers interested in smaller models to [35], [36], [32].”
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
“We refer the readers interested in smaller models to [35], [36], [32].”
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Information · doi:10.3390/info15090517 · Cited by 732
COVIDSenti: A Large-Scale Benchmark Twitter Data Set for COVID-19 Sentiment Analysis
Naseem, U., Razzak, I., Khushi, M., Eklund, P. W., & Kim, J. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3051189 · Cited by 343
“In addition, we used hybrid models, such as hybrid ranking (HyRank) and Improved Word Vector (IWV), that incorporate sentiment and context of tweets for Twitter sentiment analysis [36].”
MethodSelf-citationEssel, H. B., Vlachopoulos, D., Essuman, A. B., & Amankwa, J. O. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100198 · Cited by 296
“ChatGPT is trained to estimate the probability of a specific succession of words founded on the context of the words that preceded it (Liu et al., 2023; Naseem et al., 2021).”
Background
Metadata
- Authors
- Naseem, U., Razzak, I., Khan, S. K., & Prasad, M.
- Affiliations
- The University of Sydney (AU)
- Deakin University (AU)
- RMIT University (AU)
- University of Technology Sydney (AU)
- Venue
- ACM Transactions on Asian and Low-Resource Language Information Processing
- Published
- 2021-06-30
- Volume
- 20
- Issue
- 5
- Pages
- 1-35
- ISSN
- 2375-4699 (ISSN-L)
- 2375-4702
- OpenAlex
- W3095760691
- Cited by
- 194
- References
- 168
- Citations by year
- 2026: 22 · 2025: 31 · 2024: 37 · 2023: 61 · 2022: 32 · 2021: 10
- FWCI
- 12.26
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Asian and Low-Resource Language Information Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- ResearchOnline at James Cook University (James Cook University) (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- RMIT Research Repository (RMIT University Library) (submittedVersion)
- Deakin Research Online (Deakin University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Figshare (submittedVersion)
- OPUS - Open Publications of UTS Scholars (University of Technology Sydney) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- word representation, word embeddings, text representation, evaluation metrics, low-resource languages
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.