Fuzzy Bag-of-Words Model for Document Representation
The DOI matches its Crossref record; 53 of the 56 references it cites were found in a registry or an index.
Abstract
One key issue in text mining and natural language processing is how to effectively represent documents using numerical vectors. One classical model is the Bag-of-Words (BoW). In a BoW-based vector representation of a document, each element denotes the normalized number of occurrence of a basis term in the document. To count the number of occurrence of a basis term, BoW conducts exact word matching, which can be regarded as a hard mapping from words to the basis term. BoW representation suffers from its intrinsic extreme sparsity, high dimensionality, and inability to capture high-level semantic meanings behind text data. To address the aforementioned issues, we propose a new document representation method named fuzzy Bag-of-Words (FBoW) in this paper. FBoW adopts a fuzzy mapping based on semantic correlation among words quantified by cosine similarity measures between word embeddings. Since word semantic matching instead of exact word string matching is used, the FBoW could encode more semantics into the numerical representation. In addition, we propose to use word clusters instead of individual words as basis terms and develop fuzzy Bag-of-WordClusters (FBoWC) models. Three variants under the framework of FBoWC are proposed based on three different similarity measures between word clusters and words, which are named as FBoWCmean, FBoWCmax, and FBoWCmin , respectively. Document representations learned by the proposed FBoW and FBoWC are dense and able to encode high-level sema…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Traditional methods for extracting emotional features use the bag-of-words (BoW) model [58,59].”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [59]
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · cited 4×
“Linear support vector machine (SVM) [48] was applied to the document representations learned by the aforementioned”
MethodInfluential(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“Another is deep model: multilayers neural networks with various forms including recursive [19], [20], recurrent [21], [22], and convolution [23]–[25] neural networks to learn and perform compositions over word embeddings.”
BackgroundDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“In the popular word embeddings word2vec [11], [15], [18], a two-layer neural network language model is designed to learn vector representations for each word.”
BackgroundConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 3×
“Another is deep model: multilayers neural networks with various forms including recursive [19], [20], recurrent [21], [22], and convolution [23]–[25] neural networks to learn and perform compositions over word embeddings.”
BackgroundRepresentation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 1×
“resentation of data and the good representation determines the upper bound of the performance of machine learning algorithms [1].”
BackgroundTerm-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740 · cited 1×
“It should be noted that the basis terms are the high frequency words in a corpus, and the number of basis terms or the dimensionality of BoW vectors is less than the size of vocabulary [7], [8], [9], [10].”
BackgroundRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · cited 2×
“Another is deep model: multilayers neural networks with various forms including recursive [19], [20], recurrent [21], [22], and convolution [23]–[25] neural networks to learn and perform compositions over word embeddings.”
BackgroundOpinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,720
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962 · cited 1×
“Word embeddings use real-valued, dense, and low-dimensional vectors to represent semantics of words [15], [16].”
BackgroundAn introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880 · cited 1×
“3) LSA: Latent Semantic Analysis [47].”
A STATISTICAL INTERPRETATION OF TERM SPECIFICITY AND ITS APPLICATION IN RETRIEVAL
SPARCK JONES, K. (1972) · Journal of Documentation · doi:10.1108/eb026526 · Cited by 4,505 · cited 1×
“words in a corpus, and the number of basis terms or the dimensionality of BoW vectors is less than the size of vocabulary [7]–[10].”
BackgroundA Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601 · cited 1×
“Another is deep model: multilayers neural networks with various forms including recursive [19], [20], recurrent [21], [22] and convolution [23], [24], [25] neural networks to learn and perform compositions over word embeddings.”
BackgroundA Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,479
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736 · cited 1×
“Representation learning of text is the keystone for various text mining tasks including text classification, document clustering and so on [2]–[6].”
BackgroundFrom Word Embeddings To Document Distances
Kusner, M. J., Sun, Y., Kolkin, N., & Weinberger, K. Q. (2015) · PolyPublie (École Polytechnique de Montréal) · Cited by 2,700
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
A Comparison of Document Clustering Techniques
Steinbach, M. S., Karypis, G., & Kumar, V. (2000) · Cited by 2,532
Unsupervised Learning by Probabilistic Latent Semantic Analysis
Hofmann, T. (2001) · Machine Learning · doi:10.1023/a:1007617005950 · Cited by 2,456 · cited 1×
“In topic models including probabilistic latent semantic analysis [13] and latent dirichlet allocation [14], probability distributions are introduced to describe words and the generation process of each word in a document.”
Background(2007) · Cited by 2,018
Handbook of Latent Semantic Analysis
(2007) · Psychology Press eBooks · doi:10.4324/9780203936399 · Cited by 1,554
Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
Socher, R., Pennington, J., Huang, E., Ng, A. Y., & Manning, C. D. (2011) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 1,432
A possibilistic fuzzy c-means clustering algorithm
Pal, N. R., Pal, K., Keller, J. M., & Bezdek, J. C. (2005) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2004.840099 · Cited by 1,279
Semantic Compositionality through Recursive Matrix-Vector Spaces
Socher, R., Huval, B., Manning, C. D., & Ng, A. Y. (2012) · Cited by 1,259
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1176 · Cited by 1,232 · cited 2×
“Another is deep model: multilayers neural networks with various forms including recursive [19], [20], recurrent [21], [22], and convolution [23]–[25] neural networks to learn and perform compositions over word embeddings.”
BackgroundConvolutional Neural Network Architectures for Matching Natural Language Sentences
Hu, B., Lu, Z., Li, H., & Chen, Q. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.03244 · Cited by 1,098 · cited 1×
“Another is deep model: multilayers neural networks with various forms including recursive [19], [20], recurrent [21], [22], and convolution [23]–[25] neural networks to learn and perform compositions over word embeddings.”
BackgroundA Scalable Hierarchical Distributed Language Model
Mnih, A., & Hinton, G. E. (2008) · Cited by 838
Interpreting TF-IDF term weights as making relevance decisions
Wu, H. C., Luk, R. W. P., Wong, K. F., & Kwok, K. L. (2008) · ACM Transactions on Information Systems · doi:10.1145/1361684.1361686 · Cited by 821 · cited 1×
“words in a corpus, and the number of basis terms or the dimensionality of BoW vectors is less than the size of vocabulary [7]–[10].”
BackgroundPractical solutions to the problem of diagonal dominance in kernel document clustering
Greene, D., & Cunningham, P. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143892 · Cited by 616
Supervised and Traditional Term Weighting Methods for Automatic Text Categorization
Lan, M., Tan, C. L., Su, J., & Lu, Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.110 · Cited by 562 · cited 1×
“It should be noted that the basis terms are the high frequency words in a corpus, and the number of basis terms or the dimensionality of BoW vectors is less than the size of vocabulary [7], [8], [9], [10].”
BackgroundLatent Semantic Indexing: A Probabilistic Analysis
Papadimitriou, C. H., Raghavan, P., Tamaki, H., & Vempala, S. (2000) · Journal of Computer and System Sciences · doi:10.1006/jcss.2000.1711 · Cited by 556
Latent aspect rating analysis without aspect keyword supervision
Wang, H., Lu, Y., & Zhai, C. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020505 · Cited by 327 · cited 1×
“This dataset was originally published for sentiment analysis [44], but we used it for categorization.”
MethodDo Multi-Sense Embeddings Improve Natural Language Understanding?
Li, J., & Jurafsky, D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1200 · Cited by 232 · cited 1×
“Although there are some multi-sense word embeddings in the literature, the disambiguation process of each word sense is quite challenging, and therefore hinders the application of multi-sense word embeddings [38], [39], [40].”
BackgroundFault Detection for Fuzzy Systems With Intermittent Measurements
Zhao, Y., Lam, J., & Gao, H. (2009) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2009.2014860 · Cited by 230 · cited 1×
“many fields, including automatic control, clustering, image processing, optimization, and so on [28]–[32].”
BackgroundA generalized k-nearest neighbor rule
Patrick, E. A., & Fischer, F. P. (1970) · Information and Control · doi:10.1016/s0019-9958(70)90081-1 · Cited by 188
Latent semantic indexing (LSI) : TREC-3 report
Dumais, S. (1994) · Text REtrieval Conference · Cited by 164
A Fuzzy Self-Constructing Feature Clustering Algorithm for Text Classification
Jiang, J.-Y., Liou, R.-J., & Lee, S.-J. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.122 · Cited by 162
Cyberbullying Detection Based on Semantic-Enhanced Marginalized Denoising Auto-Encoder
Zhao, R., & Mao, K. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2531682 · Cited by 133
Self-citationAn introduction to fuzzy systems
Dubois, D., & Prade, H. M. (1998) · Clinica Chimica Acta · doi:10.1016/s0009-8981(97)00232-5 · Cited by 132
Dubois, D., & Prade, H. M. (1989) · Elsevier eBooks · doi:10.1016/b978-0-12-685242-4.50005-1 · Cited by 113
Robust Relief-Feature Weighting, Margin Maximization, and Fuzzy Optimization
Deng, Z., Chung, F.-L., & Wang, S. (2010) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2010.2047947 · Cited by 64 · cited 1×
“many fields, including automatic control, clustering, image processing, optimization, and so on [28]–[32].”
BackgroundA Fuzzy Approach for Multitype Relational Data Clustering
Mei, J.-P., & Chen, L. (2011) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2011.2174366 · Cited by 49 · cited 1×
“Fuzzy system has been extensively applied in many fields, including automatic control, clustering, image processing, optimization and so on [28], [29], [30],”
MethodMultilabel Text Categorization Based on Fuzzy Relevance Clustering
Lee, S.-J., & Jiang, J.-Y. (2014) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2013.2294355 · Cited by 41 · cited 1×
“[36] presented a fuzzy-based method named ML-FRC to address multiclass text categorization problems.”
MethodDiscovering Latent Semantics in Web Documents Using Fuzzy Clustering
Chiang, I., Liu, C. C.-H., Tsai, Y., & Kumar, A. (2015) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2015.2403878 · Cited by 40 · cited 2×
“al [34] presented a hierarchical web document clustering method named fuzzy latent semantic clustering (FLSC).”
MethodDual Fuzzy-Possibilistic Coclustering for Categorization of Documents
Tjhi, W.-C., & Chen, L. (2009) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2008.924332 · Cited by 37 · cited 1×
“Since our work focuses on text data, some previous papers incorporating fuzzy logic into text mining problems are reviewed here [33]–[37].”
BackgroundIncremental Evolution of Fuzzy Grammar Fragments to Enhance Instance Matching and Text Mining
Martin, T., Shen, Y. C., & Azvine, B. (2008) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2008.925920 · Cited by 30 · cited 2×
“Since our work focuses on text data, some previous papers incorporating fuzzy logic into text mining problems are reviewed here [33]–[37].”
BackgroundZhao, R., & Mao, K. (2015) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2015.2437316 · Cited by 29 · cited 1×
“Some models including latent semantic analysis (LSA) and random projection (RP) are applied extensively in many text mining applications [12], [41].”
BackgroundSelf-citationTopic-Aware Deep Compositional Models for Sentence Classification
Zhao, R., & Mao, K. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2632521 · Cited by 29 · cited 1×
“disambiguation process of each word sense is quite challenging, and therefore, hinders the application of multisense word embeddings [38]–[40].”
BackgroundSelf-citationFuzzy bag of words for social image description
Li, Y., Liu, W., Huang, Q., & Li, X. (2014) · Multimedia Tools and Applications · doi:10.1007/s11042-014-2138-4 · Cited by 22 · cited 3×
“It should be noted that [29] also proposed a model named FBoW model for com-”
BackgroundInfluentialSense-Aaware Semantic Analysis: A Multi-Prototype Word Representation Model Using Wikipedia
Wu, Z., & Giles, C. L. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9496 · Cited by 19
Supervised Adaptive-Transfer PLSA for Cross-Domain Text Classification
Zhao, R., & Mao, K. (2014) · doi:10.1109/icdmw.2014.163 · Cited by 17 · cited 1×
“sg learning of text is the keystone for various text mining tasks including text classification, document clustering and so on [2], [3], [4], [5], [6].”
BackgroundSelf-citation
Cited by
Internet of Underwater Things and Big Marine Data Analytics—A Comprehensive Survey
Jahanbakht, M., Xiang, W., Hanzo, L., & Rahimi Azghadi, M. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3053118 · Cited by 425
“[222]–[226].”
Itahara, S., Nishio, T., Koda, Y., Morikura, M., & Yamamoto, K. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3070013 · Cited by 338
Itahara, S., Nishio, T., Koda, Y., Morikura, M., & Yamamoto, K. (2020) · arXiv · doi:10.48550/arxiv.2008.06180 · Cited by 338
Internet of underwater things and big marine data analytics – a comprehensive survey
Jahanbakht, M., Xiang, W., Hanzo, L., & Azghadi, M. R. (2021) · ResearchOnline at James Cook University (James Cook University) · Cited by 317
Zhang, S., Yang, Y., Chen, C., Zhang, X., Leng, Q., & Zhao, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121692 · Cited by 251
Metadata
- Authors
- Zhao, R., & Mao, K.
- Affiliations
- Nanyang Technological University (SG)
- ORCID
- Venue
- IEEE Transactions on Fuzzy Systems
- Published
- 2018-04
- Volume
- 26
- Issue
- 2
- Pages
- 794-804
- ISSN
- 1063-6706 (ISSN-L)
- 1941-0034
- OpenAlex
- W2597485909
- Cited by
- 252
- References
- 56
- Citations by year
- 2026: 10 · 2025: 20 · 2024: 21 · 2023: 40 · 2022: 39 · 2021: 47 · 2020: 31 · 2019: 30 · 2018: 11
- FWCI
- 12.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Fuzzy Systems (publishedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Keywords
- document representation, fuzzy mappings, word embeddings, cosine similarity, word clusters, document classification, high-level semantics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.