The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
An Entropy Weighting k-Means Algorithm for Subspace Clustering of High-Dimensional Sparse Data
Abstract
This paper presents a new k-means type algorithm for clustering high-dimensional objects in sub-spaces. In high-dimensional data, clusters of objects often exist in subspaces rather than in the entire space. For example, in text clustering, clusters of documents of different topics are categorized by different subsets of terms or keywords. The keywords for one cluster may not occur in the documents of other clusters. This is a data sparsity problem faced in clustering high-dimensional data. In the new algorithm, we extend the k-means clustering process to calculate a weight for each dimension in each cluster and use the weight values to identify the subsets of important dimensions that categorize different clusters. This is achieved by including the weight entropy in the objective function that is minimized in the k-means clustering process. An additional step is added to the k-means clustering process to automatically compute the weights of all dimensions in each cluster. The experiments on both synthetic and real data have shown that the new algorithm can generate better clustering results than other subspace clustering algorithms. The new algorithm is also scalable to large data sets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Jain, A. K., Murty, M. N., & Flynn, P. J. (1999) · ACM Computing Surveys · doi:10.1145/331499.331504 · Cited by 13,199
A Comparison of Document Clustering Techniques
Steinbach, M. S., Karypis, G., & Kumar, V. (2000) · Cited by 2,532
Automatic subspace clustering of high dimensional data for data mining applications
Agrawal, R., Gehrke, J., Gunopulos, D., & Raghavan, P. (1998) · Proceedings of the 1998 ACM SIGMOD international conference on Management of data · doi:10.1145/276304.276314 · Cited by 2,412
Efficient and Effective Clustering Methods for Spatial Data Mining
Ng, R. T., & Han, J. (1994) · Cited by 1,778
Subspace clustering for high dimensional data: a review
Parsons, L., Haque, E., & Liu, H. (2004) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1007730.1007731 · Cited by 1,346
CLARANS: a method for clustering objects for spatial data mining
Ng, R. T., & Han, J. (2002) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2002.1033770 · Cited by 1,202
Fast algorithms for projected clustering
Aggarwal, C. C., Wolf, J. L., Yu, P. S., Procopiuc, C., & Park, J. S. (1999) · ACM SIGMOD Record · doi:10.1145/304181.304188 · Cited by 900
Locally adaptive dimensionality reduction for indexing large time series databases
Keogh, E., Chakrabarti, K., Pazzani, M., & Mehrotra, S. (2001) · Proceedings of the 2001 ACM SIGMOD international conference on Management of data · doi:10.1145/375663.375680 · Cited by 866
Document clustering using locality preserving indexing
Cai, D., He, X., & Han, J. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.198 · Cited by 806
Entropy-based subspace clustering for mining numerical data
Cheng, C.-H., Fu, A. W., & Zhang, Y. (1999) · Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/312129.312199 · Cited by 543
Finding generalized projected clusters in high dimensional spaces
Aggarwal, C. C., & Yu, P. S. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335383 · Cited by 530
The Challenges of Clustering High Dimensional Data
Steinbach, M., Ertöz, L., & Kumar, V. (2004) · New Directions in Statistical Physics · doi:10.1007/978-3-662-08968-2_16 · Cited by 508
Fast algorithms for projected clustering
Aggarwal, C. C., Wolf, J. L., Yu, P. S., Procopiuc, C., & Park, J. S. (1999) · Proceedings of the 1999 ACM SIGMOD international conference on Management of data · doi:10.1145/304182.304188 · Cited by 484
Clustering Objects on Subsets of Attributes (with Discussion)
Friedman, J. H., & Meulman, J. J. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.02059.x · Cited by 418
Feature Weighting in k-Means Clustering
Modha, D. S., & Spangler, W. S. (2003) · Machine Learning · doi:10.1023/a:1024016609528 · Cited by 382
A Monte Carlo algorithm for fast projective clustering
Procopiuc, C. M., Jones, M., Agarwal, P. K., & Murali, T. M. (2002) · Proceedings of the 2002 ACM SIGMOD international conference on Management of data · doi:10.1145/564691.564739 · Cited by 340
δ-clusters: capturing subspace correlation in a large data set
Yang, J., Wang, W., Wang, H., & Yu, P. (2003) · Proceedings 18th International Conference on Data Engineering · doi:10.1109/icde.2002.994771 · Cited by 300
Finding generalized projected clusters in high dimensional spaces
Aggarwal, C. C., & Yu, P. S. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335383 · Cited by 295
Medical literature as a potential source of new knowledge.
Swanson, D. R. (1990) · Europe PMC (PubMed Central) · Cited by 267
Generative model-based document clustering: a comparative study
Zhong, S., & Ghosh, J. (2005) · Knowledge and Information Systems · doi:10.1007/s10115-004-0194-1 · Cited by 244
An optimization algorithm for clustering using weighted dissimilarity measures
Chan, E. Y., Ching, W. K., Ng, M. K., & Huang, J. Z. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2003.11.003 · Cited by 235
Self-citationLocal Dimensionality Reduction: A New Approach to Indexing High Dimensional Spaces
Chakrabarti, K., & Mehrotra, S. (2000) · Cited by 230
HARP: a practical projected clustering algorithm
Yip, K. Y., Cheung, D. W., & Ng, M. K. (2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.74 · Cited by 160
Self-citationDeSarbo, W. S., Carroll, J. D., Clark, L. A., & Green, P. E. (1984) · Psychometrika · doi:10.1007/bf02294206 · Cited by 153
Subspace Clustering of High Dimensional Data
Domeniconi, C., Papadopoulos, D., Gunopulos, D., & Ma, S. (2004) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972740.58 · Cited by 151
FINDIT: a fast and intelligent subspace clustering algorithm using dimension voting
Woo, K.-G., Lee, J.-H., Kim, M., & Yoon-Joon, L. E. E. (2003) · Information and Software Technology · doi:10.1016/j.infsof.2003.07.003 · Cited by 147
A comparative study of clustering methods
Zaït, M., & Messatfa, H. (1997) · Future Generation Computer Systems · doi:10.1016/s0167-739x(97)00018-6 · Cited by 121
Makarenkov, V., & Legendre, P. (2001) · Journal of Classification · doi:10.1007/s00357-001-0018-x · Cited by 100
A Monte Carlo algorithm for fast projective clustering
Procopiuc, C. M., Jones, M., Agarwal, P., & Murali, T. M. (2002) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/564736.564739 · Cited by 94
A validation study of a variable weighting algorithm for cluster analysis
Milligan, G. W. (1989) · Journal of Classification · doi:10.1007/bf01908588 · Cited by 84
Simultaneous Clustering and Dynamic Keyword Weighting for Text Documents
Frigui, H., & Nasraoui, O. (2004) · Springer eBooks · doi:10.1007/978-1-4757-4305-0_3 · Cited by 61
On Discovery of Extremely Low-Dimensional Clusters Using Semi-Supervised Projected Clustering
Yip, K. Y.-L., Cheung, D. W.-L., & Ng, M. (2005) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2005.96 · Cited by 61
Self-citation
Cited by
The k-means Algorithm: A Comprehensive Survey and Performance Evaluation
Ahmed, M., Seraj, R., & Islam, S. M. S. (2020) · Electronics · doi:10.3390/electronics9081295 · Cited by 1,717
Kriegel, H.-P., Kröger, P., & Zimek, A. (2009) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1497577.1497578 · Cited by 1,093
Fan, C., Xiao, F., & Wang, S. (2014) · Applied Energy · doi:10.1016/j.apenergy.2014.04.016 · Cited by 565
A comprehensive survey of anomaly detection techniques for high dimensional big data
Thudumu, S., Branch, P., Jin, J., & Singh, J. (2020) · Journal of Big Data · doi:10.1186/s40537-020-00320-x · Cited by 425
Feature Selection for Clustering: A Review
Alelyani, S., Tang, J., & Liu, H. (2018) · Data Clustering · doi:10.1201/9781315373515-2 · Cited by 371
Metadata
- Authors
- Jing, L., Ng, M. K., & Huang, J. Z.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2007-08
- Volume
- 19
- Issue
- 8
- Pages
- 1026-1041
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2125070513
- Cited by
- 625
- References
- 32
- FWCI
- 30.34
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.