The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis
Open accessAbstract
Clustering algorithms have emerged as an alternative powerful meta-learning tool to accurately analyze the massive volume of data generated by modern applications. In particular, their main goal is to categorize data into clusters such that objects are grouped in the same cluster when they are similar according to specific metrics. There is a vast body of knowledge in the area of clustering and there has been attempts to analyze and categorize them for a larger number of applications. However, one of the major issues in using clustering algorithms for big data that causes confusion amongst practitioners is the lack of consensus in the definition of their properties as well as a lack of formal categorization. With the intention of alleviating these problems, this paper introduces concepts and algorithms related to clustering, a concise survey of existing (clustering) algorithms as well as providing a comparison, both from a theoretical and an empirical perspective. From a theoretical perspective, we developed a categorizing framework based on the main properties pointed out in previous studies. Empirically, we conducted extensive experiments where we compared the most representative algorithm from each of the categories using a large number of real (big) data sets. The effectiveness of the candidate clustering algorithms is measured through a number of internal and external validity metrics, stability, runtime, and scalability tests. In addition, we highlighted the set of clustering algorithms that are the best performing for big data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Data mining: concepts and techniques
(2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,531
(1967) · Cited by 23,719
(1996) · Cited by 20,179
Finding groups in data: an introduction to cluster analysis
(1991) · Choice Reviews Online · doi:10.5860/choice.28-4558 · Cited by 10,474
Witten, I. H. (2008) · Cited by 9,143
Hubert, L., & Arabie, P. (1985) · Journal of Classification · doi:10.1007/bf01908075 · Cited by 8,219
Algorithms for Clustering Data
Sarle, W. S. (1990) · Technometrics · doi:10.1080/00401706.1990.10484648 · Cited by 7,865
FCM: The fuzzy c-means clustering algorithm
Bezdek, J. C., Ehrlich, R., & Full, W. (1984) · Computers & Geosciences · doi:10.1016/0098-3004(84)90020-7 · Cited by 6,608
Algorithms for clustering data
Flury, B. (1989) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(89)90033-5 · Cited by 6,432
Survey of Clustering Algorithms
Xu, R., & WunschII, D. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.845141 · Cited by 6,197
Ankerst, M. (1999) · doi:10.1145/304181.304187 · Cited by 3,982
BIRCH: an efficient data clustering method for very large databases
Zhang, T., Ramakrishnan, R., & Livny, M. (1996) · Proceedings of the 1996 ACM SIGMOD international conference on Management of data - SIGMOD '96 · doi:10.1145/233269.233324 · Cited by 3,935
Finding Groups in Data: An Introduction to Cluster Analysis.
Sarle, W. S., Kaufman, L., & Rousseeuw, P. J. (1991) · Journal of the American Statistical Association · doi:10.2307/2290430 · Cited by 3,539
A survey on clustering algorithms for wireless sensor networks
Abbasi, A. A., & Younis, M. (2007) · Computer Communications · doi:10.1016/j.comcom.2007.05.024 · Cited by 2,383
A simple and fast algorithm for K-medoids clustering
Park, H.-S., & Jun, C.-H. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.01.039 · Cited by 2,104
BIRCH: an efficient data clustering method for very large databases
Zhang, T., Ramakrishnan, R., & Livny, M. (1996) · ACM SIGMOD Record · doi:10.1145/235968.233324 · Cited by 2,097
Chameleon: hierarchical clustering using dynamic modeling
Karypis, G., Han, E.-H., & Kumar, V. (1999) · Computer · doi:10.1109/2.781637 · Cited by 2,097
Knowledge acquisition via incremental conceptual clustering
Fisher, D. H. (1987) · Machine Learning · doi:10.1007/bf00114265 · Cited by 1,991
Kohonen, T. (1998) · Neurocomputing · doi:10.1016/s0925-2312(98)00030-7 · Cited by 1,968
CURE: an efficient clustering algorithm for large databases
Guha, S., Rastogi, R., & Shim, K. (1998) · Proceedings of the 1998 ACM SIGMOD international conference on Management of data · doi:10.1145/276304.276312 · Cited by 1,907
Efficient and Effective Clustering Methods for Spatial Data Mining
Ng, R. T., & Han, J. (1994) · Cited by 1,778
Rock: A robust clustering algorithm for categorical attributes
Guha, S., Rastogi, R., & Shim, K. (2000) · Information Systems · doi:10.1016/s0306-4379(00)00022-3 · Cited by 1,472
OPTICS: ordering points to identify the clustering structure
Ankerst, M., Breunig, M. M., Kriegel, H.-P., & Sander, J. (1999) · Proceedings of the 1999 ACM SIGMOD international conference on Management of data · doi:10.1145/304182.304187 · Cited by 1,335
Internet traffic classification using bayesian analysis techniques
Moore, A. W., & Zuev, D. (2005) · Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems · doi:10.1145/1064212.1064220 · Cited by 1,208
CLARANS: a method for clustering objects for spatial data mining
Ng, R. T., & Han, J. (2002) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2002.1033770 · Cited by 1,202
An efficient approach to clustering in large multimedia databases with noise
Hinneburg, A., & Keim, D. A. (1998) · KOPS (University of Konstanz) · Cited by 1,181
WaveCluster: A Multi-Resolution Clustering Approach for Very Large Spatial Databases
Sheikholeslami, G., Chatterjee, S., & Zhang, A. (1998) · Cited by 756
Cost-based modeling for fraud and intrusion detection: results from the JAM project
Stolfo, S. J., Fan, W., Lee, W., Prodromidis, A., & Chan, P. K. (2002) · Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00 · doi:10.1109/discex.2000.821515 · Cited by 610
A Survey of Text Clustering Algorithms
Aggarwal, C. C., & Zhai, C. (2012) · Mining Text Data · doi:10.1007/978-1-4614-3223-4_4 · Cited by 581
A SURVEY OF ONTOLOGY EVALUATION TECHNIQUES
Brank, J., & Grobelnik, M. (2005) · Cited by 552
A Fast Clustering Algorithm to Cluster Very Large Categorical Data Sets in Data Mining.
Huang, J. Z. (1997) · Cited by 551
Models of Incremental Concept Formation
Gennari, J. H., Langley, P., & Fisher, D. (1988) · doi:10.21236/ada191597 · Cited by 525
Models of incremental concept formation
Gennari, J. H., Langley, P., & Fisher, D. (1989) · Artificial Intelligence · doi:10.1016/0004-3702(89)90046-5 · Cited by 453
A distribution-based clustering algorithm for mining in large spatial databases
Xu, X., Ester, M., Kriegel, H.-P., & Sander, J. (2002) · Proceedings 14th International Conference on Data Engineering · doi:10.1109/icde.1998.655795 · Cited by 352
Optimal Grid-Clustering: Towards Breaking the Curse of Dimensionality in High-Dimensional Clustering
Hinneburg, A., & Keim, D. A. (1999) · KOPS (University of Konstanz) · Cited by 339
Labelled data collection for anomaly detection in wireless sensor networks
Suthaharan, S., Alzahrani, M., Rajasegarar, S., et al. (2010) · International Conference on Intelligent Sensors, Sensor Networks and Information Processing · doi:10.1109/issnip.2010.5706782 · Cited by 144
An Experimental Comparison of Several Clustering and Initialization MethodsPreprint
Meilă, M., & Heckerman, D. E. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7401 · Cited by 109
Architecture of a network monitor
Moore, A. W., Hall, J., Kreibich, C., Harris, E. P., & Pratt, I. (2003) · Cited by 101
Toward an efficient and scalable feature selection approach for internet traffic classification
Fahad, A., Tari, Z., Khalil, I., Habib, I., & Alnuweiri, H. (2013) · Computer Networks · doi:10.1016/j.comnet.2013.04.005 · Cited by 96
Self-citationSCADAVT-A framework for SCADA security testbed based on virtualization technology
Almalawi, A., Tari, Z., Khalil, I., & Fahad, A. (2013) · Conference on Local Computer Networks · doi:10.1109/lcn.2013.6761301 · Cited by 63
Self-citationPPFSCADA: Privacy preserving framework for SCADA data publishing
Fahad, A., Tari, Z., Almalawi, A., Goscinski, A. M., Khalil, I., & Mahmood, A. N. (2014) · Future Generation Computer Systems · doi:10.1016/j.future.2014.03.002 · Cited by 47
Self-citationAn Efficient Clustering Scheme to Exploit Hierarchical Data in Network Traffic Analysis
Mahmood, A. N., Leckie, C. A., & Parampalli, U. (2008) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190725 · Cited by 44
A Framework for Improving the Accuracy of Unsupervised Intrusion Detection for SCADA Systems
Almalawi, A., Tari, Z., Fahad, A., & Khalil, I. (2013) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom.2013.40 · Cited by 11
Self-citation(1997)
Cited by
Big Data Analytics in Operations Management
Choi, T., Wallace, S. W., & Wang, Y. (2018) · Production and Operations Management · doi:10.1111/poms.12838 · Cited by 1,027
Digital twin paradigm: A systematic literature review
Semeraro, C., Lezoche, M., Panetto, H., & Dassisti, M. (2021) · Computers in Industry · doi:10.1016/j.compind.2021.103469 · Cited by 986
Ezugwu, A. E., Ikotun, A. M., Oyelade, O. O., Abualigah, L., Agushaka, J. O., Eke, C. I., & Akinyelu, A. A. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104743 · Cited by 924
Tsai, C.-W., Lai, C.-F., Chao, H.-C., & Vasilakos, A. V. (2015) · Journal of Big Data · doi:10.1186/s40537-015-0030-3 · Cited by 818
Multi-view clustering: A survey
Yang, Y., & Wang, H. (2018) · Big Data Mining and Analytics · doi:10.26599/bdma.2018.9020003 · Cited by 506
Metadata
- Authors
- Fahad, A., Alshatri, N., Tari, Z., Alamri, A., Khalil, I., Zomaya, A. Y., Foufou, S., & Bouras, A.
- Venue
- IEEE Transactions on Emerging Topics in Computing
- Published
- 2014-09
- Volume
- 2
- Issue
- 3
- Pages
- 267-279
- ISSN
- 2168-6750 (ISSN-L)
- OpenAlex
- W2057923756
- Cited by
- 1,042
- References
- 45
- FWCI
- 57.11
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.