Unsupervised Feature Selection for Outlier Detection by Modelling Hierarchical Value-Feature Couplings
Open accessAbstract
Proper feature selection for unsupervised outlier detection can improve detection performance but is very challenging due to complex feature interactions, the mixture of relevant features with noisy/redundant features in imbalanced data, and the unavailability of class labels. Little work has been done on this challenge. This paper proposes a novel Coupled Unsupervised Feature Selection framework (CUFS for short) to filter out noisy or redundant features for subsequent outlier detection in categorical data. CUFS quantifies the outlierness (or relevance) of features by learning and integrating both the feature value couplings and feature couplings. Such value-to-feature couplings capture intrinsic data characteristics and distinguish relevant features from those noisy/redundant features. CUFS is further instantiated into a parameter-free Dense Subgraph-based Feature Selection method, called DSFS. We prove that DSFS retains a 2-approximation feature subset to the optimal subset. Extensive evaluation results on 15 real-world data sets show that DSFS obtains an average 48% feature reduction rate, and enables three different types of pattern-based outlier detection methods to achieve substantially better AUC improvements and/or perform orders of magnitude faster than on the original feature set. Compared to its feature selection contender, on average, all three DSFS-based detectors achieve more than 20% AUC improvement.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Data Mining: Practical Machine Learning Tools and Techniques
Witten, I. H., Frank, E., & Hall, M. A. (2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,493 · see citing works
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,614 · see citing works
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,940 · see citing works
Toward integrating feature selection algorithms for classification and clustering
Liu, H., & Yu, L. (2005) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2005.66 · Cited by 2,721 · see citing works
Self-citationA Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems
Hand, D. J., & Till, R. J. (2001) · Machine Learning · doi:10.1023/a:1010920819831 · Cited by 2,276 · see citing works
Outlier detection for high dimensional data
Aggarwal, C. C., & Yu, P. S. (2001) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/375663.375668 · Cited by 1,066 · see citing works
Feature Selection: A Data PerspectivePreprint
Li, J., Cheng, K., Wang, S., Morstatter, F., Treviño, R. P., Tang, J., & Liu, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.07996 · Cited by 891 · see citing works
Self-citationA survey on unsupervised outlier detection in high‐dimensional numerical data
Zimek, A., Schubert, E., & Kriegel, H. (2012) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.11161 · Cited by 864 · cited 2× · see citing works
“Consequently, they have high computational cost and they are also ineffective for handling data sets with many irrelevant/noisy features due to the curse of dimensionality [1], [2], [20].”
BackgroundOn the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study
Campos, G. O., Zimek, A., Sander, J., Campello, R. J. G. B., Micenková, B., Schubert, E., Assent, I., & Houle, M. E. (2016) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0444-8 · Cited by 744 · see citing works
Chakrabarti, D., & Faloutsos, C. (2006) · ACM Computing Surveys · doi:10.1145/1132952.1132954 · Cited by 606 · see citing works
Greedy Approximation Algorithms for Finding Dense Components in a Graph
Charikar, M. (2000) · Lecture notes in computer science · doi:10.1007/3-540-44436-x_10 · Cited by 493 · see citing works
Outlier detection for high dimensional data
Aggarwal, C. C., & Yu, P. S. (2001) · ACM SIGMOD Record · doi:10.1145/376284.375668 · Cited by 443 · see citing works
Feature selection for high-dimensional class-imbalanced data sets using Support Vector Machines
Maldonado, S., Weber, R. W., & Famili, F. A. (2014) · Information Sciences · doi:10.1016/j.ins.2014.07.015 · Cited by 320 · see citing works
Khuller, S., & Saha, B. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02927-1_50 · Cited by 248 · see citing works
Unsupervised Feature Selection with Adaptive Structure Learning
Du, L., & Shen, Y.-D. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783345 · Cited by 232 · see citing works
Chen, X., & Wasikowski, M. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1401910 · Cited by 218 · see citing works
FP-outlier: Frequent pattern based outlier detection
He, Z., Xu, X., Huang, Z., & Deng, S. (2005) · Computer Science and Information Systems · doi:10.2298/csis0501103h · Cited by 188 · see citing works
Detecting anomalous records in categorical datasets
Das, K., & Schneider, J. (2007) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1281192.1281219 · Cited by 182 · see citing works
Coupled Behavior Analysis with Applications
Cao, L., Ou, Y., & Yu, P. S. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.129 · Cited by 150 · see citing works
Self-citationInformation-Theoretic Outlier Detection for Large-Scale Categorical Data
Wu, S., & Wang, S. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.261 · Cited by 150 · see citing works
Coupling learning of complex interactions
Cao, L. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2014.08.007 · Cited by 136 · see citing works
Self-citationFast and reliable anomaly detection in categorical data
Akoglu, L., Tong, H., Vreeken, J., & Faloutsos, C. (2012) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2396761.2396816 · Cited by 127 · see citing works
Towards Scalable and Accurate Online Feature Selection for Big Data
Yu, K., Wu, X., Ding, W., & Pei, J. (2014) · Proceedings · doi:10.1109/icdm.2014.63 · Cited by 105 · see citing works
Non-IIDness Learning in Behavioral and Social Data
Cao, L. (2013) · The Computer Journal · doi:10.1093/comjnl/bxt084 · Cited by 101 · see citing works
Self-citationCoupled Attribute Similarity Learning on Categorical Data
Wang, C., Dong, X., Zhou, F., Cao, L., & Chi, C. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2325872 · Cited by 70 · see citing works
Self-citationThe Odd One Out: Identifying and Characterising Anomalies
Smets, K., & Vreeken, J. (2011) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972818.69 · Cited by 69 · see citing works
LeSiNN: Detecting Anomalies by Identifying Least Similar Nearest Neighbours
Pang, G., Ting, K. M., & Albrecht, D. W. (2015) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2015.62 · Cited by 63 · see citing works
Self-citationOutlier detection in complex categorical data by modelling the feature value couplings
Pang, G., Cao, L., & Chen, L. (2016) · OPUS - Open Publications of UTS Scholars (University of Technology Sydney) · Cited by 54 · see citing works
Self-citationGPU-Accelerated Feature Selection for Outlier Detection Using the Local Kernel Density Ratio
Azmandian, F., Yılmazer, A., Dy, J., Aslam, J. A., & Kaeli, D. R. (2012) · Proceedings · doi:10.1109/icdm.2012.51 · Cited by 40 · see citing works
Mining multidimensional contextual outliers from categorical relational data
Tang, G., Pei, J., Bailey, J., & Dong, G. (2015) · Intelligent Data Analysis · doi:10.3233/ida-150764 · Cited by 36 · see citing works
Coselection of features and instances for unsupervised rare category analysis
He, J., & Carbonell, J. (2010) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.10091 · Cited by 18 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [8][8] S. Wu and S. Wang, 'Information-theoretic outlier detection for largescale categorical data,' IEEE Transactions on Knowledge and Data Engineering , vol. 25, no. 3, pp. 589-602, 2013. -
- [12][12] X. Chen and M. Wasikowski, 'FAST: A ROC-based feature selection metric for small samples and imbalanced data classification problems,' in SIGKDD , 2008, pp. 124-132. -
Cited by
Sorted by citations
A Survey on Explainable Anomaly Detection
Li, Z., Zhu, Y., & Van Leeuwen, M. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3609333 · Cited by 139 · see citing works
“[158] propose two ilter-based unsupervised selection methods, namely CBRW_FS and CBRW, which select a subset of features independently from subsequent anomaly detection methods.”
BackgroundMethodToward Explainable Deep Anomaly Detection
Pang, G., & Aggarwal, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3470794 · Cited by 23 · see citing works
Self-citationIsolation-based feature Selection for Unsupervised Outlier Detection
Yang, Q., Singh, J., & Lee, J. (2019) · Annual Conference of the PHM Society · doi:10.36001/phmconf.2019.v11i1.824 · Cited by 19 · see citing works
- Authors
- Pang, G., Cao, L., Chen, L., & Liu, H.
- Affiliations
- University of Technology Sydney (AU)
- Arizona State University (US)
- Venue
- 2016 IEEE 16th International Conference on Data Mining (ICDM)
- Publisher
- IEEE
- Published
- 2016-12
- Volume
- 2016
- Pages
- 410-419
- ISSN
- 1550-4786 (ISSN-L)
- OpenAlex
- W2583608529
- Cited by
- 43
- References
- 31
- Citations by year
- 2026: 1 · 2025: 3 · 2023: 4 · 2022: 3 · 2021: 10 · 2020: 4 · 2019: 4 · 2018: 10 · 2017: 4
- FWCI
- 3.23
- Type
- conference paper
- Language
- en
- Open access
- Licence
- other-oa
- Locations
- Proceedings (publishedVersion)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, other-oa)
- UTS ePRESS (University of Technology Sydney) (submittedVersion)
- IEEE Conference Proceedings
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- unsupervised feature selection, outlier detection, categorical data, feature redundancy, noisy features, dense subgraphs, approximation guarantees, class imbalance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
