Sparse Modeling-Based Sequential Ensemble Learning for Effective Outlier Detection in High-Dimensional Numeric Data
Open accessAbstract
The large proportion of irrelevant or noisy features in real-life high-dimensional data presents a significant challenge to subspace/feature selection-based high-dimensional outlier detection (a.k.a. outlier scoring) methods. These methods often perform the two dependent tasks: relevant feature subset search and outlier scoring independently, consequently retaining features/subspaces irrelevant to the scoring method and downgrading the detection performance. This paper introduces a novel sequential ensemble-based framework SEMSE and its instance CINFO to address this issue. SEMSE learns the sequential ensembles to mutually refine feature selection and outlier scoring by iterative sparse modeling with outlier scores as the pseudo target feature. CINFO instantiates SEMSE by using three successive recurrent components to build such sequential ensembles. Given outlier scores output by an existing outlier scoring method on a feature subset, CINFO first defines a Cantelli's inequality-based outlier thresholding function to select outlier candidates with a false positive upper bound. It then performs lasso-based sparse regression by treating the outlier scores as the target feature and the original features as predictors on the outlier candidate set to obtain a feature subset that is tailored for the outlier scoring method. Our experiments show that two different outlier scoring methods enabled by CINFO (i) perform significantly better on 11 real-life high-dimensional data sets, and (ii) have much better resilience to noisy features, compared to their bare versions and three state-of-the-art competitors. The source code of CINFO is available at https://sites.google.com/site/gspangsite/sourcecode.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pang et al. propose two filter-based unsupervised selection methods, CBRW_FS and CBRW, which select a subset of features independently from subsequent anomaly detection methods.
“Specifically, Pang et al. [156] and Pang et al. [158] propose two filter-based unsupervised selection methods, namely, CBRW_FS and CBRW, which select a subset of features independently from subsequent anomaly detection methods.”
From A Survey on Explainable Anomaly Detection (2024) · cites this paper as [158]
The Abstract (p. 1) says this paper introduces SEMSE and CINFO, which iteratively connect feature selection with outlier scoring rather than selecting features independently. The Introduction (p. 1) describes independent feature selection as a limitation of earlier methods. Our copy does not identify CBRW_FS or CBRW as methods proposed in this paper. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured[158] feature selection is only applicable to categorical data.
“Only applicable to”
From A Survey on Explainable Anomaly Detection (2024) · cites this paper as [158]
The paper distinguishes some feature-selection methods designed for categorical data from feature selection in general (High-dimensional Outlier Detection Methods, p. 2). Its Conclusions (p. 7) say that CINFO refines feature selection and outlier scoring in high-dimensional numeric data, contrary to the claim that feature selection is only applicable to categorical data. (LLM, conf=0.91)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredCBRW_FS and CBRW work only on categorical data through modelling feature-value couplings.
“These two methods work only on categorical data through modelling the feature-value couplings.”
From A Survey on Explainable Anomaly Detection (2024) · cites this paper as [158]
The High-dimensional Outlier Detection Methods section (p. 2) describes a group of methods for categorical data and says attempts to adapt them to numeric data by discretization performed poorly. The References (p. 8) include works on feature-value couplings, but the paper does not identify CBRW_FS or CBRW or establish that both work only through those couplings. (LLM, conf=0.77)
Experimental — accuracy not yet measuredThe above three methods are all filter-based, which independently select subsets of features regardless of subsequent anomaly detection methods.
“The above three methods are all filter-based, which independently select subsets of features regardless of subsequent anomaly detection methods.”
From A Survey on Explainable Anomaly Detection (2024) · cites this paper as [158]
[158] feature selection has linear time complexity to data size.
“Linear time”
From A Survey on Explainable Anomaly Detection (2024) · cites this paper as [158]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 44 · Cited by 59
Research works printed but not linked: 33
- Aggarwal, C., and Yu, S. 2005. An effective and efficient algorithm for high-dimensional outlier detection. The VLDB Journal 14(2):211-221.
- Aggarwal, C. C. 2013. Outlier ensembles: Position paper. ACM SIGKDD Explorations Newsletter 14(2):49-58.
- Aggarwal, C. C. 2017. Outlier analysis . Springer.
- Akoglu, L.; Tong, H.; Vreeken, J.; and Faloutsos, C. 2012. Fast and reliable anomaly detection in categorical data. In CIKM , 415424. ACM.
- Angiulli, F., and Pizzuti, C. 2005. Outlier mining in large highdimensional data sets. IEEE Transactions on Knowledge and Data Engineering 17(2):203-215.
- Angiulli, F.; Fassetti, F.; and Palopoli, L. 2009. Detecting outlying properties of exceptional objects. ACMTransactions on Database Systems 34(1):7.
- Breiman, L. 1996. Bagging predictors. Machine Learning 24(2):123-140.
- Campos, G. O.; Zimek, A.; Sander, J.; Campello, R. J.; Micenkov´ a, B.; Schubert, E.; Assent, I.; and Houle, M. E. 2016. On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data Mining and Knowledge Discovery 30(4):891-927.
- Cao, L.; Ou, Y.; and Yu, P. S. 2012. Coupled behavior analysis with applications. IEEE Transactions on Knowledge and Data Engineering 24(8):1378-1392.
- Dang, X. H.; Assent, I.; Ng, R. T.; Zimek, A.; and Schubert, E. 2014. Discriminative features for identifying and interpreting outliers. In ICDE , 88-99. IEEE.
- Dubhashi, D. P., and Panconesi, A. 2009. Concentration of measure for the analysis of randomized algorithms . Cambridge University Press.
- Freund, Y., and Schapire, R. E. 1995. A desicion-theoretic generalization of on-line learning and an application to boosting. In COLT , 23-37. Springer.
- Hadi, A. S., and Simonoff, J. S. 1993. Procedures for the identification of multiple outliers in linear models. Journal of the American Statistical Association 88(424):1264-1272.
- Hand, D. J., and Till, R. J. 2001. A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning 45(2):171-186.
- Hastie, T.; Tibshirani, R.; and Wainwright, M. 2015. Statistical learning with sparsity: The lasso and generalizations . CRC Press.
- Hong, M., and Luo, Z.-Q. 2017. On the linear convergence of the alternating direction method of multipliers. Mathematical Programming 162(1-2):165-199.
- Keller, F.; Muller, E.; and Bohm, K. 2012. HiCS: High contrast subspaces for density-based outlier ranking. In ICDE , 10371048. IEEE.
- Kriegel, H.-P., and Zimek, A. 2008. Angle-based outlier detection in high-dimensional data. In SIGKDD , 444-452.
- Lazarevic, A., and Kumar, V. 2005. Feature bagging for outlier detection. In SIGKDD , 157-166. ACM.
- Li, J.; Cheng, K.; Wang, S.; Morstatter, F.; Trevino, R. P.; Tang, J.; and Liu, H. 2016. Feature selection: A data perspective. CoRR abs/1601.07996. · arXiv
- Liu, F. T.; Ting, K. M.; and Zhou, Z.-H. 2012. Isolation-based anomaly detection. ACM Transactions on Knowledge Discovery from Data 6(1):3:1-3:39.
- Nguyen, H. V.; Ang, H. H.; and Gopalkrishnan, V. 2010. Mining outliers with ensemble of heterogeneous detectors on random subspaces. In DSFAA , 368-383. Springer.
- Noto, K.; Brodley, C.; and Slonim, D. 2012. FRaC: a feature-modeling approach for semi-supervised and unsupervised anomaly detection. Data Mining and Knowledge Discovery 25(1):109-133.
- Pang, G.; Cao, L.; Chen, L.; and Liu, H. 2016. Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings. In ICDM , 410-419. IEEE.
- Pang, G.; Cao, L.; Chen, L.; and Liu, H. 2017a. Learning homophily couplings from non-iid data for joint feature selection and noise-resilient outlier detection. In IJCAI , 2585-2591.
- Pang, G.; Xu, H.; Cao, L.; and Zhao, W. 2017b. Selective value coupling learning for detecting outliers in high-dimensional categorical data. In CIKM , 807-816. ACM.
- Pang, G.; Cao, L.; and Chen, L. 2016. Outlier detection in complex categorical data by modelling the feature value couplings. In IJCAI , 1902-1908. AAAI Press.
- Pang, G.; Ting, K. M.; and Albrecht, D. 2015. LeSiNN: Detecting anomalies by identifying least similar nearest neighbours. In ICDM Workshop , 623-630. IEEE.
- Paulheim, H., and Meusel, R. 2015. A decomposition of the outlier detection problem into a set of supervised learning problems. Machine Learning 100(2-3):509-531.
- Rayana, S.; Zhong, W.; and Akoglu, L. 2016. Sequential ensemble learning for outlier detection: A bias-variance perspective. In ICDM , 1167-1172. IEEE.
- Sugiyama, M., and Borgwardt, K. 2013. Rapid distance-based outlier detection via sampling. In NIPS , 467-475.
- Zimek, A.; Gaudet, M.; Campello, R. J.; and Sander, J. 2013. Subsampling for efficient and effective unsupervised outlier detection ensembles. In SIGKDD , 428-436. ACM.
- Zimek, A.; Campello, R. J.; and Sander, J. 2013. Ensembles for unsupervised outlier detection: Challenges and research questions. SIGKDD Explorations Newsletter 15(1):11-22.
- Authors
- Pang, G., Cao, L., Chen, L., Lian, D., & Liu, H.
- Affiliations
- University of Technology Sydney (AU)
- University of Electronic Science and Technology of China (CN)
- Arizona State University (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-29
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2788154850
- Cited by
- 59
- References
- 44
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 5 · 2023: 5 · 2022: 7 · 2021: 15 · 2020: 7 · 2019: 9 · 2018: 6
- FWCI
- 3.76
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, other-oa)
- UTS ePRESS (University of Technology Sydney) (submittedVersion)
- Funders
- Australian Research Council (DP140100545)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- outlier detection, high-dimensional data, sparse models, feature selection, LASSO, recurrent neural networks, noisy features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
