The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Ensembles for unsupervised outlier detection: challenges and research questions a position paper
Abstract
Ensembles for unsupervised outlier detection is an emerging topic that has been neglected for a surprisingly long time (although there are reasons why this is more difficult than supervised ensembles or even clustering ensembles). Aggarwal recently discussed algorithmic patterns of outlier detection ensembles, identified traces of the idea in the literature, and remarked on potential as well as unlikely avenues for future transfer of concepts from supervised ensembles. Complementary to his points, here we focus on the core ingredients for building an outlier ensemble, discuss the first steps taken in the literature, and identify challenges for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · cited 1×
“Some popular models are based on the distance between objects [37; 60; 4; 74], or on the density of the neighborhood of an object [9; 56; 34; 38; 42], or based on the variance of angles between object vectors [42; 58], or on other principles of outlierness in various domains [12; 13; 3].”
BackgroundFinding groups in data: an introduction to cluster analysis
(1991) · Choice Reviews Online · doi:10.5860/choice.28-4558 · Cited by 10,474
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Hubert, L., & Arabie, P. (1985) · Journal of Classification · doi:10.1007/bf01908075 · Cited by 8,219
Finding Groups in Data: An Introduction to Cluster Analysis
Kaufman, L., & Rousseeuw, P. J. (1990) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316801 · Cited by 7,344
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
Objective Criteria for the Evaluation of Clustering Methods
Rand, W. M. (1971) · Journal of the American Statistical Association · doi:10.1080/01621459.1971.10482356 · Cited by 6,566
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
Self-citationLOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335388 · Cited by 4,149
Self-citationProcedures for Detecting Outlying Observations in Samples
Grubbs, F. E. (1969) · Technometrics · doi:10.1080/00401706.1969.10490657 · Cited by 4,105 · cited 1×
“…campello@icmc.usp.br jsander@ualberta.ca ABSTRACT Ensembles for unsupervised outlier detection is an emerging topic that has been neglected for a surprisingly long time (although there are reasons why this is more di.cult than supervised ensembles or even clustering ensembles).”
BackgroundHawkins, D. M. (1980) · doi:10.1007/978-94-015-3994-4 · Cited by 2,982
Barnett, V., Lewis, T., & Abeles, F. (1979) · Physics Today · doi:10.1063/1.2995737 · Cited by 2,824
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy
Kuncheva, L. I., & Whitaker, C. J. (2003) · Machine Learning · doi:10.1023/a:1022859003006 · Cited by 2,335 · cited 1×
“Using ensemble techniques to improve classification is based on a sound theory [16; 70; 10; 44; 62].”
MethodIsolation-Based Anomaly Detection
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2012) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/2133360.2133363 · Cited by 2,063
Efficient algorithms for mining outliers from large data sets
Ramaswamy, S., Rastogi, R., & Shim, K. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335437 · Cited by 1,860
Atkinson, A., Barnett, V., & Lewis, T. (1995) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983451 · Cited by 1,682
Essai sur l'application de l'analyse à la probabilité des décisions rendues à la pluralité des voix
Condorcet, N. D. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139923972 · Cited by 1,417
Combining multiple clusterings using evidence accumulation
Fred, A. L. N., & Jain, A. K. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.113 · Cited by 1,195
Efficient algorithms for mining outliers from large data sets
Ramaswamy, S., Rastogi, R., & Shim, K. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335437 · Cited by 1,171
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303321897735 · Cited by 1,092
Diversity creation methods: a survey and categorisation
Brown, G., Wyatt, J., Harris, R., & Yao, X. (2005) · Information Fusion · doi:10.1016/j.inffus.2004.04.004 · Cited by 985
A survey on unsupervised outlier detection in high‐dimensional numerical data
Zimek, A., Schubert, E., & Kriegel, H. (2012) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.11161 · Cited by 864
Self-citationLOCI: fast outlier detection using the local correlation integral
Papadimitriou, S., Kitagawa, H., Gibbons, P. B., & Faloutsos, C. (2004) · Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405) · doi:10.1109/icde.2003.1260802 · Cited by 846
Fast Outlier Detection in High Dimensional Spaces
Angiulli, F., & Pizzuti, C. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45681-3_2 · Cited by 836
Aggarwal, C. C. (2013) · Cited by 832
Angle-based outlier detection in high-dimensional data
Kriegel, H.-P., Schubert, M., & Zimek, A. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401946 · Cited by 828
Self-citationGionis, A., Mannila, H., & Tsaparas, P. (2007) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1217299.1217303 · Cited by 819
Anomaly Detection for Discrete Sequences: A Survey
Chandola, V., Banerjee, A., & Kumar, V. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.235 · Cited by 715
Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier.
Domingos, P., & Pazzani, M. J. (1996) · Cited by 690
Clustering ensembles: models of consensus and weak partitions
Topchy, A., Jain, A. K., & Punch, W. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.237 · Cited by 675
Feature bagging for outlier detection
Lazarevic, A., & Kumar, V. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081891 · Cited by 656
Random projection for high dimensional data clustering: a cluster ensemble approach
Fern, X. Z., & Brodley, C. E. (2003) · Cited by 595
Comparing clusterings: an axiomatic view
Meilǎ, M. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102424 · Cited by 589
Aggarwal, C. C. (2013) · Springer eBooks · doi:10.1007/978-1-4614-6396-2 · Cited by 500
LoOP: local outlier probabilities
Kriegel, H.-P., Kröger, P., Schubert, E., & Zimek, A. (2009) · Proceedings of the 18th ACM conference on Information and knowledge management · doi:10.1145/1645953.1646195 · Cited by 492
Self-citationProceedings of the 26th International Conference on Data Engineering, ICDE 2010
Li, F., Moro, M. M., Ghandeharizadeh, S., Haritsa, J. R., Weikum, G., Carey, M. J., Casati, F., Chang, E. Y., Manolescu, I., Mehrotra, S., et al. (2010) · Max Planck Institute for Plasma Physics · Cited by 452
Barnett, V., & Lewis, T. (1974) · Cited by 445
Kafadar, K., Barnett, V., & Lewis, T. (1995) · Journal of the American Statistical Association · doi:10.2307/2291180 · Cited by 439
Ranking Outliers Using Symmetric Neighborhood Relationship
Jin, W., Tung, A. K. H., Han, J., & Wang, W. (2006) · Lecture Notes in Computer Science · doi:10.1007/11731139_68 · Cited by 415
A New Local Distance-Based Outlier Detection Approach for Scattered Real-World Data
Zhang, K., Hutter, M., & Jin, H. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-01307-2_84 · Cited by 392
HiCS: High Contrast Subspaces for Density-Based Outlier Ranking
Keller, F., Muller, E., & Bohm, K. (2012) · 2012 IEEE 28th International Conference on Data Engineering · doi:10.1109/icde.2012.88 · Cited by 373
Outlier detection by active learning
Abe, N., Zadrozny, B., & Langford, J. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150459 · Cited by 322
Outlier Detection in Axis-Parallel Subspaces of High Dimensional Data
Kriegel, H.-P., Kröger, P., Schubert, E., & Zimek, A. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-01307-2_86 · Cited by 314
Self-citationSchubert, E., Zimek, A., & Kriegel, H.-P. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-012-0300-z · Cited by 300
Self-citationUsing diversity in cluster ensembles
Kuncheva, L. I., & Hadjitodorov, S. T. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1399790 · Cited by 289
Ensembles of Learning Machines
Valentini, G., & Masulli, F. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45808-5_1 · Cited by 276
A unified notion of outliers: properties and computation
Knorr, E. M., & Ng, R. T. (1997) · Cited by 275
Density-Based Clustering Validation
Moulavi, D., Jaskowiak, P. A., Campello, R. J. G. B., Zimek, A., & Sander, J. (2014) · Proceedings of the 2014 SIAM International Conference on Data Mining · doi:10.1137/1.9781611973440.96 · Cited by 275
Self-citationInterpreting and Unifying Outlier Scores
Kriegel, H.-P., Kroger, P., Schubert, E., & Zimek, A. (2011) · Proceedings of the 2011 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972818.2 · Cited by 266
Self-citationNguyen, N., & Caruana, R. (2007) · Seventh IEEE International Conference on Data Mining (ICDM 2007) · doi:10.1109/icdm.2007.73 · Cited by 252
Characterization and evaluation of similarity measures for pairs of clusterings
Pfitzner, D., Leibbrandt, R., & Powers, D. (2009) · Knowledge and Information Systems · doi:10.1007/s10115-008-0150-6 · Cited by 246
Moderate diversity for better cluster ensembles
Hadjitodorov, S. T., Kuncheva, L. I., & Todorova, L. (2005) · Information Fusion · doi:10.1016/j.inffus.2005.01.008 · Cited by 224
Outlier ensembles: position paper
Aggarwal, C. C. (2013) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2481244.2481252 · Cited by 192
Subsampling for efficient and effective unsupervised outlier detection ensembles
Zimek, A., Gaudet, M., Campello, R. J. G. B., & Sander, J. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487676 · Cited by 190
Self-citationSystematic construction of anomaly detection benchmarks from real data
Emmott, A., Das, S., Dietterich, T. G., Fern, A., & Wong, W. (2013) · doi:10.1145/2500853.2500858 · Cited by 187
Ghosh, J., & Acharya, A. (2011) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.32 · Cited by 187
Caruana, R., Elhawary, M., Nguyen, N., & Smith, C. (2006) · Sixth International Conference on Data Mining (ICDM'06) · doi:10.1109/icdm.2006.103 · Cited by 180
A survey on enhanced subspace clustering
Sim, K., Gopalkrishnan, V., Zimek, A., & Cong, G. (2012) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-012-0258-x · Cited by 171
Self-citationOn Evaluation of Outlier Rankings and Outlier Scores
Schubert, E., Wojdanowski, R., Zimek, A., & Kriegel, H. (2012) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972825.90 · Cited by 166
Self-citationAnalysis of Consensus Partition in Cluster Ensemble
Topchy, A. P., Law, M. H. C., Jain, A. K., & Fred, A. L. (2005) · Fourth IEEE International Conference on Data Mining (ICDM'04) · doi:10.1109/icdm.2004.10100 · Cited by 160
Generalized Outlier Detection with Flexible Kernel Density Estimates
Schubert, E., Zimek, A., & Kriegel, H. (2014) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611973440.63 · Cited by 153
Self-citationConverting Output Scores from Outlier Detection Algorithms into Probability Estimates
Gao, J., & Tan, P. (2006) · Proceedings · doi:10.1109/icdm.2006.43 · Cited by 145
Statistical selection of relevant subspace projections for outlier ranking
Muller, E., Schiffer, M., & Seidl, T. (2011) · 2011 IEEE 27th International Conference on Data Engineering · doi:10.1109/icde.2011.5767916 · Cited by 126
Pham, N., & Pagh, R. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339669 · Cited by 114
Mining Outliers with Ensemble of Heterogeneous Detectors on Random Subspaces
Nguyen, H. V., Ang, H. H., & Gopalkrishnan, V. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12026-8_29 · Cited by 109
On Using Class-Labels in Evaluation of Clusterings
Färber, I., Günnemann, S., Kriegel, H., Kröger, P., Müller, E., Schubert, E., Seidl, T., Zimek, A., & München, L.-M.-U. (2010) · Cited by 100
Self-citationAdaptive cluster ensemble selection
Azimi, J., & Fern, X. Z. (2009) · Cited by 99
Kriegel, H., Kröger, P., & Zimek, A. (2012) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1057 · Cited by 85
Self-citationDiscriminative features for identifying and interpreting outliers
Dang, X. H., Assent, I., Ng, R. T., Zimek, A., & Schubert, E. (2014) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2014.6816642 · Cited by 84
Self-citationOutlier Ranking via Subspace Analysis in Multiple Views of the Data
Müller, E., Assent, I., Iglesias, P., Mülle, Y., & Böhm, K. (2012) · Proceedings · doi:10.1109/icdm.2012.112 · Cited by 81
Cluster ensemble selection based on relative validity indexes
Naldi, M. C., de Carvalho, A. C. P. L. F., & Campello, R. J. G. B. (2012) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-012-0290-x · Cited by 76
Self-citationZimek, A., & Vreeken, J. (2013) · Machine Learning · doi:10.1007/s10994-013-5334-y · Cited by 51
Self-citationComparative study of matrix refinement approaches for ensemble clustering
Iam-On, N., & Boongoen, T. (2013) · Machine Learning · doi:10.1007/s10994-013-5342-y · Cited by 44
Local peculiarity factor and its application in outlier detection
Yang, J. J., Zhong, N., Yao, Y. Y., & Wang, J. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1401983 · Cited by 33
On the combination of relative clustering validity criteria
Vendramin, L., Jaskowiak, P. A., & Campello, R. J. G. B. (2013) · International Conference on Scientific and Statistical Database Management · doi:10.1145/2484838.2484844 · Cited by 32
Self-citationUnifying dependent clustering and disparate clustering for non-homogeneous data
Hossain, M. S., Tadepalli, S., Watson, L. T., Davidson, I. A. K., Helm, R. F., & Ramakrishnan, N. (2010) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835880 · Cited by 26
Selecting Diversifying Heuristics for Cluster Ensembles
Hadjitodorov, S. T., & Kuncheva, L. I. (2007) · Springer eBooks · doi:10.1007/978-3-540-72523-7_21 · Cited by 23
Outlier Detection with Space Transformation and Spectral Analysis
Dang, X. H., Micenková, B., Assent, I., & Ng, R. T. (2013) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972832.25 · Cited by 21
Cited by
Graph based anomaly detection and description: a survey
Akoglu, L., Tong, H., & Koutra, D. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0365-y · Cited by 1,504
A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data
Goldstein, M., & Uchida, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0152173 · Cited by 1,032
Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection
Campello, R. J. G. B., Moulavi, D., Zimek, A., & Sander, J. (2015) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/2733381 · Cited by 915
“Let us note that some methods have been proposed to combine single outlier detectors into an ensemble [Lazarevic and Kumar 2005; Gao and Tan 2006; Nguyen et al. 2010; Kriegel et al. 2011a; Schubert et al. 2012; Liu et al. 2012; Zimek et al. 2013a, 2013, 2014].”
MethodSelf-citationOn the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study
Campos, G. O., Zimek, A., Sander, J., Campello, R. J. G. B., Micenková, B., Schubert, E., Assent, I., & Houle, M. E. (2016) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0444-8 · Cited by 735
Self-citationProgress in Outlier Detection Techniques: A Survey
Wang, H., Bah, M. J., & Hammad, M. (2019) · IEEE Access · doi:10.1109/access.2019.2932769 · Cited by 523
Metadata
- Authors
- Zimek, A., Campello, R. J. G. B., & Sander, J.
- Venue
- ACM SIGKDD Explorations Newsletter
- Published
- 2014-03-17
- Volume
- 15
- Issue
- 1
- Pages
- 11-22
- ISSN
- 1931-0145 (ISSN-L)
- 1931-0153
- OpenAlex
- W2101549186
- Cited by
- 278
- References
- 81
- FWCI
- 21.46
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.