The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions
Open accessAbstract
Outlier detection refers to the identification of data points that deviate from a general data distribution. Existing unsupervised approaches often suffer from high computational cost, complex hyperparameter tuning, and limited interpretability, especially when working with large, high-dimensional datasets. To address these issues, we present a simple yet effective algorithm calledECOD(Empirical-Cumulative-distribution-based Outlier Detection), which is inspired by the fact that outliers are often the “rare events” that appear in the tails of a distribution. In a nutshell,ECODfirst estimates the underlying distribution of the input data in a nonparametric fashion by computing the empirical cumulative distribution per dimension of the data.ECODthen uses these empirical distributions to estimate tail probabilities per dimension for each data point. Finally,ECODcomputes an outlier score of each data point by aggregating estimated tail probabilities across dimensions. Our contributions are as follows: (1) we propose a novel outlier detection method calledECOD, which is both parameter-free and easy to interpret; (2) we perform extensive experiments on 30 benchmark datasets, where we find thatECODoutperforms 11 state-of-the-art baselines in terms of accuracy, efficiency, and scalability; and (3) we release an easy-to-use and scalable (with distributed support) Python implementation for accessibility and reproducibility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601
Estimating the Support of a High-Dimensional Distribution
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001) · Neural Computation · doi:10.1162/089976601750264965 · Cited by 6,168
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.03.013 · Cited by 4,147 · cited 1×
“This has motivated commonly used heuristic OD approaches such as the “three-sigma” rule that declares points more than three standard deviations from the mean to be outliers ([27], [28]; there’s also a robust variant called the “1.5 IQR” rule [29]).”
MethodInfluentialDeep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342 · cited 1×
“We report both the raw comparison results and the critical difference (CD) plots to show statistical difference [55], [56], where it visualize the statistical comparison by Wilcoxon signed-rank test with Holm’s correction.”
MethodDeep Learning for Anomaly Detection: A Review
Pang, G., Shen, C., Cao, L., & Hengel, A. V. D. (2022) · ACM Computing Surveys · doi:10.1145/3439950 · Cited by 2,837
Efficient algorithms for mining outliers from large data sets
Ramaswamy, S., Rastogi, R., & Shim, K. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335437 · Cited by 1,860
Ruff, L., Vandermeulen, R. A., Görnitz, N., Deecke, L., Siddiqui, S. A., Binder, A., Müller, E., & Kloft, M. (2018) · International Conference on Machine Learning · Cited by 1,719
Statistical Comparisons of Classifiers over Multiple Data Sets
DemšarJanez (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248548 · Cited by 1,074
Discovering cluster-based local outliers
He, Z., Xu, X., & Deng, S. (2003) · Pattern Recognition Letters · doi:10.1016/s0167-8655(03)00003-5 · Cited by 1,055
LOCI: fast outlier detection using the local correlation integral
Papadimitriou, S., Kitagawa, H., Gibbons, P. B., & Faloutsos, C. (2004) · Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405) · doi:10.1109/icde.2003.1260802 · Cited by 846
Breast Cancer Diagnosis and Prognosis Via Linear Programming
Mangasarian, O. L., Street, W. N., & Wolberg, W. H. (1995) · Operations Research · doi:10.1287/opre.43.4.570 · Cited by 843
Angle-based outlier detection in high-dimensional data
Kriegel, H.-P., Schubert, M., & Zimek, A. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401946 · Cited by 828
Measuring Skewness and Kurtosis
Groeneveld, R. A., & Meeden, G. (1984) · The Statistician · doi:10.2307/2987742 · Cited by 818
Optimal Rates of Convergence for Nonparametric Estimators
Stone, C. J. (1980) · The Annals of Statistics · doi:10.1214/aos/1176345206 · Cited by 795
On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study
Campos, G. O., Zimek, A., Sander, J., Campello, R. J. G. B., Micenková, B., Schubert, E., Assent, I., & Houle, M. E. (2016) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-015-0444-8 · Cited by 735
Statistical Learning with Sparsity: The Lasso and Generalizations
O'Brien, C. M. (2016) · International Statistical Review · doi:10.1111/insr.12167 · Cited by 726
A Modern Introduction to Probability and Statistics: Understanding Why and How
Dekking, F. M., Kraaikamp, C., Lopuhaä, H. P., & Meester, L. E. (2005) · Springer Texts in Statistics · doi:10.1007/1-84628-168-7 · Cited by 660
Feature bagging for outlier detection
Lazarevic, A., & Kumar, V. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081891 · Cited by 656
Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters
Dou, Y., Liu, Z., Sun, L., Deng, Y., Peng, H., & Yu, P. S. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3411903 · Cited by 612
A Novel Anomaly Detection Scheme Based on Principal Component Classifier
Shyu, M., Chen, S., Sarinnapakorn, K., & Chang, L. (2003) · Cited by 602
Xu, R., Xiang, H., Xia, X., Han, X., Li, J., & Ma, J. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9812038 · Cited by 586
Histogram-based Outlier Score (HBOS): A fast Unsupervised Anomaly Detection Algorithm
Goldstein, M., & Dengel, A. R. (2012) · Cited by 449
Outlier Detection with Autoencoder Ensembles
Chen, J., Sathe, S., Aggarwal, C., & Turaga, D. (2017) · Proceedings of the 2017 SIAM International Conference on Data Mining · doi:10.1137/1.9781611974973.11 · Cited by 438
Incremental Local Outlier Detection for Data Streams
Pokrajac, D., Lazarevic, A., & Latecki, L. J. (2007) · 2007 IEEE Symposium on Computational Intelligence and Data Mining · doi:10.1109/cidm.2007.368917 · Cited by 386
Loda: Lightweight on-line detector of anomalies
Pevný, T. (2016) · Machine Learning · doi:10.1007/s10994-015-5521-0 · Cited by 357
Generative Adversarial Active Learning for Unsupervised Outlier Detection
Liu, Y., Li, Z., Zhou, C., Jiang, Y., Sun, J., Wang, M., & He, X. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2019.2905606 · Cited by 335
Self-citationDependence Modeling: Vine Copula Handbook
Kurowicka, D., & Joe, H. (2010) · RePEc: Research Papers in Economics · doi:10.1142/7699 · Cited by 323
Aggarwal, C. C. (2015) · Data Mining · doi:10.1007/978-3-319-14142-8_8 · Cited by 305
OpenCDA: An Open Cooperative Driving Automation Framework Integrated with Co-Simulation
Xu, R., Guo, Y., Han, X., Xia, X., Xiang, H., & Ma, J. (2021) · 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) · doi:10.1109/itsc48978.2021.9564825 · Cited by 260
Toward Deep Supervised Anomaly Detection: Reinforcement Learning from Partially Labeled Anomaly Data
Pang, G., van den Hengel, A., Shen, C., & Cao, L. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467417 · Cited by 179
XGBOD: Improving Supervised Outlier Detection with Unsupervised Representation Learning
Zhao, Y., & Hryniewicki, M. K. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489605 · Cited by 163
Self-citationLSCP: Locally Selective Combination in Parallel Outlier Ensembles
Zhao, Y., Nasrullah, Z., Hryniewicki, M. K., & Li, Z. (2019) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611975673.66 · Cited by 160
Self-citationA Comparative Study for Outlier Detection Techniques in Data Mining
Bakar, Z. B. A., Mohemad, R., Ahmad, A., & Deris, M. M. (2006) · IEEE Conference on Cybernetics and Intelligent Systems · doi:10.1109/iccis.2006.252287 · Cited by 157
A robust least squares support vector machine for regression and classification with noise
Yang, X., Tan, L., & He, L. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.03.037 · Cited by 138
Fast and reliable anomaly detection in categorical data
Akoglu, L., Tong, H., Vreeken, J., & Faloutsos, C. (2012) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2396761.2396816 · Cited by 128
Multi-attributed heterogeneous graph convolutional network for bot detection
Zhao, J., Liu, X., Yan, Q., Li, B. H., Shao, M., & Peng, H. M. (2020) · Information Sciences · doi:10.1016/j.ins.2020.03.113 · Cited by 112
Outlier Detection with Globally Optimal Exemplar-Based GMM
Yang, X., Latecki, L. J., & Pokrajac, D. (2009) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972795.13 · Cited by 111
Aggarwal, C. C., & Sathe, S. (2017) · Springer eBooks · doi:10.1007/978-3-319-54765-7 · Cited by 107
A comparison of outlier detection algorithms for ITS data
Chen, S., Wang, W., & van Zuylen, H. J. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.06.008 · Cited by 88
SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier DetectionPreprint
Zhao, Y., Hu, X., Cheng, C., Wang, C., Wan, C., Wang, W., Yang, J., Bai, H., Li, Z., Xiao, C., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05731 · Cited by 63
Self-citationOn the tight constant in the multivariate Dvoretzky–Kiefer–Wolfowitz inequality
Naaman, M. (2021) · Statistics & Probability Letters · doi:10.1016/j.spl.2021.109088 · Cited by 63
Beyond Outlier Detection: LookOut for Pictorial Explanation
Gupta, N., Eswaran, D., Shah, N., Akoglu, L., & Faloutsos, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-10925-7_8 · Cited by 57
Ring: Real-Time Emerging Anomaly Monitoring System Over Text Streams
Yu, W., Li, J., Bhuiyan, M. Z. A., Zhang, R., & Huai, J. (2017) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2017.2672672 · Cited by 50
MVAN: Multi-view Attention Networks for Real Money Trading Detection in Online Games
Tao, J., Lin, J., Zhang, S., Zhao, S., Wu, R., Fan, C., & Cui, P. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330687 · Cited by 31
Outlier-Robust Multi-Aspect Streaming Tensor Completion and Factorization
Najafi, M., He, L., & Yu, P. S. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/442 · Cited by 31
Kernel Density Outlier Detector
Meropi, P., & Zioutas, G. (2014) · Springer proceedings in mathematics & statistics · doi:10.1007/978-1-4939-0569-0_22 · Cited by 19
Anomaly Mining - Past, Present and Future
Akoglu, L. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482495 · Cited by 12
A Large-scale Study on Unsupervised Outlier Model Selection: Do Internal Strategies Suffice?Preprint
Ma, M. Q., Zhao, Y., Zhang, X., & Akoglu, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.01422 · Cited by 10
Self-citationHuman-Algorithmic Bias: Source, Evolution, and ImpactPreprint
Hu, X., Huang, Y. H., Li, B., & Lu, T. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4195014 · Cited by 4
Self-citation
Cited by
Deep Isolation Forest for Anomaly Detection
Xu, H., Pang, G., Wang, Y., & Wang, Y. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3270293 · Cited by 556
Multimodal Industrial Anomaly Detection via Hybrid Fusion
Wang, Y., Peng, J., Zhang, J., Yi, R., Wang, Y., & Wang, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00776 · Cited by 246
Unsupervised detection of coordinated fake-follower campaigns on social media
Zouzou, Y., & Varol, O. (2024) · EPJ Data Science · doi:10.1140/epjds/s13688-024-00499-6 · Cited by 25
Auto-Validate by-History: Auto-Program Data Quality Constraints to Validate Recurring Data Pipelines
Tu, D., He, Y., Cui, W., Ge, S., Zhang, H., Han, S., Zhang, D., & Chaudhuri, S. (2023) · Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599776 · Cited by 14
Metadata
- Authors
- Li, Z., Zhao, Y., Hu, X., Botta, N., Ionescu, C., & Chen, G. H.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2023-12-01
- Volume
- 35
- Issue
- 12
- Pages
- 12181-12193
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W4226060502
- Cited by
- 494
- References
- 52
- FWCI
- 29.49
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.