Prediction-based outlier detection with explanations
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
General outlier detection strategies, be a distribution-based, clustering-based, or distance-based method, all resort to the comparison among instances to define abnormality. In this paper we introduce an additional dimension into the outlier definition. That is, we not only consider externally how one instance dyers from others but internally the dependency and abnormality among its own attributes, denoted as the prediction-based outlier detection. Prediction-based outliers possess certain attributes which are difficult to be predicted based on the neighborhood information. Furthermore, we propose three neighborhood functions to generate predictions. Finally, acknowledging the lack of the gold standard to evaluate an outlier detection system, we propose four general evaluation strategies. Experiments conducted on several real-world datasets demonstrate the validity, novelty, power-law distribution, and robustness of our method.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
(1996) · Cited by 20,179
Finding and evaluating community structure in networks
Newman, M. E. J., & Girvan, M. (2004) · Physical Review E · doi:10.1103/physreve.69.026113 · Cited by 13,931 · cited 1×
“If the data are connected through a social network or a graph, Ns can be defined by using the conventional community detection algorithm [15] or rolebased clustering algorithm [22].”
MethodRobust Locally Weighted Regression and Smoothing Scatterplots
Cleveland, W. S. (1979) · Journal of the American Statistical Association · doi:10.1080/01621459.1979.10481038 · Cited by 11,594
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
Social Structure from Multiple Networks. I. Blockmodels of Roles and Positions
White, H. C., Boorman, S. A., & Breiger, R. L. (1976) · American Journal of Sociology · doi:10.1086/226141 · Cited by 2,097 · cited 1×
“In addition to using the numerical x to determine the neighbors, it is possible to utilize other two kinds of neighborhood functions to identify different kinds of outliers.”
MethodRobust Locally Weighted Regression and Smoothing Scatterplots
Cleveland, W. S. (1979) · Journal of the American Statistical Association · doi:10.2307/2286407 · Cited by 2,063
Algorithms for Mining Distance-Based Outliers in Large Datasets
Knorr, E. M., & Ng, R. T. (1998) · Cited by 1,544
Efficient algorithms for mining outliers from large data sets
Ramaswamy, S., Rastogi, R., & Shim, K. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335437 · Cited by 1,171 · cited 4×
“Assuming there are n data points {p1, p2, …, pn} in the dataset, our goal is to learn whether a point pi={pi1, pi2,…pim} is a prediction-based outlier, where pik represents the k-th attribute of pi.”
BackgroundMethodInfluentialDiscovering cluster-based local outliers
He, Z., Xu, X., & Deng, S. (2003) · Pattern Recognition Letters · doi:10.1016/s0167-8655(03)00003-5 · Cited by 1,055 · cited 1×
“Cluster-based outlier detection finds clusters first, and then declares data points that do not belong to any cluster as outliers [5][6][8][14].”
LOCI: fast outlier detection using the local correlation integral
Papadimitriou, S., Kitagawa, H., Gibbons, P. B., & Faloutsos, C. (2004) · Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405) · doi:10.1109/icde.2003.1260802 · Cited by 846 · cited 2×
“We conduct scaling of data to make sure each attribute is treated equally in the analysis.”
BackgroundMethodAngle-based outlier detection in high-dimensional data
Kriegel, H.-P., Schubert, M., & Zimek, A. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401946 · Cited by 828 · cited 3×
“Studies of outlier detection can generally be divided into three categories: distributionbased [1][25], cluster-based [5][6][8][14], and distancebased [2][9][10][11][19] approaches.”
BackgroundROCK: a robust clustering algorithm for categorical attributes
Guha, S., Rastogi, R., & Shim, K. (1999) · Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337) · doi:10.1109/icde.1999.754967 · Cited by 609 · cited 1×
“Studies of outlier detection can generally be divided into three categories: distribution based [1][25], cluster-based [5][6][8][14], and distance based [2][9][10][11][19] approaches.”
BackgroundFinding Intensional Knowledge of Distance-Based Outliers
Knorr, E. M., & Ng, R. T. (1999) · Very Large Data Bases · Cited by 535
Noble, C. C., & Cook, D. J. (2003) · Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/956750.956831 · Cited by 454 · cited 1×
“Some outlier detection methods have been designed for graphs in which data instances are connected to each other [4][16][21].”
MethodNeighborhood Formation and Anomaly Detection in Bipartite Graphs
Sun, J., Qu, H., Chakrabarti, D., & Faloutsos, C. (2006) · Fifth IEEE International Conference on Data Mining (ICDM'05) · doi:10.1109/icdm.2005.103 · Cited by 308
Bremer, R. (1995) · Technometrics · doi:10.1080/00401706.1995.10485900 · Cited by 202
Discovering Structural Anomalies in Graph-Based Data
Eberle, W., & Holder, L. B. (2007) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2007.91 · Cited by 87
Discovering and Explaining Abnormal Nodes in Semantic Graphs
Lin, S.-D., & Chalupsky, H. (2008) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190691 · Cited by 44 · cited 2×
“Being able to generate explanations is important for many outlier applications in domains like security [13].”
BackgroundSelf-citationEXPLANATIONS, MACHINE LEARNING, AND CREATIVITY
Schank, R. C., & Kass, A. (1990) · Elsevier eBooks · doi:10.1016/b978-0-08-051055-2.50005-5 · Cited by 16
Yao, Y. Y., Zhao, Y., & Maguire, R. B. (2003) · Lecture notes in computer science · doi:10.1007/3-540-44886-1_45 · Cited by 15
Modeling, searching, and explaining abnormal instances in multi-relational networks
Knight, K. K., Chalupsky, H., & Lin, S.-D. (2006) · Cited by 4
Self-citation
Research works printed but not linked: 6
- pitt, 1988, Theory of Explanation
- nanopoulos, 2001, C2P Clustering Based on Closest Pairs Proceedings of the 27th International Conference on Very Large Data Bases
- yamanishi, 0, Discovering Outlier Filtering Rules from Unlabeled Data Combining A Supervised Learner with An Unsupervised Learner, 389
- zhang, 2008, LDBOD A Novel Local Distribution Based Outlier Detector Pattern Recognition Letters 29, 967
- haynes, 2001, Explanation in Information Systems A Design Rationale Approach the London School of Economics
- knorr, 1998, Algorithms for mining distancebased outliers in large datasets, Proc 24th Int Conf Very Large Data Bases, 392
Cited by
A Survey on Explainable Anomaly Detection
Li, Z., Zhu, Y., & Van Leeuwen, M. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3609333 · Cited by 139
“Moreover, model-specific techniques such as References [7, 31, 94] generally have consistent Explanation-Definition and Detection-Definition of anomaly as they explore the internal structure of anomaly detection model to generate explanations.”
BackgroundMethodInfluentialEXPERIENCE: Algorithms and Case Study for Explaining Repairs with Uniform Profiles over IoT Data
Liu, Z., Zhang, Y., Huang, R., Chen, Z., Song, S., & Wang, J. (2021) · Journal of Data and Information Quality · doi:10.1145/3436239 · Cited by 4
“[5] propose outlier detection and interpretation based on prediction.”
Metadata
- Authors
- Chen, L.-C., Kuo, T.-T., Lai, W.-C., Lin, S.-D., & Tsai, C.-H.
- Affiliations
- National Taiwan University (TW)
- Institute for Information Industry (TW)
- Venue
- 2012 IEEE International Conference on Granular Computing
- Publisher
- IEEE
- Published
- 2012-08
- Volume
- 20
- Pages
- 44-49
- OpenAlex
- W2090448575
- Cited by
- 2
- References
- 20
- Citations by year
- 2023: 1 · 2021: 1
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Granular Computing (GrC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- outlier detection, explanation, neighborhood function, attribute dependencies, power-law distribution, robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.