Discovering outlier filtering rules from unlabeled data: combining a supervised learner with an unsupervised learner
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
This paper is concerned with the problem of detecting outliers from unlabeled data. In prior work we have developed SmartSifter, which is an on-line outlier detection algorithm based on unsupervised learning from data. On the basis of SmartSifter this paper yields a new framework for outlier filtering using both supervised and unsupervised learning techniques iteratively in order to make the detection process more effective and more understandable. The outline of the framework is as follows: In the first round, for an initial dataset, we run SmartSifter to give each data a score, with a high score indicating a high possibility of being an outlier. Next, giving positive labels to a number of higher scored data and negative labels to a number of lower scored data, we create labeled examples. Then we construct an outlier filtering rule by supervised learning from them. Here the rule is generated based on the principle of minimizing extended stochastic complexity. In the second round, for a new dataset, we filter the data using the constructed rule, then among the filtered data, we run SmartSifter again to evaluate the data in order to update the filtering rule. Applying of our framework to the network intrusion detection, we demonstrate that 1) it can significantly improve the accuracy of SmartSifter, and 2) outlier filtering rules can help the user to discover a general pattern of an outlier group.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187 · cited 1×
“The SDLE algorithm was developed in [21] as an on-line discounting variant of the Laplace law based estimation algorithm, while the SDEM algorithm was developed in [21] as an on-line discounting variant of the incremental EM algorithm (see [15]).”
MethodFisher information and stochastic complexity
Rissanen, J. J. (1996) · IEEE Transactions on Information Theory · doi:10.1109/18.481776 · Cited by 814
Cited by
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687
Discovering cluster-based local outliers
He, Z., Xu, X., & Deng, S. (2003) · Pattern Recognition Letters · doi:10.1016/s0167-8655(03)00003-5 · Cited by 1,055
Fast Outlier Detection in High Dimensional Spaces
Angiulli, F., & Pizzuti, C. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45681-3_2 · Cited by 836
Angle-based outlier detection in high-dimensional data
Kriegel, H.-P., Schubert, M., & Zimek, A. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401946 · Cited by 828
Combining unsupervised and supervised learning in credit card fraud detection
Carcillo, F., Le Borgne, Y.-A., Caelen, O., Kessaci, Y., Oblé, F., & Bontempi, G. (2021) · Information Sciences · doi:10.1016/j.ins.2019.05.042 · Cited by 478
Metadata
- Authors
- Yamanishi, K., & Takeuchi, J.-I.
- Venue
- Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2001-08-26
- Pages
- 389-394
- OpenAlex
- W1963624219
- Cited by
- 103
- References
- 2
- FWCI
- 2.46
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Discovering outlier filtering rules from unlabeled data: combining a supervised learner with an unsupervised learner. https://app.cochat.ai/papers/10.1145/502512.502570
- The paper
- Yamanishi, K., & Takeuchi, J.-I. (2001). Discovering outlier filtering rules from unlabeled data: combining a supervised learner with an unsupervised learner. Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining. https://doi.org/10.1145/502512.502570
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.