Clustering and Unsupervised Anomaly Detection with l2 Normalized Deep Auto-Encoder Representations
Abstract
Clustering is essential to many tasks in pattern recognition and computer vision. With the advent of deep learning, there is an increasing interest in learning deep unsupervised representations for clustering analysis. Many works on this domain rely on variants of auto-encoders and use the encoder outputs as representations/features for clustering. In this paper, we show that an l2normalization constraint on these representations during auto-encoder training, makes the representations more separable and compact in the Euclidean space after training. This greatly improves the clustering accuracy when k-means clustering is employed on the representations. We also propose a clustering based unsupervised anomaly detection method using l2normalized deep auto-encoder representations. We show the effect of l2normalization on anomaly detection accuracy. We further show that the proposed anomaly detection method greatly improves accuracy compared to previously proposed deep methods such as reconstruction error based anomaly detection.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120 · see citing works
Empirical Evaluation of Rectified Activations in Convolutional Network
Xu, B., Wang, N., Chen, T., & Li, M. (2015) · arXiv · doi:10.48550/arxiv.1505.00853 · Cited by 2,264 · see citing works
(2015) · Cited by 1,279 · see citing works
A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data
Goldstein, M., & Uchida, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0152173 · Cited by 1,032 · see citing works
Improved Deep Embedded Clustering with Local Structure Preservation
Guo, X., Gao, L., Liu, X., & Yin, J. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/243 · Cited by 1,002 · see citing works
Deep Clustering with Convolutional Autoencoders
Guo, X., Liu, X., Zhu, E., & Yin, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70096-0_39 · Cited by 575 · see citing works
Deep Unsupervised Clustering with Gaussian Mixture Variational AutoencodersPreprint
Dilokthanakul, N., Mediano, P. A. M., Garnelo, M., Lee, M. C. H., Salimbeni, H., Arulkumaran, K., & Shanahan, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02648 · Cited by 486 · see citing works
Learning Discriminative Reconstructions for Unsupervised Outlier Removal
Xia, Y., Cao, X., Wen, F., Hua, G., & Sun, J. Z. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.177 · Cited by 297 · see citing works
Handwritten zip code recognition with multilayer networks
LeCun, Y., Matan, O., Boser, B. E., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., Jacket, L. D., & Baird, H. S. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.1990.119325 · Cited by 198 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
VT-ADL: A Vision Transformer Network for Image Anomaly Detection and Localization
Mishra, P., Verk, R., Fornasier, D., Piciarelli, C., & Foresti, G. L. (2021) · 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie45552.2021.9576231 · Cited by 462 · see citing works
“Some of the works handled these two steps independently [25], [26], while others achieved better results by doing the two steps collectively, in order to extract the best features for subsequent anomaly detection [27], [28].”
- Authors
- Aytekin, C., Ni, X., Cricri, F., & Aksu, E.
- Affiliations
- Nokia (Finland) (FI)
- Venue
- 2018 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2018-07
- Pages
- 1-6
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2962849408
- Cited by
- 107
- References
- 13
- Citations by year
- 2026: 7 · 2025: 7 · 2024: 9 · 2023: 9 · 2022: 19 · 2021: 17 · 2020: 16 · 2019: 17 · 2018: 6
- FWCI
- 5.69
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Clustering Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- deep autoencoder, L2 normalization, unsupervised clustering, unsupervised anomaly detection, K-means clustering, reconstruction error
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
