The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Unsupervised Anomaly Detection With LSTM Neural Networks
Open accessAbstract
We investigate anomaly detection in an unsupervised framework and introduce long short-term memory (LSTM) neural network-based algorithms. In particular, given variable length data sequences, we first pass these sequences through our LSTM-based structure and obtain fixed-length sequences. We then find a decision function for our anomaly detectors based on the one-class support vector machines (OC-SVMs) and support vector data description (SVDD) algorithms. As the first time in the literature, we jointly train and optimize the parameters of the LSTM architecture and the OC-SVM (or SVDD) algorithm using highly effective gradient and quadratic programming-based training methods. To apply the gradient-based training method, we modify the original objective criteria of the OC-SVM and SVDD algorithms, where we prove the convergence of the modified objective criteria to the original criteria. We also provide extensions of our unsupervised formulation to the semisupervised and fully supervised frameworks. Thus, we obtain anomaly detection algorithms that can process variable length data sequences while providing high performance, especially for time series data. Our approach is generic so that we also apply this approach to the gated recurrent unit (GRU) architecture by directly replacing our LSTM-based structure with the GRU-based structure. In our experiments, we illustrate significant performance gains achieved by our algorithms with respect to the conventional methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Chandola, V., Banerjee, A., & Kumar, V. (2009) · ACM Computing Surveys · doi:10.1145/1541880.1541882 · Cited by 11,687 · cited 10×
“, an anomaly is an event, which cannot be predicted from the past nominal data [1].”
BackgroundMethodInfluentialEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“However, since the basic RNN architecture does not have control structures (gates) to regulate the amount of information to be stored [16], [17], a more advanced RNN architecture with several control structures, i.”
BackgroundThe use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 3×
“, the LSTM architecture without peephole connections [17].”
BackgroundEstimating the Support of a High-Dimensional Distribution
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001) · Neural Computation · doi:10.1162/089976601750264965 · Cited by 6,168 · cited 10×
“4) Through an extensive set of experiments involving real and simulated data, we illustrate significant performance improvements achieved by our algorithms with respect to the conventional methods [6], [7], [10].”
BackgroundMethodInfluentialPackard, N. H., Crutchfield, J. P., Farmer, J. D., & Shaw, R. S. (1980) · Physical Review Letters · doi:10.1103/physrevlett.45.712 · Cited by 3,869
Support Vector Data Description
Tax, D. M. J., & Duin, R. P. W. (2004) · Machine Learning · doi:10.1023/b:mach.0000008084.60811.49 · Cited by 3,512
(1998) · Cited by 2,679
Fundamentals of Adaptive Filtering
Ramirez, P. S. (2002) · The Kluwer International Series in Engineering and Computer Science · doi:10.1007/978-1-4757-3637-3_2 · Cited by 1,958
Fundamentals of adaptive filtering
Sayed, A. H. (2003) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,673
Long Short Term Memory Networks for Anomaly Detection in Time Series.
Malhotra, P., Vig, L., Shroff, G. M., & Agarwal, P. (2015) · The European Symposium on Artificial Neural Networks · Cited by 1,259
A feasible method for optimization with orthogonality constraints
Wen, Z., & Yin, W. (2013) · Mathematical Programming · doi:10.1007/s10107-012-0584-1 · Cited by 849
Semi-Supervised Support Vector Machines
Bennett, K. P., & Demiriz, A. (1998) · Cited by 826
Candanedo, L. M., & Feldheim, V. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2015.11.071 · Cited by 586
A neural network component for an intrusion detection system
Debar, H., Becker, M., & Siboni, D. (2003) · Proceedings 1992 IEEE Computer Society Symposium on Research in Security and Privacy · doi:10.1109/risp.1992.213257 · Cited by 461
Novelty Detection and Neural Network Validation
Bishop, C. M. (1993) · ICANN ’93 · doi:10.1007/978-1-4471-2063-6_225 · Cited by 436
Anomaly detection in ECG time signals via deep long short-term memory networks
Chauhan, S., & Vig, L. (2015) · 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2015.7344872 · Cited by 405
Novelty detection and neural network validation
Bishop, C. M. (1994) · IEE Proceedings - Vision, Image, and Signal Processing · doi:10.1049/ip-vis:19941330 · Cited by 394
Time-series novelty detection using one-class support vector machines
Ma, J., & Perkins, S. (2004) · Proceedings of the International Joint Conference on Neural Networks, 2003. · doi:10.1109/ijcnn.2003.1223670 · Cited by 392
Toward Supervised Anomaly Detection
Goernitz, N., Kloft, M., Rieck, K., & Brefeld, U. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3623 · Cited by 371
Full-Capacity Unitary Recurrent Neural NetworksPreprint
Wisdom, S., Powers, T., Hershey, J. R., Roux, J. L., & Atlas, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00035 · Cited by 264
Beck, A. (2015) · SIAM Journal on Optimization · doi:10.1137/13094829x · Cited by 221
Regression Modeling with Actuarial and Financial Applications
Frees, E. W. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511814372 · Cited by 214
Anomaly intrusion detection using one class SVM
Wang, Y., Wong, J., & Miner, A. S. (2005) · IEEE SMC Information Assurance Workshop (IAW) · doi:10.1109/iaw.2004.1437839 · Cited by 166
Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences
Ergen, T., & Kozat, S. S. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2770179 · Cited by 105
Self-citationEfficient Unsupervised Parameter Estimation for One-Class Support Vector Machines
Ghafoori, Z., Erfani, S., Rajasegarar, S., Bezdek, J. C., Karunasekera, S. A., & Leckie, C. A. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2785792 · Cited by 83
Detecting Anomalous Activity on Networks With the Graph Fourier Scan Statistic
Sharpnack, J., Rinaldo, A., & Singh, A. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2481866 · Cited by 47
Optimal Index Policies for Anomaly Localization in Resource-Constrained Cyber Systems
Cohen, K., Zhao, Q. Q., & Swami, A. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2332982 · Cited by 44
Support Vector Data Descriptions and $k$ -Means Clustering: One Class?
Görnitz, N., Lima, L. A., Müller, K., Kloft, M., & Nakajima, S. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2737941 · Cited by 44
Classifying Time Series Using Local Descriptors with Hybrid Sampling
Zhao, J., & Itti, L. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2492558 · Cited by 39
AnRAD: A Neuromorphic Anomaly Detection Framework for Massive Concurrent Data Streams
Chen, Q., Luley, R. S., Wu, Q., Bishop, M. A., Linderman, R., & Qiu, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2676110 · Cited by 34
Classification of Time-Series Data Using a Generative/Discriminative Hybrid
Abou–Moustafa, K. T., Cheriet, M., & Suen, C. Y. (2004) · International Workshop on Frontiers in Handwriting Recognition (IWFHR) · doi:10.1109/iwfhr.2004.26 · Cited by 15
Gated Orthogonal Recurrent Units: On Learning to ForgetPreprint
Jing, L., Gülçehre, Ç., Peurifoy, J., Shen, Y., Tegmark, M., Marin Soljacic, A., & Bengio, Y. (2017) · arXiv (Cornell University) · Cited by 15
Learning to Classify With Possible Sensor Failures
Xie, T., Nasrabadi, N. M., & Hero, A. O. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2626263 · Cited by 9
Fisher information framework for time series modeling
Venkatesan, R. C., & Plastino, Á. L. (2017) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2017.02.076 · Cited by 7
Cited by
Deep Learning for Anomaly Detection: A Survey
Chalapathy, R., & Chawla, S. (2019) · arXiv · doi:10.48550/arxiv.1901.03407 · Cited by 1,240
Deep Learning for Time Series Anomaly Detection: A Survey
Zamanzadeh Darban, Z., Webb, G. I., Pan, S., Aggarwal, C., & Salehi, M. (2025) · ACM Computing Surveys · doi:10.1145/3691338 · Cited by 613
A survey on anomaly detection for technical systems using LSTM networks
Lindemann, B., Maschler, B., Sahlab, N., & Weyrich, M. (2021) · Computers in Industry · doi:10.1016/j.compind.2021.103498 · Cited by 445
Deep Semi-Supervised Anomaly Detection
Ruff, L., Vandermeulen, R. A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R., & Kloft, M. (2019) · arXiv · doi:10.48550/arxiv.1906.02694 · Cited by 388
Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals
Zhang, Y., Chen, Y., Wang, J., & Pan, Z. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3102110 · Cited by 312
Metadata
- Authors
- Ergen, T., & Kozat, S. S.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2020-08
- Volume
- 31
- Issue
- 8
- Pages
- 3127-3141
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2766761849
- Cited by
- 386
- References
- 39
- FWCI
- 16.69
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.