CoChat

Anomaly detection: A survey

Chandola, Varun, Banerjee, Arindam, Kumar, Vipin · 2009 · ACM Computing Surveys

The DOI matches its Crossref record; 392 of the 395 references it cites were found in a registry or an index.

Abstract

Anomaly detection is an important problem that has been researched within diverse research areas and application domains. Many anomaly detection techniques have been specifically developed for certain application domains, while others are more generic. This survey tries to provide a structured and comprehensive overview of the research on anomaly detection. We have grouped existing techniques into different categories based on the underlying approach adopted by each technique. For each category we have identified key assumptions, which are used by the techniques to differentiate between normal and anomalous behavior. When applying a given technique to a particular domain, these assumptions can be used as guidelines to assess the effectiveness of the technique in that domain. For each category, we provide a basic anomaly detection technique, and then show how the different existing techniques in that category are variants of the basic technique. This template provides an easier and more succinct understanding of the techniques belonging to each category. Further, for each category, we identify the advantages and disadvantages of the techniques in that category. We also provide a discussion on the computational complexity of the techniques since it is an important issue in real application domains. We hope that this survey will provide a better understanding of the different directions in which research has been done on this topic, and how techniques developed in one area can be applied in domains for which they were not intended to begin with.

Overview

Integrity signals

Registry

Crossref · 10.1145/1541880.1541882

Cites

392 of 392 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 392 references

Research works printed but not linked: 35

  1. Aeyels , D. 1991. On the dynamic behaviour of the novelty detector and the novelty filter . In Analysis of Controlled Dynamical Systems: Progress in Systems and Control Theory , B. Bonnard, B. Bride, J. Gauthier, and I. Kupka, Eds. vol. 8 . Springer , Berlin , 1--10. Aeyels, D. 1991. On the dynamic behaviour of the novelty detector and the novelty filter. In Analysis of Controlled Dynamical Systems: Progress in Systems and Control Theory, B. Bonnard, B. Bride, J. Gauthier, and I. Kupka, Eds. vol. 8. Springer, Berlin, 1--10.
  2. Allan , J. , Carbonell , J. , Doddington , G. , Yamron , J. , and Yang , Y . 1998. Topic detection and tracking pilot study . In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop. 194--218 . Allan, J., Carbonell, J., Doddington, G., Yamron, J., and Yang, Y. 1998. Topic detection and tracking pilot study. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop. 194--218.
  3. Baker , D. , Hofmann , T. , McCallum , A. , and Yang , Y . 1999. A hierarchical probabilistic model for novelty detection in text . In Proceedings of the International Conference on Machine Learning. Baker, D., Hofmann, T., McCallum, A., and Yang, Y. 1999. A hierarchical probabilistic model for novelty detection in text. In Proceedings of the International Conference on Machine Learning.
  4. Beckman , R. J. and Cook , R. D. 1983 . Outlier...s . Technometrics 25 , 2, 119 -- 149 . Beckman, R. J. and Cook, R. D. 1983. Outlier...s. Technometrics 25, 2, 119--149.
  5. Bronstein , A. , Das , J. , Duro , M. , Friedrich , R. , Kleyner , G. , Mueller , M. , Singhal , S. , and Cohen , I . 2001. Bayesian networks for detecting anomalies in Internet-based services . In Proceedings of the International Symposium on Integrated Network Management. Bronstein, A., Das, J., Duro, M., Friedrich, R., Kleyner, G., Mueller, M., Singhal, S., and Cohen, I. 2001. Bayesian networks for detecting anomalies in Internet-based services. In Proceedings of the International Symposium on Integrated Network Management.
  6. Caudell , T. and Newman , D . 1993. An adaptive resonance architecture to define normality and detect novelties in time series and databases . In Proceedings of the IEEE World Congress on Neural Networks. IEEE, 166--176 . Caudell, T. and Newman, D. 1993. An adaptive resonance architecture to define normality and detect novelties in time series and databases. In Proceedings of the IEEE World Congress on Neural Networks. IEEE, 166--176.
  7. Chandola V. Eilertson E. Ertoz L. Simon G. and Kumar V. 2006. Data mining for cyber security. In Data Warehousing and Data Mining Techniques for Computer Security A. Singhal Ed. Springer. Chandola V. Eilertson E. Ertoz L. Simon G. and Kumar V. 2006. Data mining for cyber security. In Data Warehousing and Data Mining Techniques for Computer Security A. Singhal Ed. Springer.
  8. 10.5555/784589.784654 · doi:10.5555/784589.784654
  9. Ertoz L. Eilertson E. Lazarevic A. Tan P.-N. Kumar V. Srivastava J. and Dokas P. 2004. MINDS—Minnesota Intrusion Detection System. In Data Mining—Next Generation Challenges and Future Directions. MIT Press. Ertoz L. Eilertson E. Lazarevic A. Tan P.-N. Kumar V. Srivastava J. and Dokas P. 2004. MINDS—Minnesota Intrusion Detection System. In Data Mining—Next Generation Challenges and Future Directions. MIT Press.
  10. Eskin , E. , Arnold , A. , Prerau , M. , Portnoy , L. , and Stolfo , S . 2002. A geometric framework for unsupervised anomaly detection . In Proceedings of the Conference on Applications of Data Mining in Computer Security. Kluwer Academics, 78--100 . Eskin, E., Arnold, A., Prerau, M., Portnoy, L., and Stolfo, S. 2002. A geometric framework for unsupervised anomaly detection. In Proceedings of the Conference on Applications of Data Mining in Computer Security. Kluwer Academics, 78--100.
  11. Ghosh , A. K. , Schwartzbard , A. , and Schatz , M . 1999b. Using program behavior profiles for intrusion detection . In Proceedings of the SANS 3rd Conference and Workshop on Intrusion Detection and Response. Ghosh, A. K., Schwartzbard, A., and Schatz, M. 1999b. Using program behavior profiles for intrusion detection. In Proceedings of the SANS 3rd Conference and Workshop on Intrusion Detection and Response.
  12. 10.5555/2993952.2994022 · doi:10.5555/2993952.2994022
  13. Hickinbotham , S. J. and Austin , J . 2000b. Novelty detection in airframe strain data . In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. vol. 6 . 24--27. Hickinbotham, S. J. and Austin, J. 2000b. Novelty detection in airframe strain data. In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. vol. 6. 24--27.
  14. 10.5555/646257.685234 · doi:10.5555/646257.685234
  15. 10.5555/1298081.1298084 · doi:10.5555/1298081.1298084
  16. Hu , W. , Liao , Y. , and Vemuri , V. R . 2003. Robust anomaly detection using support vector machines . In Proceedings of the International Conference on Machine Learning. Morgan Kaufmann Publishers Inc., 282--289 . Hu, W., Liao, Y., and Vemuri, V. R. 2003. Robust anomaly detection using support vector machines. In Proceedings of the International Conference on Machine Learning. Morgan Kaufmann Publishers Inc., 282--289.
  17. Huber , P. 1974. Robust Statistics . Wiley , New York . Huber, P. 1974. Robust Statistics. Wiley, New York.
  18. Kohonen , T. , Ed. 1997. Self-Organizing Maps . Springer-Verlag . Kohonen, T., Ed. 1997. Self-Organizing Maps. Springer-Verlag.
  19. 10.5555/645328.650009 · doi:10.5555/645328.650009
  20. Manevitz , L. M. and Yousef , M . 2000. Learning from positive data for document classification using neural networks . In Proceedings of the 2nd Bar-Ilan Workshop on Knowledge Discovery and Learning. Manevitz, L. M. and Yousef, M. 2000. Learning from positive data for document classification using neural networks. In Proceedings of the 2nd Bar-Ilan Workshop on Knowledge Discovery and Learning.
  21. Manson , G. 2002 . Identifying damage sensitive, environment insensitive features for damage detection . In Proceedings of IES Conference. Manson, G. 2002. Identifying damage sensitive, environment insensitive features for damage detection. In Proceedings of IES Conference.
  22. 10.5555/647593.728889 · doi:10.5555/647593.728889
  23. Marsland , S. , Nehmzow , U. , and Shapiro , J . 1999. A model of habituation applied to mobile robots . In Proceedings of Towards Intelligent Mobile Robots Conference. Department of Computer Science , Manchester University, Technical rep. UMCS-99-3-1. Marsland, S., Nehmzow, U., and Shapiro, J. 1999. A model of habituation applied to mobile robots. In Proceedings of Towards Intelligent Mobile Robots Conference. Department of Computer Science, Manchester University, Technical rep. UMCS-99-3-1.
  24. Mingming , N. Y. 2000 . Probabilistic networks with undirected links for anomaly detection . In Proceedings of the IEEE Systems, Man, and Cybernetics Information Assurance and Security Workshop. 175--179 . Mingming, N. Y. 2000. Probabilistic networks with undirected links for anomaly detection. In Proceedings of the IEEE Systems, Man, and Cybernetics Information Assurance and Security Workshop. 175--179.
  25. Motulsky , H. 1995 . Intuitive Biostatistics: Choosing a Statistical Test . Oxford University Press , Chapter 37. Motulsky, H. 1995. Intuitive Biostatistics: Choosing a Statistical Test. Oxford University Press, Chapter 37.
  26. Saunders , R. and Gero , J . 2000. The importance of being emergent . In Proceedings of the Conference on Artificial Intelligence in Design. Saunders, R. and Gero, J. 2000. The importance of being emergent. In Proceedings of the Conference on Artificial Intelligence in Design.
  27. Scarth , G. , McIntyre , M. , Wowk , B. , and Somorjai , R . 1995. Detection of novelty in functional images using fuzzy clustering . In Proceedings of the 3rd Meeting of the International Society for Magnetic Resonance in Medicine. 238 . Scarth, G., McIntyre, M., Wowk, B., and Somorjai, R. 1995. Detection of novelty in functional images using fuzzy clustering. In Proceedings of the 3rd Meeting of the International Society for Magnetic Resonance in Medicine. 238.
  28. Scott , S. L. 2001 . Detecting network intrusion using a Markov modulated nonhomogeneous Poisson Process . Journal of the American Statistical Association. Scott, S. L. 2001. Detecting network intrusion using a Markov modulated nonhomogeneous Poisson Process. Journal of the American Statistical Association.
  29. Smith , R. , Bivens , A. , Embrechts , M. , Palagiri , C. , and Szymanski , B . 2002. Clustering approaches for anomaly-based intrusion detection . In Proceedings of the Intelligent Engineering Systems through Artificial Neural Networks. ASME Press, 579--584 . Smith, R., Bivens, A., Embrechts, M., Palagiri, C., and Szymanski, B. 2002. Clustering approaches for anomaly-based intrusion detection. In Proceedings of the Intelligent Engineering Systems through Artificial Neural Networks. ASME Press, 579--584.
  30. Sohn , H. , Worden , K. , and Farrar , C . 2001. Novelty detection under changing environmental conditions . In Proceedings of the 8th Annual SPIE International Symposium on Smart Structures and Materials. Sohn, H., Worden, K., and Farrar, C. 2001. Novelty detection under changing environmental conditions. In Proceedings of the 8th Annual SPIE International Symposium on Smart Structures and Materials.
  31. Torr , P. and Murray , D . 1993. Outlier detection and motion segmentation . In Proceedings of the SPIE. Sensor Fusion VI, S. Schenker, Ed. vol. 2059 . 432--443. Torr, P. and Murray, D. 1993. Outlier detection and motion segmentation. In Proceedings of the SPIE. Sensor Fusion VI, S. Schenker, Ed. vol. 2059. 432--443.
  32. 10.5555/645838.670720 · doi:10.5555/645838.670720
  33. Vasconcelos , G. , Fairhurst , M. , and Bisset , D . 1994. Recognizing novelty in classification tasks . In Proceedings of the Neural Information Processing Systems Workshop on Novelty Detection and Adaptive Systems Monitoring. Vasconcelos, G., Fairhurst, M., and Bisset, D. 1994. Recognizing novelty in classification tasks. In Proceedings of the Neural Information Processing Systems Workshop on Novelty Detection and Adaptive Systems Monitoring.
  34. Yairi , T. , Kato , Y. , and Hori , K . 2001. Fault detection by mining association rules from housekeeping data . In Proceedings of the International Symposium on Artificial Intelligence, Robotics and Automation in Space. Yairi, T., Kato, Y., and Hori, K. 2001. Fault detection by mining association rules from housekeeping data. In Proceedings of the International Symposium on Artificial Intelligence, Robotics and Automation in Space.
  35. 10.5555/3225635.3225781 · doi:10.5555/3225635.3225781

Cited by

Items 701–800 of 1,084

Metadata

Authors
Chandola, V., Banerjee, A., & Kumar, V.
Affiliations
University of Minnesota (US)
Venue
ACM Computing Surveys
Published
2009-07
Volume
41
Issue
3
Pages
1-58
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W2122646361
Cited by
11,687
References
395
Citations by year
2026: 1020 · 2025: 1095 · 2024: 905 · 2023: 921 · 2022: 816 · 2021: 982 · 2020: 1052 · 2019: 889 · 2018: 752 · 2017: 604 · 2016: 569 · 2015: 466 · 2014: 463 · 2013: 399 · 2012: 319
FWCI
269.95
Type
journal article
Language
en
Open access
closed
Locations
ACM Computing Surveys (publishedVersion)
Funders
  • Advanced Cyberinfrastructure (ACI-0325949)
  • National Science Foundation (ACI-0325949)
  • National Aeronautics and Space Administration (NNX08AC36A)
  • Division of Information and Intelligent Systems (CNS-0551551IIS-0713227IIS-0308264)
  • Division of Computer and Network Systems (CNS-0551551IIS-0713227IIS-0308264)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Immune Systems Applications
Keywords
anomaly detection, classification methods, computational complexity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.