Training a neural-network based intrusion detector to recognize novel attacks

Abstract

While many commercial intrusion detection systems (IDS) are deployed, the protection they afford is modest. State-of-the-art IDS produce voluminous alerts, most false alarms, and function mainly by recognizing the signatures of known attacks so that novel attacks slip past them. Attempts have been made to create systems that recognize the signature of "normal," in the hope that they will then detect attacks, known or novel. These systems are often confounded by the extreme variability of nominal behavior. The paper describes an experiment with an IDS composed of a hierarchy of neural networks (NN) that functions as a true anomaly detector. This result is achieved by monitoring selected areas of network behavior, such as protocols, that are predictable in advance. While this does not cover the entire attack space, a considerable number of attacks are carried out by violating the expectations of the protocol/operating system designer. Within this focus, the NNs are trained using data that spans the entire normal space. These detectors are able to recognize attacks that were not specifically presented during training. We show that using small detectors in a hierarchy gives a better result than a single large detector. Some techniques can be used not only to detect anomalies, but to distinguish among them.