Insider threat prediction based on unsupervised anomaly detection scheme for proactive forensic investigation

Abstract

The complexity, concealment and infrequency of malicious internal actions make it difficult to detect insider threats. In the process of traditional reactive forensic investigation, analysis and interpretation of the digital evidence are performed after a crime has been committed. Even if insiders can be detected, they have already caused huge damage. In this paper, we propose a novel general unsupervised anomaly detection scheme based on cascaded autoencoders (CAEs) and joint optimization network. Our core idea is to utilize CAEs to do data purification among unlabeled digital evidence, then jointly optimize the dimension reduction and density estimation network to avoid sub-optimal problems. Based on this scheme, we design an end-to-end insider threat prediction framework for proactive forensic investigation, through which we can make real time response to prevent the harmful influences of insider threats in advance. We extract the tractable and scalable feature representation automatically through the data driven Bidirectional Long Short-Term Memory (BiLSTM) feature extractor, waiving the time-consuming and customarily expert dependable feature engineering work. Additionally, a hypergraph correction module is applied to solve the commonly existed relatively high false positive rate problem in insider threat detection. We evaluate our scheme and framework on public benchmark datasets. The empirical experiments demonstrate that our models outperform state-of-the-art unsupervised methods.