Evaluating Insider Threat Detection Workflow Using Supervised and Unsupervised Learning
Open accessAbstract
Insider threat is a prominent cyber-security danger faced by organizations and companies. In this research, we study and evaluate an insider threat detection workflow using supervised and unsupervised learning algorithms. To this end, we study data exploration and analysis, anomaly detection and malicious behaviour classification on a publicly available data set. We evaluate several supervised and unsupervised learning algorithms - HMM, SOM, and DT - using this workflow.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hybrid pipelines combining NLP and DL offer a promising avenue for real-time log analysis, addressing static nature of datasets like CERT (17-month window) and delay in models like LSTM-RNN (93.85% accuracy).
“Hybrid pipelines combining NLP and DL offer a promising avenue for real-time log analysis, addressing the static nature of datasets like CERT (17-month window [7]) and the delay in models like LSTM-RNN (93.85% accuracy [21]).”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The citation [7] is attached to the CERT dataset’s claimed 17-month window, not the LSTM-RNN result. The paper’s “A. CERT insider threat dataset” section says that 18 months of data were extracted, contrary to the cited figure. The sentence’s proposal for a hybrid pipeline is not directly attributed to this source. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredComparing CERT 4.1 and 4.2, the latter's increased malicious activity (7% vs. lower rates in 4.1) and expanded feature set make it more suitable for dense threat detection.
“Comparing 4.1 and 4.2, the latter's increased malicious activity (7% vs. lower rates in 4.1) and expanded feature set make it more suitable for dense threat detection, as evidenced by higher accuracy in recent models [6,7].”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT insider threat dataset section says release 4.2 has significantly more insider-threat incidents than other releases, allowing tests across more diverse scenarios. It does not give a 7% malicious-activity rate, identify an expanded feature set relative to 4.1, or compare model accuracy across the two releases. The Conclusion instead reports that malicious instances comprise 0.39% of weekly and 0.29% of daily preprocessed data in 4.2. (LLM, conf=0.88)
Experimental — accuracy not yet measuredCERT Dataset is best suited for log-based and behavioral analysis due to comprehensive coverage of user activities, supporting NER and graph-based models with 93.97% accuracy.
“The CERT Dataset is best suited for log-based and behavioral analysis due to its comprehensive coverage of user activities, supporting techniques like NER and graph-based models with 93.97% accuracy [7,39].”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT insider threat dataset section describes records of logins, email, web, file, and device activity, supporting its use for log-based behavioral analysis. The paper evaluates hidden Markov models, self-organizing maps, and decision trees; it does not support the attributed NER or graph-based models, a 93.97% accuracy for them, or the claim that CERT is 'best suited' for this analysis. Page numbers are unavailable in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measuredStudies using CERT 4.2 (7% malicious activity) and Contrastive Learning (100% precision on CERT) predominantly frame detection as a two-class problem, achieving high metrics but overlooking diversity of insider threat types.
“The manuscript's studies, such as those using CERT 4.2 (7% malicious activity [7]) and Contrastive Learning (100% precision on CERT [16]), predominantly frame detection as a two-class problem, achieving high metrics but overlooking the diversity of insider threat types.”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT insider threat dataset section says the study used release 4.2. But the passage labeled V. Conclusion reports that insider-threat instances were 0.39% of weekly and 0.29% of daily preprocessed data, not 7%. The contrastive-learning result is attached to a different citation, and the broader characterization is the citing authors’ framing. (LLM, conf=0.89)
Experimental — accuracy not yet measuredAdvanced models such as BERT, RoBERTa, and DD-GCN achieve high accuracies (e.g., >94% on CERT 4.2 and 96.26%-96.70% on CERT 6.2), leveraging deep learning to capture intricate patterns in textual data like emails and logs.
“Advanced models such as BERT, RoBERTa, and Dual-Domain Graph Convolutional Networks (DD-GCN) achieve high accuracies (e.g., >94% on CERT 4.2 [25] and 96.26%-96.70% on CERT 6.2 [7]), leveraging deep learning to capture intricate patterns in textual data like emails and logs.”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT insider threat dataset section says the study used release 4.2, and Table I reports accuracies above 94% for its C4.5 decision tree. Those figures do not establish the attributed performance of BERT, RoBERTa, or DD-GCN: the paper evaluates HMM, SOM, and decision-tree methods, not those deep-learning models. The CERT 6.2 figure is attached to a different citation in the citing sentence and is not assessed here. (LLM, conf=0.87)
Experimental — accuracy not yet measured
CERT's simulated logs (32 million actions) enable high accuracies (e.g., 96.70%) but fail to capture dynamic real-world behaviors, with accuracy dropping to 73.4% when applied to live data.
“For instance, CERT's simulated logs (32 million actions [7]) enable high accuracies (e.g., 96.70% [7]) but fail to capture dynamic real-world behaviors, with accuracy dropping to 73.4% when applied to live data [3].”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT dataset and decision-tree results sections report shifts and drifts in user behavior and say a conventional supervised model may not adapt well. They evaluate simulated CERT data, however, not live data; the reported accuracy values do not include a drop to 73.4%. The paper also does not establish that CERT logs fail to capture dynamic real-world behavior. (LLM, conf=0.82)
Experimental — accuracy not yet measuredThe CERT dataset's 17-month static window exemplifies limitation, as real-time anomalies require continuous adaptation, a capability few models demonstrate.
“The CERT dataset's 17-month static window [7] exemplifies this limitation, as real-time anomalies (e.g., unauthorized access spikes) require continuous adaptation, a capability few models demonstrate.”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT insider threat dataset section describes a finite dataset, but says the analysis extracted 18 months of data, not a 17-month window. The Conclusion also refers to 72 weeks of data, approximately 17 months. The citation supports the use of a bounded CERT dataset, but not the precise duration stated. (LLM, conf=0.84)
Experimental — accuracy not yet measuredCERT r4.2's scale in simulating insider scenarios enabled 96.70% accuracy in hybrid models.
“CERT r4.2 as the most prevalent (used in 40.9% of studies for its scale in simulating insider scenarios, enabling 96.70% accuracy in hybrid models [7]”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT insider threat dataset section (page unavailable) says release 4.2 has more insider-threat incidents than other releases, allowing tests across a more diverse set of scenarios. It does not report that CERT r4.2 was used in 40.9% of studies or that hybrid models achieved 96.70% accuracy; the reported results concern HMM, SOM, and decision-tree methods. (LLM, conf=0.88)
Experimental — accuracy not yet measuredCurrent NLP insider threat detection relies on synthetic datasets like CERT 4.2 and privacy-constrained Enron.
“The reliance on synthetic datasets like CERT 4.2 (96.26% accuracy [7]) and privacy-constrained Enron (94.28% [4]) underscores the need for better datasets.”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The CERT insider threat dataset section identifies release 4.2 as a synthesized dataset used in this study. The Feature extraction section says the study excluded email, web, and file content from analysis, so it does not establish reliance by NLP detection. The reported accuracy results also do not substantiate the cited 96.26% figure; the Enron figure is attached to a different citation. (LLM, conf=0.88)
Experimental — accuracy not yet measuredRQ2's dataset analysis corroborates Glasser and Lindauer on synthetic data challenges and identifies new gaps in real-time logs.
“RQ2's dataset analysis (CERT, Enron [4,7]) corroborates Glasser and Lindauer [web:9] on synthetic data challenges, though it identifies new gaps in real-time logs [12].”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The Data preprocessing section (page unavailable) describes challenges in combining continuously generated logs from multiple sources, and the Conclusion identifies streaming-data analysis as future work. The paper therefore supports a gap related to timely log analysis, but does not present an RQ2 dataset analysis or identify a new gap specifically in real-time logs. The Glasser and Lindauer comparison is attached to a different citation. (LLM, conf=0.82)
Experimental — accuracy not yet measuredDual-Domain Graph Convolutional Networks ensemble graph-based ML with DL, achieving 96.26%-96.70% accuracy on CERT datasets by fusing behavioral and log features.
“For instance, Dual-Domain Graph Convolutional Networks (DD-GCN) ensemble graph-based ML with DL, achieving 96.26%-96.70%accuracy on CERT datasets by fusing behavioral and log features [7].”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The Learning algorithms section describes self-organizing maps, hidden Markov models, and decision trees—not DD-GCN. Table I reports accuracy results on the CERT dataset, but not 96.26%–96.70%. Searches for DD-GCN and those figures found no passage addressing the claimed method or result. (LLM, conf=0.76)
Experimental — accuracy not yet measuredThe complexity of current NLP models, such as BERT (>94% accuracy) and DD-GCN (96.26%-96.70%), poses challenges for real-time deployment and trust, necessitating development of lightweight and interpretable alternatives.
“The complexity of current NLP models, such as BERT (>94% accuracy [25]) and DD-GCN (96.26%-96.70%[7]), poses challenges for real-time deployment and trust, necessitating the development of lightweight and interpretable alternatives.”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [7]
The Learning algorithms section studies self-organizing maps, hidden Markov models, and decision trees, while the CERT insider threat dataset results report their detection performance. A search of our copy found no DD-GCN result, 96.26%–96.70% accuracy, or discussion of that model’s real-time deployment or trust challenges; the cited paper therefore does not verify the part attached to [7]. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1992) · Cited by 23,431 · see citing works
Maniezzo, V., & calcolare, P. (2004) · Cited by 393 · see citing works
An artificial immune system architecture for computer security applications
Harmer, P. K., Williams, P. D., Gunsch, G. H., et al. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2002.1011540 · Cited by 356 · see citing works
Bridging the Gap: A Pragmatic Approach to Generating Insider Threat Data
Glasser, J., & Lindauer, B. (2013) · 2013 IEEE Security and Privacy Workshops · doi:10.1109/spw.2013.37 · Cited by 344 · see citing works
A New Take on Detecting Insider Threats: Exploring the Use of Hidden Markov Models
Rashid, T., Agrafiotis, I., & Nurse, J. R. C. (2016) · Proceedings of the 8th ACM CCS International Workshop on Managing Insider Security Threats · doi:10.1145/2995959.2995964 · Cited by 152 · see citing works
Detecting insider threats in a real corporate database of computer usage activity
Senator, T. E., Goldberg, H. G., Memory, A., Young, W. T., Rees, B., Pierce, R., Huang, D., Reardon, M., Bader, D. A., Chow, E., et al. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2488213 · Cited by 132 · see citing works
Benchmarking the Effect of Flow Exporters and Protocol Filters on Botnet Traffic Classification
Haddadi, F., & Zincir-Heywood, N. (2014) · IEEE Systems Journal · doi:10.1109/jsyst.2014.2364743 · Cited by 75 · see citing works
Towards a Conceptual Model and Reasoning Structure for Insider Threat Detection
Legg, P., Moffat, N., Nurse, J. R. C., Happa, J., Agrafiotis, I., Goldsmith, M. H., & Creese, S. (2013) · Kent Academic Repository (University of Kent) · doi:10.22667/jowua.2013.12.31.020 · Cited by 67 · see citing works
Data analytics on network traffic flows for botnet behaviour detection
Le, D. C., Zincir-Heywood, N., & Heywood, M. I. (2016) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7850078 · Cited by 42 · see citing works
Self-citationKorczyński, M., Hamieh, A., Huh, J. H., Holm, H., Rajagopalan, S. R., & Fefferman, N. H. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7497768 · Cited by 36 · see citing works
Padayachee, K. (2016) · Decision Support Systems · doi:10.1016/j.dss.2016.09.012 · Cited by 31 · see citing works
Common Sense Guide to Mitigating Insider Threats, Fifth Edition
Collins, M. L., Theis, M., Trzeciak, R., Strozer, J., Clark, J., Costa, D., Cassidy, T., Albrethsen, M., & Moore, A. (2016) · Figshare · doi:10.1184/r1/12890918 · Cited by 19 · see citing works
Packet Momentum for Identificationof Anonymity Networks
Shahbar, K., & Zincir-Heywood, N. (2017) · Journal of Cyber Security and Mobility · doi:10.13052/jcsm2245-1439.612 · Cited by 17 · see citing works
Explaining and Aggregating Anomalies to Detect Insider Threats
Goldberg, H. G., Young, W. T., Memory, A., & Senator, T. E. (2016) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2016.344 · Cited by 12 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [3]3. T. Rashid, I. Agrafiotis, and J. R. Nurse, "A new take on detecting insider threats," in International Workshop on Managing Insider Security Threats, 2016, pp. 47-56.
- [4]4. G. Gavai et al., "Supervised and unsupervised methods to detect insider threat from enterprise social and online activ- ity data," Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications (JoWUA), vol. 6, no. 4, pp. 47-63, December 2015.
- [16]16. T. Kohonen, Self-Organizing Maps, ser. Springer Series in Information Sciences, 2001, vol. 30.
Cited by
Sorted by citations
Deep learning for insider threat detection: Review, challenges and opportunities
Yuan, S., & Wu, X. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102221 · Cited by 271 · see citing works
Analyzing Data Granularity Levels for Insider Threat Detection Using Machine Learning
Le, D. C., Zincir-Heywood, N., & Heywood, M. I. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.2967721 · Cited by 191 · see citing works
Self-citationAnomaly Detection for Insider Threats Using Unsupervised Ensembles
Le, D. C., & Zincir-Heywood, N. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3071928 · Cited by 147 · see citing works
Self-citationAl-Shehari, T. A., Rosaci, D., Al-Razgan, M., Alfakih, T., Kadrie, M., Afzal, H., & Nawaz, R. (2024) · IEEE Access · doi:10.1109/access.2024.3373694 · Cited by 51 · see citing works
“In [13], a model for categorizing and presenting malicious and normal users’ activities within an organization was proposed.”
BackgroundInsider threat detection using supervised machine learning algorithms
Manoharan, P., Yin, J., Wang, H., Zhang, Y., & Ye, W. (2024) · Telecommunication Systems · doi:10.1007/s11235-023-01085-3 · Cited by 34 · see citing works
Double layer detection of internal threat in enterprise systems based on deep learning
He, D., Lv, X., Xu, X., Chan, S., & Choo, K.-K. R. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3372771 · Cited by 31 · see citing works
Exploring Adversarial Properties of Insider Threat Detection
Le, D. C., & Zincir-Heywood, N. (2020) · 2020 IEEE Conference on Communications and Network Security (CNS) · doi:10.1109/cns48642.2020.9162254 · Cited by 17 · see citing works
Self-citationThe Insider Threat Detection Method of University Website Clusters Based on Machine Learning
Li, Y., & Su, Y. (2023) · 2023 6th International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd57115.2023.10206282 · Cited by 14 · see citing works
Insider Attack: Internal Cyber Attack Detection Using Machine Learning
Varsha Suresh, P., & Lalitha Madhavu, M. (2021) · 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt51525.2021.9579549 · Cited by 13 · see citing works
A systematic review on insider threat detection using natural language processing
Kundiya, K., & Haribhakta, Y. (2025) · International Journal of Information Security · doi:10.1007/s10207-025-01145-6 · Cited by 9 · see citing works
BS-Net: A Behavior Sequence Network for Insider Threat Detection
Zhu, D., Sun, H., Li, N., Mi, B., & Xi, T. (2021) · 2021 IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc53001.2021.9631445 · Cited by 3 · see citing works
- Authors
- Le, D. C., & Zincir-Heywood, A. N.
- Affiliations
- Dalhousie University (CA)
- ORCID
- Le, Duc C.
- Venue
- 2018 IEEE Security and Privacy Workshops (SPW)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 270-275
- OpenAlex
- W2887799638
- Semantic Scholar
- 3ec4b77336efe11d9ce63c3734775468abb86c7d
- Cited by
- 66
- References
- 14
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 19 · 2023: 12 · 2022: 11 · 2021: 8 · 2020: 7 · 2019: 3 · 2018: 1
- FWCI
- 3.93
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- IEEE Security and Privacy Workshops (SPW) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Keywords
- insider threat detection, anomaly detection, hidden Markov models, self-organizing maps, decision tree, supervised learning, unsupervised learning, cybersecurity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review