Insider Threat Detection Using Bidirectional Encoder Representations From Transformers and Optimized AdaBoost Classifier
Abstract
Cybersecurity plays an increasingly vital role in contemporary times, particularly as large companies face escalating threats and substantial losses, with ransomware attacks standing out as one of the more sophisticated forms of intrusion. In such attacks, hackers infiltrate systems, encrypt crucial company data, and then demand a ransom for data decryption or the restoration of control over the system. The application of natural language processing (NLP) emerges as a promising avenue, offering the potential to analyze attack patterns and recognize malicious intentions before actual attempts occur. Given the inherent specificity of each system, fine-tuning AI algorithms with special settings becomes imperative for optimal efficiency. This research delves into the synergy of NLP and robust classification algorithms to detect cyber attacks through website content analysis. Recognizing that classifier performance hinges on the judicious selection of hyperparameters, we explore a modified version of the recently introduced red fox algorithm (RFO). This adaptation is combined with bidirectional encoder representations from transformers to identify malicious intent in email content. The proposed approach undergoes testing on a real-world dataset, with the best models demonstrating an accuracy of 0.975822 suggesting viability.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Kumpf et al. (2024) introduced an approach combining modified Red Fox algorithm with BERT to detect cyber-attacks through website content analysis, achieving accuracy of 0.975822 on a real-world dataset.
“Kumpf et al. (2024) [54] introduced a novel approach that combines a modified Red Fox (RFO) algorithm with bidirectional encoder representations from transformers to detect cyber-attacks through website content analysis, achieving an accuracy of 0.975822 on a real-world dataset.”
From A systematic review on insider threat detection using natural language processing (2025) · cites this paper as [54]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
The Whale Optimization Algorithm
Mirjalili, S., & Lewis, A. (2016) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2016.01.008 · Cited by 14,829 · see citing works
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237 · see citing works
A review on genetic algorithm: past, present, and future
Katoch, S., Chauhan, S. S., & Kumar, V. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10139-6 · Cited by 4,655 · see citing works
Particle Swarm Optimization Algorithm and Its Applications: A Systematic Review
Gad, A. G. (2022) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-021-09694-4 · Cited by 2,010 · see citing works
Hastie, T. J., Rosset, S., Zhu, J., & Zou, H. (2009) · Statistics and Its Interface · doi:10.4310/sii.2009.v2.n3.a8 · Cited by 1,907 · see citing works
Reptile Search Algorithm (RSA): A nature-inspired meta-heuristic optimizer
Abualigah, L., Elaziz, M. A., Sumari, P., Geem, Z. W., & Gandomi, A. H. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116158 · Cited by 1,443 · see citing works
Crayfish optimization algorithm
Jia, H., Rao, H., Wen, C., & Mirjalili, S. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10567-4 · Cited by 644 · see citing works
Red fox optimization algorithm
Połap, D., & Woźniak, M. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114107 · Cited by 464 · see citing works
Bridging the Gap: A Pragmatic Approach to Generating Insider Threat Data
Glasser, J., & Lindauer, B. (2013) · 2013 IEEE Security and Privacy Workshops · doi:10.1109/spw.2013.37 · Cited by 344 · see citing works
Ileberi, E., Sun, Y., & Wang, Z. (2021) · IEEE Access · doi:10.1109/access.2021.3134330 · Cited by 244 · see citing works
Ransomware payments in the Bitcoin ecosystem
Paquet-Clouston, M., Haslhofer, B., & Dupont, B. (2019) · Journal of Cybersecurity · doi:10.1093/cybsec/tyz003 · Cited by 240 · see citing works
Kunang, Y. N., Nurmaini, S., Stiawan, D., & Suprapto, B. Y. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2021.102804 · Cited by 236 · see citing works
COLSHADE for Real-World Single-Objective Constrained optimization Problems
Gurrola-Ramos, J., Hernández-Aguirre, A., & Dalmau, O. (2020) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec48606.2020.9185583 · Cited by 193 · see citing works
Multi-Head Attention: Collaborate Instead of ConcatenatePreprint
Cordonnier, J.-B., Loukas, A., & Jaggi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16362 · Cited by 180 · see citing works
Yang, X., & Słowik, A. (2020) · CRC Press eBooks · doi:10.1201/9780429422614-13 · Cited by 150 · see citing works
Savanović, N., Toskovic, A., Petrović, A., Zivkovic, M. Z., Damaševičius, R., Jovanović, L., Bačanin, N., & Nikolić, B. (2023) · Sustainability · doi:10.3390/su151612563 · Cited by 105 · see citing works
Self-citationACO-IM: maximizing influence in social networks using ant colony optimization
Singh, S. S., Singh, K., Kumar, A., & Biswas, B. (2019) · Soft Computing · doi:10.1007/s00500-019-04533-y · Cited by 100 · see citing works
Basha, J., Bačanin, N., Vukobrat, N., Zivkovic, M. Z., Venkatachalam, K., Hubálovský, Š., & Trojovský, P. (2021) · Sensors · doi:10.3390/s21196654 · Cited by 100 · see citing works
Self-citationMalakouti, S. M. (2023) · Case Studies in Chemical and Environmental Engineering · doi:10.1016/j.cscee.2023.100351 · Cited by 96 · see citing works
Large-Scale Differentially Private BERT
Anil, R., Ghazi, B., Gupta, V., Kumar, R., & Manurangsi, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.484 · Cited by 68 · see citing works
XGBoost Hyperparameters Tuning by Fitness-Dependent Optimizer for Network Intrusion Detection
Zivkovic, M. Z., Jovanović, L., Ivanovic, M., Bačanin, N., Strumberger, I., & Mani, J. P. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-2130-8_74 · Cited by 67 · see citing works
Self-citationReptile Search Algorithm: Theory, Variants, Applications, and Performance Evaluation
Sasmal, B., Hussien, A. G., Das, A., Dhal, K. G., & Saha, R. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09990-1 · Cited by 55 · see citing works
The XGBoost Model for Network Intrusion Detection Boosted by Enhanced Sine Cosine Algorithm
AlHosni, N., Jovanović, L., Antonijević, M., Bukumira, M., Zivkovic, M. Z., Strumberger, I., Mani, J. P., & Bačanin, N. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-12413-6_17 · Cited by 50 · see citing works
Self-citationNLP-based digital forensic investigation platform for online communications
Sun, D., Zhang, X., Choo, K. R., Hu, L. Q., & Wang, F. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102210 · Cited by 47 · see citing works
An emperor penguin optimizer application for medical diagnostics
Jovanović, L., Zivkovic, M. Z., Antonijević, M., Jovanović, D., Ivanovic, M., & Jassim, H. S. (2022) · Zooming Innovation in Consumer Technologies Conference (ZINC) · doi:10.1109/zinc55034.2022.9840612 · Cited by 46 · see citing works
Self-citationOnline AdaBoost-based methods for multiclass problems
Santos, S. G. T. D. C., & de Barros, R. S. M. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09696-6 · Cited by 43 · see citing works
Tuning XGBoost by Planet Optimization Algorithm: An Application for Diabetes Classification
Jovanović, L., Djuric, M., Zivkovic, M. Z., Jovanović, D., Strumberger, I., Antonijević, M., Budimirović, N., & Bačanin, N. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-7753-4_60 · Cited by 37 · see citing works
Self-citationTuning Extreme Learning Machine by Hybrid Planet Optimization Algorithm for Diabetes Classification
Jovanović, L., Hajdarevic, Z., Jovanović, D., Jassim, H. S., Strumberger, I., Bačanin, N., Zivkovic, M. Z., & Antonijević, M. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-19-9379-4_3 · Cited by 24 · see citing works
Self-citationOszust, M., Sroka, G., & Cymerys, K. (2021) · Information Sciences · doi:10.1016/j.ins.2021.04.082 · Cited by 9 · see citing works
Cyberwar: Attacking Critical Infrastructure
Stoddart, K. (2022) · Springer eBooks · doi:10.1007/978-3-030-97299-8_3 · Cited by 9 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Cited by
Sorted by citations
Dakić, P., Zivkovic, M. Z., Jovanović, L., Bačanin, N., Antonijević, M., Kaljević, J., & Šimić, V. (2024) · Scientific Reports · doi:10.1038/s41598-024-73932-5 · Cited by 53 · see citing works
Self-citationA systematic review on insider threat detection using natural language processing
Kundiya, K., & Haribhakta, Y. (2025) · International Journal of Information Security · doi:10.1007/s10207-025-01145-6 · Cited by 9 · see citing works
Kumpf, K., Cajic, M., Zeljkovic, V., Mravik, M., Zivkovic, M. Z., Mani, J. P., Šimić, V., & Bačanin, N. (2025) · International Journal of Information Security · doi:10.1007/s10207-025-01137-6 · Cited by 5 · see citing works
Self-citationRashed, A. E. E., AbdulAzeem, Y., Farrag, T. A., Bamaqa, A., Almaliki, M., Badawy, M., & Elhosseini, M. A. (2025) · Machines · doi:10.3390/machines13040258 · Cited by 5 · see citing works
A Survey of Large Language Models for Insider Threat Detection
Haidar, A., Lin, Y.-Z., Shi, Q., Yang, Z., Amadigwe, D., Terry, B., Battu, S. K., Satam, P., & Shao, S. (2025) · Cyber Awareness and Research Symposium (CARS) · doi:10.1109/cars67163.2025.11337474 · Cited by 2 · see citing works
Insider Threat Detection Using Encoder Decoder Transformer with Cross Attention for CERT Dataset
Kumar, K. P., Snehitha, M., Nishitha, B., & Geethika, K. (2026) · International Conference on Trends in Quantum Computing and Emerging Business Technologies (TQCEBT) · doi:10.1109/tqcebt67648.2026.11681233
ITDF: A Deep Learning Insider Threat Detection Framework Based on Abnormal Logon Behavior
Hajr, L., Alhogail, A. A., Albassam, S. A. A., & Berri, J. (2026) · IEEE Access · doi:10.1109/access.2026.3733166
- Authors
- Kumpf, K., Protic, M., Jovanovic, L., Cajic, M., Zivkovic, M., & Bacanin, N.
- Affiliations
- Singidunum University (RS)
- Singidunum University,Faculty of Informatics and Computing,Belgrade,Serbia
- Univerzitet Union Nikola Tesla (RS)
- Venue
- 2024 International Conference on Circuit, Systems and Communication (ICCSC)
- Publisher
- IEEE
- Published
- 2024-06-28
- Pages
- 1-6
- OpenAlex
- W4401358929
- Cited by
- 7
- References
- 34
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 1
- FWCI
- 2.66
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Circuit, Systems and Communication (ICCSC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- insider threat detection, ransomware, BERT, AdaBoost, hyperparameter optimization, email content analysis, malicious intent detection, website content analysis, cybersecurity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
