The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Revolutionizing Cyber Threat Detection With Large Language Models: A Privacy-Preserving BERT-Based Lightweight Model for IoT/IIoT Devices
Open accessAbstract
The field of Natural Language Processing (NLP) is currently undergoing a revolutionary transformation driven by the power of pre-trained Large Language Models (LLMs) based on groundbreaking Transformer architectures. As the frequency and diversity of cybersecurity attacks continue to rise, the importance of incident detection has significantly increased. IoT devices are expanding rapidly, resulting in a growing need for efficient techniques to autonomously identify network-based attacks in IoT networks with both high precision and minimal computational requirements. This paper presents SecurityBERT, a novel architecture that leverages the Bidirectional Encoder Representations from Transformers (BERT) model for cyber threat detection in IoT networks. During the training of SecurityBERT, we incorporated a novel privacy-preserving encoding technique called Privacy-Preserving Fixed-Length Encoding (PPFLE). We effectively represented network traffic data in a structured format by combining PPFLE with the Byte-level Byte-Pair Encoder (BBPE) Tokenizer. Our research demonstrates that SecurityBERT outperforms traditional Machine Learning (ML) and Deep Learning (DL) methods, such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs), in cyber threat detection. Employing the Edge-IIoTset cybersecurity dataset, our experimental analysis shows that SecurityBERT achieved an impressive 98.2% overall accuracy in identifying fourteen distinct attack types, surpassing previous records set by hybrid solutions such as GAN-Transformer-based architectures and CNN-LSTM models. With an inference time of less than 0.15 seconds on an average CPU and a compact model size of just 16.7MB, SecurityBERT is ideally suited for real-life traffic analysis and a suitable choice for deployment on resource-constrained IoT devices.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
Ferrag, M. A., Friha, O., Hamouda, D., Maglaras, L., & Janicke, H. (2022) · IEEE Access · doi:10.1109/access.2022.3165809 · Cited by 1,181
Self-citationDeep learning for cyber security intrusion detection: Approaches, datasets, and comparative study
Ferrag, M. A., Maglaras, L., Moschoyiannis, S., & Janicke, H. (2020) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2019.102419 · Cited by 1,130
Self-citationFELIDS: Federated learning-based intrusion detection system for agricultural Internet of Things
Friha, O., Ferrag, M. A., Shu, L., Maglaras, L., Choo, K.-K. R., & Nafaa, M. (2022) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2022.03.003 · Cited by 314
Self-citationMoustafa, N., Koroniotis, N., Keshk, M., Zomaya, A. Y., & Tari, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3280465 · Cited by 266
Securing Critical Infrastructures: Deep-Learning-Based Threat Detection in IIoT
Yu, K., Tan, L., Mumtaz, S., Al-Rubaye, S., Al-Dulaimi, A., Bashir, A. K., & Khan, F. A. (2021) · IEEE Communications Magazine · doi:10.1109/mcom.101.2001126 · Cited by 251
DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection
Chen, Y., Ding, Z., Alowain, L., Chen, X., & Wagner, D. (2023) · Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses · doi:10.1145/3607199.3607242 · Cited by 238
SecureBERT: A Domain-Specific Language Model for Cybersecurity
Aghaei, E., Niu, X., Shadid, W., & Al-Shaer, E. (2023) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-25538-0_3 · Cited by 210
An improved anomaly detection model for IoT security using decision tree and gradient boosting
Douiba, M., Benkirane, S., Guezzaz, A., & Azrour, M. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-022-04783-y · Cited by 190
Selvarajan, S., Srivastava, G., Khadidos, A. O., Khadidos, A. O., Baza, M., Alshehri, A., & Lin, J. C.-W. (2023) · Journal of Cloud Computing · doi:10.1186/s13677-023-00412-y · Cited by 167
Transformer-Based Language Models for Software Vulnerability Detection
Thapa, C., Jang, S. I., Ahmed, M. E., Camtepe, S. A., Pieprzyk, J., & Nepal, . (2022) · Annual Computer Security Applications Conference · doi:10.1145/3564625.3567985 · Cited by 157
Friha, O., Ferrag, M. A., Benbouzid, M., Berghout, T., Kantarcı, B., & Choo, K. R. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103097 · Cited by 139
Self-citationA Lightweight IoT Intrusion Detection Model Based on Improved BERT-of-Theseus
Wang, Z., Li, J., Yang, S., Luo, X., Li, D., & Mahmoodi, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122045 · Cited by 131
CyBERT: Contextualized Embeddings for the Cybersecurity Domain
Ranade, P., Piplai, A., Joshi, A., & Finin, T. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671824 · Cited by 123
Martin, C. H., Peng, T., & Mahoney, M. W. (2021) · Nature Communications · doi:10.1038/s41467-021-24025-8 · Cited by 119
BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model
Chen, S., & Liao, H. (2022) · Applied Artificial Intelligence · doi:10.1080/08839514.2022.2145642 · Cited by 109
Intelligent AI-based Healthcare Cyber Security System using Multi-Source Transfer Learning Method
Chakraborty, C., Nagarajan, S. M., Devarajan, G. G., Ramana, T. V., & Mohanty, R. (2023) · ACM Transactions on Sensor Networks · doi:10.1145/3597210 · Cited by 106
DeepAK-IoT: An effective deep learning model for cyberattack detection in IoT networks
Ding, W., Abdel‐Basset, M., & Mohamed, R. (2023) · Information Sciences · doi:10.1016/j.ins.2023.03.052 · Cited by 99
Generative AI for Cyber Threat-Hunting in 6G-enabled IoT Networks
Ferrag, M. A., Debbah, M. A., & Al-Hawawreh, M. S. (2023) · IEEE International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW) · doi:10.1109/ccgridw59191.2023.00018 · Cited by 97
Self-citationAn Attack Detection Framework Based on BERT and Deep Learning
Seyyar, Y. E., YAVUZ, A. G., & Ünver, H. M. (2022) · IEEE Access · doi:10.1109/access.2022.3185748 · Cited by 92
MalBERT: Malware Detection using Bidirectional Encoder Representations from Transformers
Rahali, A., & Akhloufi, M. A. (2021) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc52423.2021.9659287 · Cited by 90
Silvestri, S., Islam, S., Papastergiou, S., Tzagkarakis, C., & Ciampi, M. (2023) · Sensors · doi:10.3390/s23020651 · Cited by 76
CAN-BERT do it? Controller Area Network Intrusion Detection System based on BERT Language Model
Alkhatib, N., Mushtaq, M., Ghauch, H., & Danger, J. (2022) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa56895.2022.10017800 · Cited by 70
Hamouda, D., Ferrag, M. A., Benhamida, N., & Séridi, H. (2022) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2022.101738 · Cited by 69
Self-citationAn Automatic and Efficient Malware Traffic Classification Method for Secure Internet of Things
Zhang, X., Hao, L., Gui, G., Wang, Y., Adebisi, B., & Sari, H. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3318290 · Cited by 65
A Deep-Learning-Integrated Blockchain Framework for Securing Industrial IoT
Aljuhani, A., Kumar, P., Alanazi, R., Albalawi, T., Taouali, O., Islam, A. K. M. N., Kumar, N., & Alazab, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3316669 · Cited by 63
Jahangir, H., Lakshminarayana, S., Maple, C. R., & Epiphaniou, G. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3240289 · Cited by 60
Identifying Security and Privacy Violation Rules in Trigger-Action IoT Platforms With NLP Models
Breve, B., Cimino, G., & Deufemia, V. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3222615 · Cited by 59
F-BIDS: Federated-Blending based Intrusion Detection System
Aouedi, O., & Piamrat, K. (2023) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2023.101750 · Cited by 57
A Hybrid CNN-LSTM Model for IIoT Edge Privacy-Aware Intrusion Detection
de Elias, E. M., Carriel, V. S., de Oliveira, G. W., dos Santos, A. L., Nogueira, M., Hirata, R., & Batista, D. M. (2022) · IEEE Latin-American Conference on Communications · doi:10.1109/latincom56090.2022.10000468 · Cited by 38
Toward Federated Learning Models Resistant to Adversarial Attacks
Hu, F., Zhou, W., Liao, K., Li, H., & Tong, D. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3272334 · Cited by 34
Liu, J., Tang, Y., Zhao, H., Wang, X., Li, F., & Zhang, J. (2023) · ACM Transactions on Sensor Networks · doi:10.1145/3585520 · Cited by 33
Resilient Distributed Classification Learning Against Label Flipping Attack: An ADMM-Based Approach
Wang, X., Fang, C., Yang, M., Wu, X., Zhang, H., & Cheng, P. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3264918 · Cited by 15
Cited by
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
“In addition, ChatGPT is also used to detect browser history, Windows event logs, and interactions with cloud platform machines.”
Large Language Models for Cyber Security: A Systematic Literature Review
Xu, H., Wang, S., Li, N., Wang, K., Zhao, Y., Chen, K., Yu, T., Liu, Y., & Wang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3769676 · Cited by 189
Generative AI in cybersecurity: A comprehensive review of LLM applications and vulnerabilities
Ferrag, M. A., Alwahedi, F., Battah, A., Cherif, B., Mechri, A., Tihanyi, N., Bisztray, T., & Debbah, M. (2025) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2025.01.001 · Cited by 148
Self-citationYigit, Y., Ferrag, M. A., Ghanem, M. C., Sarker, I. H., Maglaras, L. A., Chrysoulas, C., Moradpoor, N., Tihanyi, N., & Janicke, H. (2025) · Sensors · doi:10.3390/s25061666 · Cited by 69
“…utilize models like GPT-4 (Generative Pre-trained Transformer-4) to analyze and interpret extensive unstructured text data from online forums, threat reports, and system logs to predict and identify potential cyber threats, including phishing attacks or malware aimed at energy grid systems [75].”
Self-citationUnveiling Shadows: Harnessing Artificial Intelligence for Insider Threat Detection
Yilmaz, E., & Can, O. (2024) · Engineering, Technology & Applied Science Research · doi:10.48084/etasr.6911 · Cited by 32
Metadata
- Authors
- Ferrag, M. A., Ndhlovu, M., Tihanyi, N., Cordeiro, L. C., Debbah, M., Lestable, T., & Thandi, N. S.
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 23733-23750
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4391582407
- Cited by
- 312
- References
- 36
- FWCI
- 52.05
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.