The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Benchmarking of Machine Learning for Anomaly Based Intrusion Detection Systems in the CICIDS2017 Dataset
Open accessAbstract
An intrusion detection system (IDS) is an important protection instrument for detecting complex network attacks. Various machine learning (ML) or deep learning (DL) algorithms have been proposed for implementing anomaly-based IDS (AIDS). Our review of the AIDS literature identifies some issues in related work, including the randomness of the selected algorithms, parameters, and testing criteria, the application of old datasets, or shallow analyses and validation of the results. This paper comprehensively reviews previous studies on AIDS by using a set of criteria with different datasets and types of attacks to set benchmarking outcomes that can reveal the suitable AIDS algorithms, parameters, and testing criteria. Specifically, this paper applies 10 popular supervised and unsupervised ML algorithms for identifying effective and efficient ML-AIDS of networks and computers. These supervised ML algorithms include the artificial neural network (ANN), decision tree (DT), k-nearest neighbor (k-NN), naive Bayes (NB), random forest (RF), support vector machine (SVM), and convolutional neural network (CNN) algorithms, whereas the unsupervised ML algorithms include the expectation-maximization (EM), k-means, and self-organizing maps (SOM) algorithms. Several models of these algorithms are introduced, and the turning and training parameters of each algorithm are examined to achieve an optimal classifier evaluation. Unlike previous studies, this study evaluates the performance of AIDS by measuring the true positive and negative rates, accuracy, precision, recall, and F-Score of 31 ML-AIDS models. The training and testing time for ML-AIDS models are also considered in measuring their performance efficiency given that time complexity is an important factor in AIDSs. The ML-AIDS models are tested by using a recent and highly unbalanced multiclass CICIDS2017 dataset that involves real-world network attacks. In general, the k-NN-AIDS, DT-AIDS, and NB-AIDS models obtain the best results and show a greater capability in detecting web attacks compared with other models that demonstrate irregular and inferior results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Nearest neighbor pattern classification
Cover, T., & Hart, P. (1967) · IEEE Transactions on Information Theory · doi:10.1109/tit.1967.1053964 · Cited by 16,936
A detailed analysis of the KDD CUP 99 data set
Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009) · 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications · doi:10.1109/cisda.2009.5356528 · Cited by 5,396
The expectation-maximization algorithm
Moon, T. K. (1996) · IEEE Signal Processing Magazine · doi:10.1109/79.543975 · Cited by 3,181
Naive (Bayes) at forty: The independence assumption in information retrieval
Lewis, D. D. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026666 · Cited by 2,104
A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks
Yin, C., Zhu, Y., Fei, J., & He, X. (2017) · IEEE Access · doi:10.1109/access.2017.2762418 · Cited by 2,100 · cited 1×
“Therefore, a comparative analysis of these models by using a recently developed dataset is necessary to evaluate their efficiency in distinguishing malicious from benign activities and in identifying different types of attacks [13].”
BackgroundKohonen, T. (1998) · Neurocomputing · doi:10.1016/s0925-2312(98)00030-7 · Cited by 1,968
McHugh, J. (2000) · ACM Transactions on Information and System Security · doi:10.1145/382912.382923 · Cited by 1,309
A Deep Learning Approach for Network Intrusion Detection System
Javaid, A., Niyaz, Q., Sun, W., & Alam, M. (2016) · Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (formerly BIONETICS) · doi:10.4108/eai.3-12-2015.2262516 · Cited by 1,274
Ahmad, I., Basheri, M., Iqbal, M. J., & Rahim, A. (2018) · IEEE Access · doi:10.1109/access.2018.2841987 · Cited by 693
Random Forest Modeling for Network Intrusion Detection System
Farnaaz, N., & Jabbar, M. A. (2016) · Procedia Computer Science · doi:10.1016/j.procs.2016.06.047 · Cited by 684
Fuzziness based semi-supervised learning approach for intrusion detection system
Ashfaq, R. A. R., Wang, X.-Z., Huang, J. Z., Abbas, H., & He, Y.-L. (2017) · Information Sciences · doi:10.1016/j.ins.2016.04.019 · Cited by 552
Intrusion Detection in 802.11 Networks: Empirical Evaluation of Threats and a Public Dataset
Kolias, C., Kambourakis, G., Stavrou, A., & Gritzalis, S. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2402161 · Cited by 550
CICIDS-2017 Dataset Feature Analysis With Information Gain for Anomaly Detection
Kurniabudi, Stiawan, D., Darmawijoyo, Bin Idris, M. Y., Bamhdi, A. M., & Budiarto, R. (2020) · IEEE Access · doi:10.1109/access.2020.3009843 · Cited by 453
CNN-Based Network Intrusion Detection against Denial-of-Service Attacks
Kim, J., Kim, J., Kim, H., Shim, M., & Choi, E. (2020) · Electronics · doi:10.3390/electronics9060916 · Cited by 420
A GA-LR wrapper approach for feature selection in network intrusion detection
Khammassi, C., & Krichen, S. (2017) · Computers & Security · doi:10.1016/j.cose.2017.06.005 · Cited by 382
Performance analysis of NSL-KDD dataset using ANN
Ingre, B., & Yadav, A. (2015) · 2015 International Conference on Signal Processing and Communication Engineering Systems · doi:10.1109/spaces.2015.7058223 · Cited by 374
Decision Tree Analysis on J48 Algorithm for Data Mining
Bhargava, N., Sharma, G., Bhargava, R., & Mathuria, M. (2013) · Computer Science and Software Engineering · Cited by 369
DeepCoin: A Novel Deep Learning and Blockchain-Based Energy Exchange Framework for Smart Grids
Ferrag, M. A., & Maglaras, L. (2020) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2019.2922936 · Cited by 358
Moustafa, N., Slay, J., & Creech, G. (2019) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2017.2715166 · Cited by 314
Towards a Reliable Intrusion Detection Benchmark Dataset
Sharafaldin, I., Canadian Institute for Cybersecurity (CIC), U., Gharib, A., Canadian Institute for Cybersecurity (CIC), U., Lashkari, A. H., Canadian Institute for Cybersecurity (CIC), U., Ghorbani, A. A., & Canadian Institute for Cybersecurity (CIC), U. (2017) · Software Networking · doi:10.13052/jsn2445-9739.2017.009 · Cited by 302
A hybrid intrusion detection system based on ABC-AFS algorithm for misuse and anomaly detection
Hajisalem, V., & Babaie, S. (2018) · Computer Networks · doi:10.1016/j.comnet.2018.02.028 · Cited by 290
Mazini, M., Shirazi, B., & Mahdavi, I. (2019) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2018.03.011 · Cited by 269
GAN-based imbalanced data intrusion detection system
Lee, J., & Park, K. (2021) · Personal and Ubiquitous Computing · doi:10.1007/s00779-019-01332-y · Cited by 248
Kumar, V., Sinha, D., Das, A. K., Pandey, S. C., & Goswami, R. T. (2020) · Cluster Computing · doi:10.1007/s10586-019-03008-x · Cited by 245
False alarm minimization techniques in signature-based intrusion detection systems: A survey
Hubballi, N., & Suryanarayanan, V. (2014) · Computer Communications · doi:10.1016/j.comcom.2014.04.012 · Cited by 242
Evaluation of Machine Learning Algorithms for Anomaly Detection
Elmrabit, N., Zhou, F., Li, F., & Zhou, H. (2020) · 2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security) · doi:10.1109/cybersecurity49315.2020.9138871 · Cited by 212
Feature Selection for Intrusion Detection Using Random Forest
Hasan, M. A. M., Nasser, M., Ahmad, S., & Molla, K. I. (2016) · Journal of Information Security · doi:10.4236/jis.2016.73009 · Cited by 211
Moustafa, N., Creech, G., & Slay, J. (2017) · Data Analytics · doi:10.1007/978-3-319-59439-2_5 · Cited by 202
Analysis of KDD Dataset Attributes - Class wise for Intrusion Detection
Aggarwal, P., & Sharma, S. K. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.07.490 · Cited by 200
Cyber threat detection based on artificial neural networks using event profiles
Lee, J., Kim, J., Kim, I., & Han, K. (2019) · IEEE Access · doi:10.1109/access.2019.2953095 · Cited by 188
Network intrusion detection system using J48 Decision Tree
Sahu, S., & Mehtre, B. M. (2015) · 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI) · doi:10.1109/icacci.2015.7275914 · Cited by 181
Khalaf, B. A., Mostafa, S. A., Mustapha, A., Mohammed, M. A., & Abduallah, W. M. (2019) · IEEE Access · doi:10.1109/access.2019.2908998 · Cited by 177
Self-citationA review of KDD99 dataset usage in intrusion detection and machine learning between 2010 and 2015Preprint
Özgür, A., & Erdem, H. (2016) · doi:10.7287/peerj.preprints.1954v1 · Cited by 174
Semi-supervised machine learning approach for DDoS detection
Idhammad, M., Afdel, K., & Belouch, M. (2018) · Applied Intelligence · doi:10.1007/s10489-018-1141-2 · Cited by 165
A Detailed Analysis of the CICIDS2017 Data Set
Sharafaldin, I., Habibi Lashkari, A., & Ghorbani, A. A. (2019) · Communications in Computer and Information Science · doi:10.1007/978-3-030-25109-3_9 · Cited by 164
Feature selection in UNSW-NB15 and KDDCUP'99 datasets
Janarthanan, T., & Zargari, S. (2017) · 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE) · doi:10.1109/isie.2017.8001537 · Cited by 160
Intrusion Detection System Based on Decision Tree over Big Data in Fog Environment
Peng, K., Leung, V. C. M., Zheng, L., Wang, S., Huang, C., & Lin, T. (2018) · Wireless Communications and Mobile Computing · doi:10.1155/2018/4680867 · Cited by 158
Ren, J., Guo, J., Wang, Q., Huang, Y., Hao, X., & Hu, J. (2019) · Security and Communication Networks · doi:10.1155/2019/7130868 · Cited by 154
Deep Learning Approach on Network Intrusion Detection System using NSL-KDD Dataset
Gurung, S., Ghose, M. K., & Subedi, A. (2019) · International Journal of Computer Network and Information Security · doi:10.5815/ijcnis.2019.03.02 · Cited by 143
An in-depth experimental study of anomaly detection using gradient boosted machine
Tama, B. A., & Rhee, K.-H. (2017) · Neural Computing and Applications · doi:10.1007/s00521-017-3128-z · Cited by 141
An Enhanced Anomaly Detection in Web Traffic Using a Stack of Classifier Ensemble
Tama, B. A., Nkenyereye, L., Islam, S. M. R., & Kwak, K. S. (2020) · IEEE Access · doi:10.1109/access.2020.2969428 · Cited by 139
MSML: A Novel Multilevel Semi-Supervised Machine Learning Framework for Intrusion Detection System
Yao, H., Fu, D., Zhang, P., Li, M., & Liu, Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2873125 · Cited by 130
Dendron : Genetic trees driven rule induction for network intrusion detection systems
Papamartzivanos, D., Mármol, F. G., & Kambourakis, G. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.09.056 · Cited by 128
A survey on intrusion detection and prevention in wireless ad-hoc networks
Khan, K., Mehmood, A., Khan, S., Khan, M. A., Iqbal, Z., & Mashwani, W. K. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.101701 · Cited by 127
A Two-Stage Classifier Approach using RepTree Algorithm for Network Intrusion Detection
Belouch, M., El, S., & Idhammad, M. (2017) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2017.080651 · Cited by 115
Bindra, N., & Sood, M. (2019) · Automatic Control and Computer Sciences · doi:10.3103/s0146411619050043 · Cited by 111
Soheily-Khah, S., Marteau, P.-F., & Béchet, N. (2018) · International Conference on Data Intelligence and Security (ICDIS) · doi:10.1109/icdis.2018.00043 · Cited by 105
Raman, M. R. G., Somu, N., Jagarapu, S., Manghnani, T., Selvam, T. M., Krithivasan, K., & Sriram, V. S. S. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09762-z · Cited by 93
Al‐Zewairi, M., Almajali, S., & Awajan, A. (2017) · International Conference on New Trends in Computing Sciences (ICTCS) · doi:10.1109/ictcs.2017.29 · Cited by 91
Balasaraswathi, V. R., Sugumaran, M., & Hamid, Y. (2017) · Journal of Communications and Information Networks · doi:10.1007/s41650-017-0033-7 · Cited by 87
Hybrid Intrusion Detection System for DDoS Attacks
Cepheli, Ö., Büyükçorak, S., & Kurt, G. K. (2016) · Journal of Electrical and Computer Engineering · doi:10.1155/2016/1075648 · Cited by 75
Deep learning in intrusion detection perspective: Overview and further challenges
Kim, K., & Aminanto, M. E. (2017) · International Workshop on Big Data and Information Security (IWBIS) · doi:10.1109/iwbis.2017.8275095 · Cited by 72
Performance evaluation of unsupervised techniques in cyber-attack anomaly detection
Meira, J., Andrade, R., Praça, I., Carneiro, J., Bolón‐Canedo, V., Alonso‐Betanzos, A., & Marreiros, G. (2019) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-019-01417-9 · Cited by 71
Evaluation of Machine Learning Techniques for Security in SDN
Ahmad, A., Harjula, E., Ylianttila, M., & Ahmad, I. (2020) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps50303.2020.9367477 · Cited by 67
Network intrusion detection using equality constrained-optimization-based extreme learning machines
Cheng-ru, W., Xu, R.-F., Lee, S.-J., & Lee, C.-H. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.02.015 · Cited by 64
Effectiveness of Machine Learning Based Intrusion Detection Systems
Alrowaily, M., Alenezi, F. N., & Lu, Z. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-24907-6_21 · Cited by 64
A two-stage hybrid classification technique for network intrusion detection system
Hussain, J., Lalmuanawma, S., & Chhakchhuak, L. (2016) · International Journal of Computational Intelligence Systems · doi:10.1080/18756891.2016.1237186 · Cited by 62
Subba, B., Biswas, S., & Karmakar, S. (2016) · IEEE International Conference on Advanced Networks and Telecommunications Systems · doi:10.1109/ants.2016.7947776 · Cited by 62
Studying Machine Learning Techniques for Intrusion Detection Systems
Dang, Q.-V. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-35653-8_28 · Cited by 58
Data Mining in Intrusion Detection: A Comparative Study of Methods, Types and Data Sets
Azad, C., & Jha, V. K. (2013) · International Journal of Information Technology and Computer Science · doi:10.5815/ijitcs.2013.08.08 · Cited by 52
Unsupervised intrusion detection through skip-gram models of network behavior
Carrasco, R. S. M., & Sicilia, M. (2018) · Computers & Security · doi:10.1016/j.cose.2018.07.003 · Cited by 41
A Deep Learning Approach for Network Intrusion Detection System.
Javaid, A. Y., Niyaz, Q., Sun, W., & Alam, M. (2015) · BICT · Cited by 39
Hadoop Based Parallel Binary Bat Algorithm for Network Intrusion Detection
Natesan, P., Rajalaxmi, R. R., Gowrison, G., et al. (2016) · International Journal of Parallel Programming · doi:10.1007/s10766-016-0456-z · Cited by 38
Effective Dimensionality Reduction of Payload-Based Anomaly Detection in TMAD Model for HTTP Payload
Kakavand, M., Mustapha, N., Mustapha, A., & Abdullah, M. T. (2016) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2016.08.025 · Cited by 15
The Best Neural Network Architecture
Kuri-Morales, Á. F. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13650-9_7 · Cited by 10
Cited by
RTIDS: A Robust Transformer-Based Approach for Intrusion Detection System
Wu, Z., Zhang, H., Wang, P., & Sun, Z. (2022) · IEEE Access · doi:10.1109/access.2022.3182333 · Cited by 317
Enhancing intrusion detection: a hybrid machine and deep learning approach
Sajid, M., Malik, K. R., Almogren, A., Malik, T. S., Khan, A. H., Tanveer, J., & Rehman, A. U. (2024) · Journal of Cloud Computing · doi:10.1186/s13677-024-00685-x · Cited by 283
Moustafa, N., Koroniotis, N., Keshk, M., Zomaya, A. Y., & Tari, Z. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3280465 · Cited by 266
Unsupervised Learning Approach for Anomaly Detection in Industrial Control Systems
Choi, W.-H., & Kim, J. (2024) · Applied System Innovation · doi:10.3390/asi7020018 · Cited by 74
Fatema, K., Dey, S. K., Anannya, M., Khan, R. T., Rashid, M., Chunhua, S., & Mazumder, R. (2025) · doi:10.20944/preprints202503.1902.v1 · Cited by 51
Metadata
- Authors
- Maseer, Z. K., Yusof, R., Bahaman, N., Mostafa, S. A., & Foozy, C. F. M.
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 22351-22370
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3126814579
- Cited by
- 473
- References
- 65
- FWCI
- 32.82
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.