The DOI matches its Crossref record.
An Evaluation Framework for Intrusion Detection Dataset
Abstract
The growing number of security threats on the Internet and computer networks demands highly reliable security solutions. Meanwhile, Intrusion Detection (IDSs) and Intrusion Prevention Systems (IPSs) have an important role in the design and development of a robust network infrastructure that can defend computer networks by detecting and blocking a variety of attacks. Reliable benchmark datasets are critical to test and evaluate the performance of a detection system. There exist a number of such datasets, for example, DARPA98, KDD99, ISC2012, and ADFA13 that have been used by the researchers to evaluate the performance of their intrusion detection and prevention approaches. However, not enough research has focused on the evaluation and assessment of the datasets themselves. In this paper we present a comprehensive evaluation of the existing datasets using our proposed criteria, and propose an evaluation framework for IDS and IPS datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization
Sharafaldin, I., Habibi Lashkari, A., & Ghorbani, A. A. (2018) · Proceedings of the 4th International Conference on Information Systems Security and Privacy · doi:10.5220/0006639801080116 · Cited by 5,318
“Regarding to the last dataset evaluation framework published on 2016 (Gharib et al., 2016), covering eleven criteria is necessary for each dataset.”
BackgroundMethodInfluentialSelf-citationDeep Learning Approach for Intelligent Intrusion Detection System
Vinayakumar, R., Alazab, M., Soman, K. P., Poornachandran, P., Al-Nemrat, A., & Venkatraman, S. (2019) · IEEE Access · doi:10.1109/access.2019.2895334 · Cited by 1,946
Network Intrusion Detection for IoT Security Based on Learning Techniques
Chaabouni, N., Mosbah, M., Zemmari, A., Sauvignac, C., & Faruki, P. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2896380 · Cited by 940
A blockchain future for internet of things security: a position paper
Banerjee, M., Lee, J., & Choo, K.-K. R. (2018) · Digital Communications and Networks · doi:10.1016/j.dcan.2017.10.006 · Cited by 585
Troubleshooting an Intrusion Detection Dataset: the CICIDS2017 Case Study
Engelen, G., Rimmer, V., & Joosen, W. (2021) · 2021 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw53761.2021.00009 · Cited by 398
A Survey of Random Forest Based Methods for Intrusion Detection Systems
Resende, P. A. A., & Drummond, A. C. (2019) · ACM Computing Surveys · doi:10.1145/3178582 · Cited by 368
Thakkar, A., & Lohiya, R. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10037-9 · Cited by 350
Zhang, H., Huang, L., Wu, C. Q., & Li, Z. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107315 · Cited by 344
Talukder, M. A., Islam, M. M., Uddin, M. A., Hasan, K. F., Sharmin, S., Alyami, S. A., & Moni, M. A. (2024) · Journal of Big Data · doi:10.1186/s40537-024-00886-w · Cited by 300
Increasing the Performance of Machine Learning-Based IDSs on an Imbalanced and Up-to-Date Dataset
Karatas, G., Demir, O., & Sahingoz, O. K. (2020) · IEEE Access · doi:10.1109/access.2020.2973219 · Cited by 293
A Review of the Advancement in Intrusion Detection Datasets
Thakkar, A., & Lohiya, R. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.03.330 · Cited by 290
A Taxonomy of Network Threats and the Effect of Current Datasets on Intrusion Detection Systems
Hindy, H., Brosset, D., Bayne, E., Seeam, A. K., Tachtatzis, C., Atkinson, R., & Bellekens, X. (2020) · IEEE Access · doi:10.1109/access.2020.3000179 · Cited by 273
Improving AdaBoost-based Intrusion Detection System (IDS) Performance on CIC IDS 2017 Dataset
Yulianto, A., Sukarno, P., & Suwastika, N. A. (2019) · Journal of Physics: Conference Series · doi:10.1088/1742-6596/1192/1/012018 · Cited by 270
A Novel Hierarchical Intrusion Detection System Based on Decision Tree and Rules-Based Models
Ahmim, A., Maglaras, L., Ferrag, M. A., Derdour, M., & Janicke, H. (2019) · 2019 15th International Conference on Distributed Computing in Sensor Systems (DCOSS) · doi:10.1109/dcoss.2019.00059 · Cited by 266
Metadata
- Authors
- Gharib, A., Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A.
- Venue
- 2016 International Conference on Information Science and Security (ICISS)
- Publisher
- IEEE
- Published
- 2016-12
- Pages
- 1-6
- OpenAlex
- W2597441556
- Cited by
- 217
- References
- 0
- FWCI
- 12.03
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.