Security Data Collection and Data Analytics in the Internet: A Survey
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Attacks over the Internet are becoming more and more complex and sophisticated. How to detect security threats and measure the security of the Internet arises a significant research topic. For detecting the Internet attacks and measuring its security, collecting different categories of data and employing methods of data analytics are essential. However, the literature still lacks a thorough review on security-related data collection and analytics on the Internet. Therefore, it becomes a necessity to review the current state of the art in order to gain a deep insight on what categories of data should be collected and which methods should be used to detect the Internet attacks and to measure its security. In this paper, we survey existing studies about security-related data collection and analytics for the purpose of measuring the Internet security. We first divide the data related to network security measurement into four categories: 1) packet-level data; 2) flow-level data; 3) connection-level data; and 4) host-level data. For each category of data, we provide a specific classification and discuss its advantages and disadvantages with regard to the Internet security threat detection. We also propose several additional requirements for security-related data analytics in order to make the analytics flexible and scalable. Based on the usage of data categories and the types of data analytic methods, we review current detection methods for distributed denial of service flooding and worm attacks by applying the proposed requirements to evaluate their performance. Finally, based on the completed review, a list of open issues is outlined and future research directions are identified.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection
Buczak, A. L., & Guven, E. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2494502 · Cited by 3,332
A survey of techniques for internet traffic classification using machine learning
Nguyen, T. T. T., & Armitage, G. (2008) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2008.080406 · Cited by 1,664 · cited 2×
“Some detailed discussions about how to employ the machine learning to detect network attacks are presented in [39] and [40].”
BackgroundUDP Flooding Attack Detection Using Information Metric Measure
Boro, D., Basumatary, H., Goswami, T., & Bhattacharyya, D. K. (2016) · Advances in Intelligent Systems and Computing · doi:10.1007/978-981-10-0129-1_16 · Cited by 10 · cited 1×
“[47] considered two detection features for UDP flooding attacks targeted to a particular destination: (i) the count of total destination port changes for non-spoofed source addresses with random destination ports, and (ii) the count of source addresses changes for randomly spoofed source addresses.”
Background
Research works printed but not linked: 160
- 10.1016/j.comnet.2007.02.001 · DOI · Paper page
- 10.3390/e17042367 · DOI · Paper page
- 2018, LoadRunner
- 2018, Collectl
- 10.1109/SURV.2010.032210.00054 · DOI · Paper page
- brutlag, 2000, Aberrant behavior detection in time series for network service monitoring, Proc USENIX 14th Syst Admin Conf, 139
- ye, 2003, Computer intrusion detection through EWMA for autocorrelated and uncorrelated data, IEEE Trans Rel, 51, 75
- yu, 2012, A nonparametric adaptive CUSUM method and its application in network anomaly detection, Int J Adv Comput Technol, 4, 280
- lee, 2001, Information-theoretic measures for anomaly detection, Proc IEEE Symp Security Privacy, 130
- 10.1109/INM.2007.374825 · doi:10.1109/inm.2007.374825
- 2018, Netflow
- li, 2013, A survey of network flow applications, J Netw Comput Appl, 36, 567, 10.1016/j.jnca.2012.12.020 · DOI · Paper page
- weber, 2006, The fundamentals of passive monitoring access
- 10.1145/948109.948116 · DOI · Paper page
- kavisankar, 2014, A pioneer scheme in the detection and defense of DRDoS attack involving spoofed flooding packets, KSII Trans Internet Inf Syst, 8, 1726, 10.3837/tiis.2014.05.013 · DOI
- 10.1109/ISSA.2015.7335069 · doi:10.1109/issa.2015.7335069
- 10.1016/j.comnet.2012.07.003 · DOI
- 10.1109/COMST.2014.2321898 · DOI · Paper page
- 10.1007/978-3-642-23235-0_25 · doi:10.1007/978-3-642-23235-0_25
- abuhmed, 2008, A survey on deep packet inspection for intrusion detection systems, Mag Korea Telecommun Soc, 24, 25
- chen, 2012, CBF: A packet filtering method for DDoS attack defense in cloud environment, Proc IEEE 9th Int Conf Depend Auton Secure Comput, 427
- 10.1049/cp.2015.0754 · doi:10.1049/cp.2015.0754
- tong, 2016, A hybrid approach of mobile malware detection in Android, J Parallel Distrib Comput, 103, 22, 10.1016/j.jpdc.2016.10.012 · DOI
- 10.1109/NOMS.2010.5488473 · doi:10.1109/noms.2010.5488473
- 10.1016/j.future.2014.06.010 · DOI
- 10.1145/2808206 · DOI
- 10.1109/ICAL.2008.4636301 · doi:10.1109/ical.2008.4636301
- anbar, 2014, Connection failure message-based approach for detecting sequential and random TCP scanning, Indian J of Sci and Technol, 7, 628, 10.17485/ijst/2014/v7i5.8 · doi:10.17485/ijst/2014/v7i5.8
- rasheed, 2009, Intelligent failure connection algorithm for detecting Internet worms, Int J Comput Sci Netw Security, 9, 280
- 10.1109/TNET.2009.2020431 · DOI
- 10.1145/1162666.1162676 · doi:10.1145/1162666.1162676
- rasheed, 2009, A new generation for intelligent anti-Internet worm early system detection, Proc Int Conf Inform Comput, 61
- 10.1109/EBISS.2009.5137885 · doi:10.1109/ebiss.2009.5137885
- 10.1016/j.comcom.2015.06.012 · DOI
- 10.1109/NSS.2010.70 · DOI
- 10.1016/j.eswa.2007.01.040 · DOI
- 10.2298/CSIS161230032K · DOI
- 10.1016/j.eswa.2010.08.137 · DOI · Paper page
- vijayasarathy, 2011, A system approach to network modeling for DDoS detection using a naive Bayesian classifier, Proc Int Conf Commun Syst Netw, 1
- 10.1016/j.neucom.2015.04.101 · DOI · Paper page
- david, 2015, DDoS attack detection using fast entropy approach on flow—Based network traffic, Procedia Comput Sci, 50, 30, 10.1016/j.procs.2015.04.007 · doi:10.1016/j.procs.2015.04.007
- 10.1016/j.ins.2015.06.025 · doi:10.1016/j.ins.2015.06.025
- 10.1016/j.ins.2016.04.009 · DOI
- 10.1016/j.ins.2016.07.008 · doi:10.1016/j.ins.2016.07.008
- 10.1109/TSMCC.2012.2236648 · DOI · Paper page
- 10.1016/j.future.2016.06.036 · DOI
- 10.1109/COMST.2015.2392629 · DOI · Paper page
- 10.1016/j.ins.2015.01.025 · DOI · Paper page
- 10.1109/SURV.2013.031413.00127 · DOI · Paper page
- 10.1145/1216370.1216373 · DOI · Paper page
- yan, 2016, Unwanted traffic detection and control based on trust management, Information Fusion for Cyber-Security Analytics Trends and Patterns, 77
- 10.1109/COMST.2008.4483668 · DOI
- 10.1109/JSAC.2006.877136 · doi:10.1109/jsac.2006.877136
- 10.1109/COMST.2015.2487361 · DOI · Paper page
- 10.1016/j.cose.2014.03.010 · DOI
- 10.1080/13614568.2013.832807 · DOI
- 10.1109/SURV.2014.022714.00160 · DOI
- 10.1109/TPDS.2007.70786 · doi:10.1109/tpds.2007.70786
- 10.1109/GLOCOM.2009.5425454 · doi:10.1109/glocom.2009.5425454
- kim, 2006, PacketScore: A statistics-based packet filtering scheme against distributed denial-of-service attacks, IEEE Trans Depend Secure Comput, 3, 141, 10.1109/TDSC.2006.25 · DOI
- 10.1007/s11219-017-9368-4 · DOI
- 10.1016/j.dcan.2017.10.002 · DOI · Paper page
- 10.1109/SURV.2013.091213.00134 · DOI · Paper page
- 10.1016/j.jnca.2016.06.012 · DOI
- 10.1109/WETICE.2005.35 · DOI · Paper page
- collins, 2007, Hit-list worm detection and bot identification in large networks using protocol graphs, 2nd Int Workshop Recent Adv Intrusion Detect, 276, 10.1007/978-3-540-74320-0_15 · DOI
- 10.1016/j.comcom.2008.12.010 · doi:10.1016/j.comcom.2008.12.010
- 10.1109/TDSC.2010.13 · doi:10.1109/tdsc.2010.13
- 10.1109/TDSC.2014.2302298 · doi:10.1109/tdsc.2014.2302298
- 10.1016/j.future.2013.08.002 · DOI
- 10.1109/ISCC.2006.142 · DOI
- 10.1109/WorldCIS.2013.6751036 · doi:10.1109/worldcis.2013.6751036
- 10.1007/978-3-540-79232-1_22 · doi:10.1007/978-3-540-79232-1_22
- alonso, 2016, Mining IP to domain name interactions to detect DNS flood attacks on recursive DNS servers, SENSORS, 16, 10.3390/s16081311 · DOI
- 10.1109/TC.2013.191 · DOI
- 10.1016/j.comnet.2007.12.004 · DOI
- bhatia, 2017, Ensemble-based model for DDoS attack detection and flash event separation, Proc Future Technol Conf, 958
- 10.1109/TIFS.2017.2758754 · doi:10.1109/tifs.2017.2758754
- 10.1109/INFCOMW.2011.5928950 · doi:10.1109/infcomw.2011.5928950
- 10.1016/j.eswa.2008.06.037 · doi:10.1016/j.eswa.2008.06.037
- 10.1109/INFCOM.2013.6567003 · DOI
- muraleedharan, 2010, ADRISYA: A flow based anomaly detection system for slow and fast scan, Int J Netw Security Appl, 2, 234
- 10.1109/TPDS.2011.262 · DOI · Paper page
- farag, 2009, Intelligent system for worm detection, Int Arab J E-Technol, 1, 58
- 10.3906/elk-1308-188 · doi:10.3906/elk-1308-188
- 10.1109/AINA.2009.132 · doi:10.1109/aina.2009.132
- chatzis, 2008, Flow level data mining of DNS query streams for email worm detection, Proc Int Workshop Comput Intell Security Inf Syst (CISIS), 186
- 10.1007/s00521-009-0238-2 · DOI
- abdulla, 2014, Employing machine learning algorithms to detect unknown scanning and email worms, Int Arab J Inf Technol, 11, 140
- wagner, 2011, Machine learning approach for IP-flow record anomaly detection, Proc Int IFIP-TC6 Netw Conf, 28
- cui, 2005, BINDER: An extrusion-based break-in detector for personal computers, Proc USENIX Annu Tech Conf, 363
- 10.1016/j.comcom.2011.01.012 · DOI · Paper page
- 10.1109/CIS.2015.105 · doi:10.1109/cis.2015.105
- 10.1049/iet-ifs.2010.0158 · doi:10.1049/iet-ifs.2010.0158
- yadav, 2016, DDA: An approach to handle DDoS (ping flood) attack, Proc Int Conf ICT Sustain Develop, 11, 10.1007/978-981-10-0129-1_2 · doi:10.1007/978-981-10-0129-1_2
- 10.1587/transinf.E96.D.1716 · doi:10.1587/transinf.e96.d.1716
- 10.1109/ICoICT.2013.6574541 · DOI
- 10.1109/CyberSecPODS.2016.7502345 · doi:10.1109/cybersecpods.2016.7502345
- 10.1109/LANMAN.2015.7114740 · doi:10.1109/lanman.2015.7114740
- 10.1109/ICCCNT.2014.6963119 · doi:10.1109/icccnt.2014.6963119
- 10.1016/j.comcom.2012.04.002 · doi:10.1016/j.comcom.2012.04.002
- 10.1145/2555615 · doi:10.1145/2555615
- 10.1109/TNET.2008.926503 · DOI · Paper page
- 10.1109/FITME.2008.150 · doi:10.1109/fitme.2008.150
- 10.1109/COMST.2015.2474118 · DOI · Paper page
- 10.1109/ICWS.2017.58 · DOI
- 10.1109/TPDS.2007.1050 · doi:10.1109/tpds.2007.1050
- 10.1109/JPROC.2016.2558521 · DOI · Paper page
- 10.1109/IMIS.2016.88 · doi:10.1109/imis.2016.88
- 10.1109/TIFS.2011.2107320 · DOI · Paper page
- 10.1145/2980258.2980431 · DOI
- 10.1002/sec.1460 · doi:10.1002/sec.1460
- 10.1109/AsiaJCIS.2016.24 · doi:10.1109/asiajcis.2016.24
- 10.1109/ICNP.2004.1348109 · DOI
- xiao, 2016, A protocol-free detection against cloud oriented reflection DoS attacks, Soft Comput, 21, 3713, 10.1007/s00500-015-2025-6 · DOI
- maciá-fernández, 2007, LoRDAS: A low-rate DoS attack against application servers, Proc Int Workshop Critical Inf Infrastruct Secur, 197
- 10.1109/LCOMM.2012.121912.122257 · doi:10.1109/lcomm.2012.121912.122257
- 10.1109/ICC.2014.6883403 · doi:10.1109/icc.2014.6883403
- ryba, 2015, Amplification and DRDoS attack defense—A survey and new perspectives, arXiv 1505 07892
- 10.1109/TNET.2006.880180 · DOI
- kuhrer, 2014, Exit from hell? Reducing the impact of amplification DDoS attacks, Proc 23rd USENIX Secur Symp (USENIX Secur ), 111
- 10.14722/ndss.2014.23233 · doi:10.14722/ndss.2014.23233
- 10.1016/j.comcom.2008.05.033 · doi:10.1016/j.comcom.2008.05.033
- kambourakis, 2007, Detecting DNS amplification attacks, Proc Int Workshop Critical Inf Infrastruct Secur, 186
- 10.1007/978-3-319-17172-2_14 · doi:10.1007/978-3-319-17172-2_14
- wu, 2012, MSABMS-based approach of detecting LDoS attack, Comput Security, 31, 402, 10.1016/j.cose.2012.03.002 · doi:10.1016/j.cose.2012.03.002
- 10.1155/2017/3691629 · doi:10.1155/2017/3691629
- 10.1109/TDSC.2015.2443807 · doi:10.1109/tdsc.2015.2443807
- 10.1002/dac.3449 · doi:10.1002/dac.3449
- 10.1109/COMST.2015.2457491 · DOI · Paper page
- 10.1016/j.cose.2017.05.009 · DOI · Paper page
- 10.1587/transinf.E92.D.799 · doi:10.1587/transinf.e92.d.799
- 10.1109/SURV.2013.052213.00046 · DOI · Paper page
- 10.1016/j.comcom.2017.03.010 · DOI · Paper page
- jacobson, 1989, The tcpdump manual page
- 10.1145/2535372.2535398 · DOI
- combs, 2018, Wireshark
- 10.1007/s12652-013-0196-5 · doi:10.1007/s12652-013-0196-5
- 10.1109/INFOCOM.2016.7524507 · doi:10.1109/infocom.2016.7524507
- 2018, SNORT
- 10.1007/978-3-319-46301-8_27 · doi:10.1007/978-3-319-46301-8_27
- lyon, 2009, Nmap Network Scanning The Official Nmap Project Guide to Network Discovery and Security Scanning
- 10.1109/ICRTIT.2011.5972281 · DOI
- 10.1016/j.future.2016.01.020 · DOI
- 10.1145/2185376.2185382 · DOI
- adi, 2017, Stealthy denial of service (DoS) attack modelling and detection for HTTP/2 services, J Netw Comput Appl, 91, 1, 10.1016/j.jnca.2017.04.015 · DOI
- kang, 2013, The crossfire attack, Proc IEEE Symp Security Privacy, 127
- huistra, 2013, Detecting reflection attacks in DNS flows, Proc 19th Twente Student Conference on IT
- xue, 2014, Towards detecting target link flooding attack, Proc USENIX Large Install Syst Admin Conf, 81
- 10.1016/j.compeleceng.2012.07.004 · doi:10.1016/j.compeleceng.2012.07.004
- 10.1109/WIFS.2015.7368594 · doi:10.1109/wifs.2015.7368594
- 10.1007/s11432-008-0132-z · doi:10.1007/s11432-008-0132-z
- 10.1007/978-3-642-31968-6_5 · doi:10.1007/978-3-642-31968-6_5
- 10.2991/aiie-15.2015.63 · doi:10.2991/aiie-15.2015.63
- 10.1109/TNET.2005.857113 · DOI · Paper page
- 10.1145/948134.948136 · doi:10.1145/948134.948136
- she, 2016, Application-layer DDoS detection by K-means algorithm, Proc Int Conf Elect Electron Comp Eng, 75
- 10.3390/e18100350 · doi:10.3390/e18100350
- 10.1016/j.cose.2009.04.007 · doi:10.1016/j.cose.2009.04.007
- 10.1002/sec.1236 · DOI
Cited by
A Survey on Machine Learning for Data Fusion
Meng, T., Jing, X., Yan, Z., & Pedrycz, W. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.001 · Cited by 677
Self-citationMachine Learning-Powered Encrypted Network Traffic Analysis: A Comprehensive Survey
Shen, M., Ye, K., Liu, X., Zhu, L., Kang, J., Yu, S., Li, Q., & Xu, K. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3208196 · Cited by 265
A survey on trust evaluation based on machine learning
Wang, J., Jing, X., Yan, Z., Fu, Y., Pedrycz, W., & Yang, L. T. (2021) · ACM Computing Surveys · doi:10.1145/3408292 · Cited by 136
Self-citationEncrypted Network Traffic Analysis and Classification Utilizing Machine Learning
Alwhbi, I. A., Zou, C. C., & Alharbi, R. N. (2024) · Sensors · doi:10.3390/s24113509 · Cited by 81
Respite for SMEs: A Systematic Review of Socio-Technical Cybersecurity Metrics
van Haastrecht, M., Yigit Ozkan, B., Brinkhuis, M., & Spruit, M. (2021) · Applied Sciences · doi:10.3390/app11156909 · Cited by 45
“…Industrial Control Systems Asghar et al. [36] 2019 Industrial Control Systems Eckhart et al. [37] 2019 Industrial Control Systems × Jing et al. [38] 2019 Internet Security × Sengupta et al. [39] 2020 Moving Target Defense × Liang and Xiao [40] 2013 Network Security × Ramos et al. [41] 2017…”
Background
Metadata
- Authors
- Jing, X., Yan, Z., & Pedrycz, W.
- Venue
- IEEE Communications Surveys & Tutorials
- Published
- 2019
- Volume
- 21
- Issue
- 1
- Pages
- 586-618
- ISSN
- 1553-877X (ISSN-L)
- 2373-745X
- OpenAlex
- W2886128149
- Cited by
- 130
- References
- 3
- FWCI
- 9.18
- Type
- journal article
- Language
- en
- Open access
- green
- Funders
- National Key Research and Development Program of China (2016YFB0800704)
- National Natural Science Foundation of China (61672410, U1536202)
- Natural Science Foundation of Shaanxi Province (2016ZDJC-06)
- 111 Project (B08038, B16037)
- Academy of Finland (308087)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Security Data Collection and Data Analytics in the Internet: A Survey. https://app.cochat.ai/papers/10.1109/comst.2018.2863942
- The paper
- Jing, X., Yan, Z., & Pedrycz, W. (2019). Security Data Collection and Data Analytics in the Internet: A Survey. IEEE Communications Surveys & Tutorials. https://doi.org/10.1109/comst.2018.2863942
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.