A comprehensive survey on machine learning for networking: evolution, applications and research opportunities

In the paper

“Though there are various surveys on ML in networking [18,61,82,142,246,339], this survey is purposefully different.”

What the sources say

• Supported

  The Introduction (p. 2) identifies this work as a survey of machine learning in wireless sensor networks and discusses earlier surveys focused on WSNs, wireless ad-hoc networks, or particular WSN challenges. It therefore serves as an example of a survey specialized to a network technology, as the citing sentence requires. (LLM, conf=0.92)

  cites [18] Alsheikh MA, Lin S, Niyato D, Tan HP. Machine learning in wireless sensor networks: Algorithms, strategies, and applications. IEEE Commun Surv Tutor. 2014; 16(4):1996–2018. source
• Not checked yetcites [61] Bkassiny M, Li Y, Jayaweera SK. A survey on machine-learning techniques in cognitive radios. IEEE Commun Surv Tutor. 2013; 15(3):1136–59. source
• Not checked yetcites [82] Buczak AL, Guven E. A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor. 2016; 18(2):1153–76. source
• Not checked yetcites [142] Fadlullah Z, Tang F, Mao B, Kato N, Akashi O, Inoue T, Mizutani K. State-of-the-art deep learning: Evolving machine intelligence toward tomorrow’s intelligent network traffic control systems. IEEE Commun Surv Tutor. 2017; PP(99):1.
• Not checked yetcites [246] Klaine PV, Imran MA, Onireti O, Souza RD. A survey of machine learning techniques applied to self organizing cellular networks. IEEE Commun Surv Tutor. 2017; PP(99):1.
• Not checked yetcites [339] Nguyen TTT, Armitage G. A survey of techniques for internet traffic classification using machine learning. IEEE Commun Surv Tutor. 2008b; 10(4):56–76. source

In the paper

“As depicted in Table 13, the majority of these proposals apply diverse supervised learning techniques, mostly for prediction.”

In the paper

“However, the evaluation was based only on simulations of wired networks, though including single and multiple bottleneck topologies.”

In the paper

“In addition, all the classifiers perform binary classification of packet losses.”

In the paper

“TCP variants built upon these ML-based CWND updating mechanisms perform better in terms of throughput and delay than standard and non-ML TCP versions particularly enhanced for specific network conditions (e.g.TCP-FeW and TCP-Cubic).”

In the paper

“Whereas, the other half either use CALA or an own RL design on a wider range of network architectures, including satellite, cellular, and data center networks.”

In the paper

“Half of these ML-based approaches apply FALA, CALA, or Q-learning (including the FK function approximation) on resource-constrained networks (i.e.WANET and IoT).”

In the paper

“The other half of ML-based schemes employ reinforcement-based methods for deducing the increment in the packet drop probability.”

In the paper

“Some of the ML-based TCP also show improvements in packet loss and fairness.”

In the paper

“As depicted in Table 11, half of the ML-based AQM schemes apply supervised OLS and NN for predicting future time series values of either traffic volume or queue length.”

In the paper

“All these ML-based AQM schemes improve and speed up the queue stabilization over non-ML AQM schemes for varying network conditions.”

In the paper

“The majority of the MLbased classifiers were tested using synthetic data collected from simulations.”

In the paper

“TCP variants built upon these ML-based classifiers outperform standard and diverse non-ML TCP versions (e.g.TCP-Veno and Burst-TCP).”

In the paper

“The evaluation has been only based on synthetic data collected from simulations and experimental testbeds.”

In the paper

“As shown in Table 14, Admission control has leveraged ML extensively in a variety of networks, including ATM networks [95,189,190], wireless networks [8,36,359], cellular networks [66,67,281,372,458], ad hoc networks [452], and next generation networks [311].”

What the sources say

• Not checked yetcites [95] Cheng RG, Chang CJ. Neural-network connection-admission control for atm networks. IEE Proc-Commun. 1997; 144(2):93–8. source
• Not checked yetcites [189] Hiramatsu A. Atm communications network control by neural networks. IEEE Trans Neural Netw. 1990; 1(1):122–30. source
• Not checked yetcites [190] Hiramatsu A. Integration of atm call admission control and link capacity control by distributed neural networks. IEEE J Sel Areas Commun. 1991; 9(7):1131–8. source
• Not checked yetcites [8] Ahn CW, Ramakrishna RS. Qos provisioning dynamic connection-admission control for multimedia wireless networks using a hopfield neural network. IEEE Trans Veh Technol. 2004; 53(1):106–117. source
• Not checked yetcites [36] Baldo N, Dini P, Nin-Guerrero J. User-driven call admission control for voip over wlan with a neural network based cognitive engine. In: Cognitive Information Processing (CIP), 2010 2nd International Workshop on. IEEE: 2010. p. 52–6. source
• Not checked yetcites [359] Piamrat K, Ksentini A, Viho C, Bonnin JM. Qoe-aware admission control for multimedia applications in ieee 802.11 wireless networks. In: Vehicular Technology Conference, 2008. VTC 2008-Fall. IEEE 68th. IEEE: 2008. p. 1–5. source
• Not checked yetcites [66] Bojovic B, Baldo N, Dini P. A cognitive scheme for radio admission control in lte systems. In: Cognitive Information Processing (CIP), 2012 3rd International Workshop on. Piscataway: IEEE: 2012. p. 1–3.
• Not checked yetcites [67] Bojovic B, Quer G, Baldo N, Rao RR. Bayesian and neural network schemes for call admission control in lte systems. In: Global Communications Conference (GLOBECOM), 2013 IEEE. Piscataway: IEEE: 2013. p. 1246–52.
• Not checked yetcites [281] Liu D, Zhang Y, Zhang H. A self-learning call admission control scheme for cdma cellular networks. IEEE trans neural netw. 2005; 16(5):1219–28. source
• Not checked yetcites [372] Quer G, Baldo N, Zorzi M. Cognitive call admission control for voip over ieee 802.11 using bayesian networks. In: Global Telecommunications Conference (GLOBECOM 2011), 2011 IEEE. IEEE: 2011. p. 1–6. source

In the paper

“The evaluation was mostly performed using synthetic data collected from simulations and laboratory testbeds.”

In the paper

“Despite the widespread application of ML for load prediction and resource management in cloud data centers [367], various challenges still prevail for different networks, including cellular networks, wireless networks and ad hoc networks.”

What the sources say

• Not checked yetcites [367] Prevost JJ, Nagothu K, Kelley B, Jamshidi M. Prediction of cloud data center networks loads using stochastic and neural models. IEEE: 2011. p. 276–281. source

In the paper

“Only the WMA-based estimators collected real data to test their approaches.”

In the paper

“All these ML-based estimators outperform the accuracy of conventional estimation mechanisms (e.g.EWMA and history-based).”

In the paper

“As final remarks, note that the majority of the ML-based solutions rely on synthetic data to conduct the evaluation.”

In the paper

“Application of unsupervised techniques in resource management is rather unexplored, with the exception of a few works.”

In the paper

“However, a common challenge faced by these techniques is the scarcity of fault data generated in a production network.”

In the paper

“As summarized in Tables 16, 17 and 18, most of the ML-based fault management approaches use different supervised learning techniques that depend on training data to predict/detect/locate faults in the network.”

In the paper

“Due to the lack of real-life traces, these approaches adopt different methods to simulate the network of interest and produce training and testing data.”

In the paper

“As evident from Tables 14 and 15, the ML-based resource management schemes studied in this paper can be broadly classified into two groups-supervised learning-based and RL-based.”

In the paper

“As a remedy, some approaches leverage unsupervised techniques that rely on detecting changes in network states instead of using labeled fault data.”

In the paper

“In addition, the evaluation settings adopted by these approaches lack uniformity.”

In the paper

“Consequently, the fault scenarios studied in these approaches vary greatly as they depend both on the layer (e.g.physical, link, or IP layer) and the type (e.g.cellular, wireless, local area network) of the network.”

In the paper

“The ML-based fault management approaches surveyed in this paper focus on a variety of networks.”

In the paper

“The same holds for feature set and output of these schemes, as both features and outputs depend on the fault scenario of a particular network.”

In the paper

“However, PCA can detect 25 anomalies with 0 FP.”

What the sources say

• Not checked yetcites [7] Ahmed T, Coates M, Lakhina A. Multivariate online anomaly detection using kernel recursive least squares.IEEE; 2007. pp. 625–33. source

In the paper

“However, detecting new attacks using the RBF-SVM with limited and full features achieve comparable TP with a high FP of approximately 18% for normal traces.”

What the sources say

• Not checked yetcites [64] Boero L, Marchese M, Zappatore S. Support vector machine meets software defined networking in ids domain. In: Proceedings of the 29th International Teletraffic Congress (ITC), vol. 3. New York: IEEE: 2017. p. 25–30.

In the paper

“The KDD dataset is used for evaluation and shows that IMM outperforms three related works [131, 139, 363] in terms of detection rate and FP.”

What the sources say

• Not checked yetcites [220] Jiang S, Song X, Wang H, Han JJ, Li QH. A clustering-based method for unsupervised intrusion detections. Pattern Recog Lett. 2006; 27(7):802–10. source

In the paper

“Therefore, the patterns proven to work in one network may not be feasible for another network of the same kind.”

In the paper

“Whereas, Fadlullah et al. [142] and Buczak et al. [82], both state-of-theart surveys, have a specialized treatment of ML to specific problems in networking.”

What the sources say

• Not checked yetcites [142] Fadlullah Z, Tang F, Mao B, Kato N, Akashi O, Inoue T, Mizutani K. State-of-the-art deep learning: Evolving machine intelligence toward tomorrow’s intelligent network traffic control systems. IEEE Commun Surv Tutor. 2017; PP(99):1.
• Not checked yetcites [82] Buczak AL, Guven E. A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor. 2016; 18(2):1153–76. source

In the paper

“The ability to program the network by leveraging SDN alleviates these obstacles.”

In the paper

“For example, in traffic classification, features that are extracted from multi-modal application classes (e.g.WWW) tend to show a non-Gaussian behavior [321].”

What the sources say

• Not checked yetcites [321] Moore AW, Zuev D. Internet traffic classification using bayesian analysis techniques. In: ACM SIGMETRICS Performance Evaluation Review, ACM, vol 33.2005. p. 50–60. source

In the paper

“However, average packet size can be skewed by intermediate fragmentation and encryption, and packet inter-arrival times and their distributions are affected by queuing in routers [492].”

In the paper

“These relationships not only become irrelevant and redundant, they contradict widely held assumptions in traffic classification, such as feature distributions being independent and following a Gaussian distribution.”

In the paper

“Average packet size and flow duration have been shown to be good discriminators [390].”

What the sources say

• Not checked yetcites [390] Roughan M, Sen S, Spatscheck O, Duffield N. Class-of-service mapping for qos: a statistical signature-based approach to ip traffic classification. In: Proceedings of the 4th ACM SIGCOMM conference on Internet measurement. ACM: 2004a. p. 135–148. source

In the paper

“Furthermore, streaming applications often behave similar to bulk data transfer applications [390].”

What the sources say

• Not checked yetcites [390] Roughan M, Sen S, Spatscheck O, Duffield N. Class-of-service mapping for qos: a statistical signature-based approach to ip traffic classification. In: Proceedings of the 4th ACM SIGCOMM conference on Internet measurement. ACM: 2004a. p. 135–148. source

In the paper

“Average packet size is fairly constant in nature [492] and packet inter-arrival times are a good discriminator for bulk data transfer (e.g.FTP) and streaming applications [390].”

What the sources say

• Not checked yetcites [390] Roughan M, Sen S, Spatscheck O, Duffield N. Class-of-service mapping for qos: a statistical signature-based approach to ip traffic classification. In: Proceedings of the 4th ACM SIGCOMM conference on Internet measurement. ACM: 2004a. p. 135–148. source

In the paper

“Therefore, careful feature extraction and selection is crucial for the performance of ML models [77].”

What the sources say

• Not checked yetcites [77] Brill E, Lin JJ, Banko M, Dumais ST, Ng AY, et al. Data-intensive question answering. In: TREC, vol. 56.2001. p. 90. source

In the paper

“However, average packet size can be skewed by intermediate fragmentation and encryption, and packet inter-arrival times and their distributions are affected by queuing in routers [492].”

What the sources say

• Not checked yetcites [492] Zander S, Nguyen T, Armitage G. Automated traffic classification and application identification using machine learning.IEEE; 2005, pp. 250–7. source

In the paper

“There is also an inherent mutual dependency on the size of the training data of one class of interest on another, impacting model performance [417].”

What the sources say

• Not checked yetcites [417] Soysal M, Schmidt EG. Machine learning algorithms for accurate flow-based network traffic classification: Evaluation and comparison. Perform Eval. 2010; 67(6):451–67. source

In the paper

“However, the accuracy metric is not reliable when the data is skewed with respect to the classes.”

In the paper

“By the end of the decade, the application domains of ML started expending to the operation and management of communication networks [57,217,289].”

What the sources say

• Not checked yetcites [57] Bernstein L, Yuhas CM. How technology shapes network management. IEEE Netw. 1989; 3(4):16–9. source
• Not checked yetcites [217] Jennings A. A learning system for communications network configuration. Eng Appl Artif Intell. 1988; 1(3):151–60. source
• Not checked yetcites [289] Macleish KJ. Mapping the integration of artificial intelligence into telecommunications. IEEE J Sel Areas Commun. 1988; 6(5):892–8. source

In the paper

“As a result, the poor performance of a single predictor or classifier can be compensated with ensemble learning at the price of (significantly) extra computation.”

In the paper

“The comparison amongst the TSF methods show that in general the NNE produces the lowest MAPE for both datasets.”

What the sources say

• Not checked yetcites [104] Cortez P, Rio M, Rocha M, Sousa P. Internet traffic forecasting using neural networks. In: Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN). IEEE: 2006. p. 2635–42. source

In the paper

“In the last decade, different types of NNs (SLP, MLP, RNN, etc.) and other supervised techniques have been employed for TSF of network traffic.”

In the paper

“These proofs showed that SLP-NN approximators, which employed sufficient number of neurons of continuous sigmoidal activation type (a constraint introduced by Cybenko and relaxed by Hornik), were universal approximators, capable of approximating any continuous function to any desired accuracy.”

What the sources say

• Not checked yetcites [106] Cybenko G. Approximation by superpositions of a sigmoidal function. Mathematics of Control. Signals Syst (MCSS). 1989; 2(4):303–14. source
• Not checked yetcites [196] Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Netw. 1991; 4(2):251–7. source
• Not checked yetcites [158] Funahashi KI. On the approximate realization of continuous mappings by neural networks. Neural netw. 1989; 2(3):183–92. source

In the paper

“To the best of our knowledge, Yu et al. [489] were the first to apply ML in traffic prediction using MLP-NN.”

What the sources say

• Not checked yetcites [489] Yu E, Chen CR. Traffic prediction using neural networks. In: Proceedings of IEEE GLOBECOM. IEEE: 1993. p. 991–5. source

In the paper

“On the other hand, NN and GPR, have an average TPR consistently above 80%.”

What the sources say

• Not checked yetcites [365] Poupart P, Chen Z, Jaini P, Fung F, Susanto H, Geng Y, Chen L, Chen K, Jin H. Online flow size prediction for improved network routing.IEEE; 2016, pp. 1–6. source

In the paper

“The combined NN and wavelet transform model reduces the peak inter-DC link utilization, i.e. the ISP's billed utilization, by about 9%.”

What the sources say

• Not checked yetcites [274] Li Y, Liu H, Yang W, Hu D, Xu W. Inter-data-center network traffic prediction with elephant flows: IEEE; 2016b, pp. 206–13. source

In the paper

“However, it is worth noting that oBMM outperforms all other approaches in terms of average TNR in all 3 datasets.”

What the sources say

• Not checked yetcites [365] Poupart P, Chen Z, Jaini P, Fung F, Susanto H, Geng Y, Chen L, Chen K, Jin H. Online flow size prediction for improved network routing.IEEE; 2016, pp. 1–6. source

In the paper

“The motive for flow size prediction in [365], is to discriminate elephant flows from mice flows in routing to speed up elephant flow completion time.”

What the sources say

• Not checked yetcites [365] Poupart P, Chen Z, Jaini P, Fung F, Susanto H, Geng Y, Chen L, Chen K, Jin H. Online flow size prediction for improved network routing.IEEE; 2016, pp. 1–6. source

In the paper

“A 14% improvement is observed when the size of the three first packets is used along with the header information.”

What the sources say

• Not checked yetcites [365] Poupart P, Chen Z, Jaini P, Fung F, Susanto H, Geng Y, Chen L, Chen K, Jin H. Online flow size prediction for improved network routing.IEEE; 2016, pp. 1–6. source

In the paper

“Thus, k-step ahead predictions, feed k × 120 features into the NN and show a relative RMSE (RRMSE) ranging from 4 to 10% for the NN-Wavelet transformation model as the prediction horizon varies from 30 s to 20 min.”

What the sources say

• Not checked yetcites [274] Li Y, Liu H, Yang W, Hu D, Xu W. Inter-data-center network traffic prediction with elephant flows: IEEE; 2016b, pp. 206–13. source

In the paper

“Evidently, wavelet transformation reduces the average prediction errors for different prediction horizons by 5.4 and 2.9% for incoming and outgoing traffic, respectively.”

What the sources say

• Not checked yetcites [274] Li Y, Liu H, Yang W, Hu D, Xu W. Inter-data-center network traffic prediction with elephant flows: IEEE; 2016b, pp. 206–13. source

In the paper

“In the latter dataset, oBMM seems to suffer the most from class imbalance.”

In the paper

“According to the authors, GPR improves the completion time by 6.6% in average for 99% of elephant flows when only the first packet header information is considered.”

What the sources say

• Not checked yetcites [365] Poupart P, Chen Z, Jaini P, Fung F, Susanto H, Geng Y, Chen L, Chen K, Jin H. Online flow size prediction for improved network routing.IEEE; 2016, pp. 1–6. source

In the paper

“Therefore, though the motive was to promote flow count based traffic prediction to overcome the cost of monitoring traffic volume, the performance is compromised.”

In the paper

“The experiments show noticeable discrepancies in the performance of the approaches with varying datasets.”

What the sources say

• Not checked yetcites [365] Poupart P, Chen Z, Jaini P, Fung F, Susanto H, Geng Y, Chen L, Chen K, Jin H. Online flow size prediction for improved network routing.IEEE; 2016, pp. 1–6. source

In the paper

“Supervised NNs (including MLP and RNN) have been successfully applied to traffic prediction, as shown in Table 9.”

In the paper

“In contrast, the linear ARIMA model has prediction error of approximately 8.5 and 6.9% for incoming and outgoing traffic, respectively.”

What the sources say

• Not checked yetcites [274] Li Y, Liu H, Yang W, Hu D, Xu W. Inter-data-center network traffic prediction with elephant flows: IEEE; 2016b, pp. 206–13. source

In the paper

“Furthermore, the hybrid PSO-ABC has a faster training time than ABC or PSO.”

What the sources say

• Not checked yetcites [500] Zhu Y, Zhang G, Qiu J. Network traffic prediction based on particle swarm bp neural network. JNW. 2013; 8(11):2685–91.

In the paper

“Nevertheless, various classifiers leverage port numbers in conjunction with other techniques [31,56,244,417] to improve the performance of the traffic classifiers.”

What the sources say

• Not checked yetcites [31] Bakhshi T, Ghita B. On internet traffic classification: A two-phased machine learning approach. J Comput Netw Commun. 2016; 2016:2016.
• Not checked yetcites [56] Bernaille L, Teixeira R, Salamatian K. Early application identification. In: Proceedings of the 2006 ACM CoNEXT Conference. ACM: 2006b. p. 61–6:12. source
• Not checked yetcites [244] Kim H, Claffy KC, Fomenkov M, Barman D, Faloutsos M, Lee K. Internet traffic classification demystified: myths, caveats, and the best practices.ACM; 2008, p. 11. source
• Not checked yetcites [417] Soysal M, Schmidt EG. Machine learning algorithms for accurate flow-based network traffic classification: Evaluation and comparison. Perform Eval. 2010; 67(6):451–67. source

In the paper

“The signatures are then used to train a SVM classifier that distinguishes between the classes of interest with an average TP of 99.6%.”

What the sources say

• Not checked yetcites [146] Finamore A, Mellia M, Meo M, Rossi D. Kiss: Stochastic packet inspection classifier for udp traffic. IEEE/ACM Trans Netw. 2010; 18(5):1505–15. source

In the paper

“TSF approaches, such as [52, 86, 104, 141, 500], where NNs are used to infer traffic volumes from past measured volumes, show high long-term and short-term prediction accuracy at low complexity with limited number of features and limited number of layers and neurons.”

What the sources say

• Not checked yetcites [52] Bermolen P, Rossi D. Support vector regression for link load prediction. Comput Netw. 2009; 53(2):191–201. source
• Not checked yetcites [86] Chabaa S, Zeroual A, Antari J. Identification and prediction of internet traffic using artificial neural networks. J Int Learn Syst Appl. 2010; 2(03):147.
• Not checked yetcites [104] Cortez P, Rio M, Rocha M, Sousa P. Internet traffic forecasting using neural networks. In: Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN). IEEE: 2006. p. 2635–42. source
• Not checked yetcites [141] Eswaradass A, Sun XH, Wu M. Network bandwidth predictor (nbp): A system for online network performance forecasting. In: Proceedings of 6th IEEE International Symposium on Cluster Computing and the Grid (CCGRID). IEEE: 2006. p. 4–pp. source
• Not checked yetcites [500] Zhu Y, Zhang G, Qiu J. Network traffic prediction based on particle swarm bp neural network. JNW. 2013; 8(11):2685–91.

In the paper

“Non-TSF approaches were investigated in [94,274,365] to infer traffic volumes from flow count and packet header fields.”

What the sources say

• Not checked yetcites [94] Chen Z, Wen J, Geng Y. Predicting future traffic using hidden markov models. In: Proceddings of 24th IEEE International Conference on Network Protocols (ICNP). IEEE: 2016. p. 1–6. source
• Not checked yetcites [274] Li Y, Liu H, Yang W, Hu D, Xu W. Inter-data-center network traffic prediction with elephant flows: IEEE; 2016b, pp. 206–13. source
• Not checked yetcites [365] Poupart P, Chen Z, Jaini P, Fung F, Susanto H, Geng Y, Chen L, Chen K, Jin H. Online flow size prediction for improved network routing.IEEE; 2016, pp. 1–6. source

In the paper

“Also, it is cumbersome to manually maintain and adapt the signatures to the ever growing number of applications and their dynamics [138].”

What the sources say

• Not checked yetcites [138] Erman J, Mahanti A, Arlitt M, Williamson C. Identifying and discriminating between web and peer-to-peer traffic in the network core. In: Proceedings of the 16th international conference on World Wide Web. ACM: 2007b. p. 883–92. source

In the paper

“Generally, network traffic classification methodologies can be decomposed into four broad categories that leverage port number, packet payload, host behavior or flow features [31,244].”

What the sources say

• Not checked yetcites [31] Bakhshi T, Ghita B. On internet traffic classification: A two-phased machine learning approach. J Comput Netw Commun. 2016; 2016:2016.
• Not checked yetcites [244] Kim H, Claffy KC, Fomenkov M, Barman D, Faloutsos M, Lee K. Internet traffic classification demystified: myths, caveats, and the best practices.ACM; 2008, p. 11. source

In the paper

“For instance, in order to predict the traffic for a particular flow f on link l, there must be a counter on link l actively measuring the traffic for that particular flow f, which can be challenging on very high speed links.”

In the paper

“Ma et al. [286] show that payload-based traffic classification can be performed without any a priori knowledge of the application classes using unsupervised clustering.”

What the sources say

• Not checked yetcites [286] Ma J, Levchenko K, Kreibich C, Savage S, Voelker GM. Unexpected means of protocol inference. In: Proceedings of the 6th ACM SIGCOMM conference on Internet measurement.2006. p. 313–26. source

In the paper

“Furthermore, relying solely on port numbers has been shown to be ineffective [125,228,320], largely due to the use of dynamic port negotiation, tunneling and misuse of port numbers assigned to well-known applications for obfuscating traffic and avoiding firewalls [54,109,176,286].”

What the sources say

• Not checked yetcites [125] Dreger H, Feldmann A, Mai M, Paxson V, Sommer R. Dynamic application-layer protocol analysis for network intrusion detection. In: USENIX Security Symposium. Berkeley: USENIX Security Symposium: 2006. p. 257–72.
• Not checked yetcites [228] Karagiannis T, Broido A, Brownlee N, Claffy KC, Faloutsos M. Is p2p dying or just hiding?[p2p traffic measurement]. In: IEEE Global Telecommunications Conference (GLOBECOM), vol. 3.2004. p. 1532–8. source
• Not checked yetcites [320] Moore AW, Papagiannaki K. Toward the accurate identification of network applications. In: PAM. Springer: 2005. p. 41–54. source
• Not checked yetcites [54] Bernaille L, Teixeira R. Implementation issues of early application identification. Lect Notes Compu Sci. 2007; 4866:156. source
• Not checked yetcites [109] Dainotti A, Pescape A, Claffy KC. Issues and future directions in traffic classification. IEEE Netw. 2012; 26(1):35–40. source
• Not checked yetcites [176] Haffner P, Sen S, Spatscheck O, Wang D. Acas: automated construction of application signatures. In: Proceedings of the 2005 ACM SIGCOMM workshop on Mining network data. New York: ACM: 2005. p. 197–202.
• Not checked yetcites [286] Ma J, Levchenko K, Kreibich C, Savage S, Voelker GM. Unexpected means of protocol inference. In: Proceedings of the 6th ACM SIGCOMM conference on Internet measurement.2006. p. 313–26. source

In the paper

“Also, temporal and spatial portability of signatures is validated with marginal degradation in performance.”

What the sources say

• Not checked yetcites [53] Bermolen P, Mellia M, Meo M, Rossi D, Valenti S. Abacus: Accurate behavioral classification of P2P-tv traffic. Comput Netw. 2011; 55(6):1394–411. source

In the paper

“However, it incurs increased inference time, especially for classifying unknown flows [347].”

What the sources say

• Not checked yetcites [347] Park J, Tyan HR, Kuo CCJ. Internet traffic classification for scalable qos provision. In: Multimedia and Expo, vol 2006 IEEE International Conference on. IEEE: 2006. p. 1221–4. source

In the paper

“Moreover, it also outperforms NB with respect to training time.”

What the sources say

• Not checked yetcites [321] Moore AW, Zuev D. Internet traffic classification using bayesian analysis techniques. In: ACM SIGMETRICS Performance Evaluation Review, ACM, vol 33.2005. p. 50–60. source

In the paper

“The authors show that these features act as good discriminators for a SVM classifier to differentiate between webmail and nonwebmail traffic.”

What the sources say

• Not checked yetcites [404] Schatzmann D, Mühlbauer W, Spyropoulos T, Dimitropoulos X. Digging into https: Flow-based classification of webmail traffic. In: Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement: 2010. p. 322–27. source

In the paper

“It overcomes the limitations of unregistered or misused port numbers and encrypted packet payload, by moving the observation point to the edge of the network and examining traffic between hosts (e.g.how many hosts are contacted, by which transport protocol, how many different ports are involved).”

In the paper

“It is not always possible to collect bidirectional flows due to routing asymmetries [138,347].”

What the sources say

• Not checked yetcites [138] Erman J, Mahanti A, Arlitt M, Williamson C. Identifying and discriminating between web and peer-to-peer traffic in the network core. In: Proceedings of the 16th international conference on World Wide Web. ACM: 2007b. p. 883–92. source
• Not checked yetcites [347] Park J, Tyan HR, Kuo CCJ. Internet traffic classification for scalable qos provision. In: Multimedia and Expo, vol 2006 IEEE International Conference on. IEEE: 2006. p. 1221–4. source

In the paper

“The higher FN is attributed to the inability of the classifier to distinguish between VPN and webmail servers.”

What the sources say

• Not checked yetcites [404] Schatzmann D, Mühlbauer W, Spyropoulos T, Dimitropoulos X. Digging into https: Flow-based classification of webmail traffic. In: Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement: 2010. p. 322–27. source

In the paper

“Unfortunately, DT suffers from overfitting with noisy data, which deteriorates performance [347].”

What the sources say

• Not checked yetcites [347] Park J, Tyan HR, Kuo CCJ. Internet traffic classification for scalable qos provision. In: Multimedia and Expo, vol 2006 IEEE International Conference on. IEEE: 2006. p. 1221–4. source

In the paper

“The data exchanged amongst P2P applications is highly discriminative [53].”

What the sources say

• Not checked yetcites [53] Bermolen P, Mellia M, Meo M, Rossi D, Valenti S. Abacus: Accurate behavioral classification of P2P-tv traffic. Comput Netw. 2011; 55(6):1394–411. source

In the paper

“It has the potential to overcome numerous limitations of other techniques, such as unregistered port numbers, encrypted packet payload, routing asymmetries, high storage and computational overhead [55,138,176,347].”

What the sources say

• Not checked yetcites [55] Bernaille L, Teixeira R, Akodkenou I, Soule A, Salamatian K. Traffic classification on the fly. ACM SIGCOMM Comput Commun Rev. 2006a; 36(2):23–6. source
• Not checked yetcites [138] Erman J, Mahanti A, Arlitt M, Williamson C. Identifying and discriminating between web and peer-to-peer traffic in the network core. In: Proceedings of the 16th international conference on World Wide Web. ACM: 2007b. p. 883–92. source
• Not checked yetcites [176] Haffner P, Sen S, Spatscheck O, Wang D. Acas: automated construction of application signatures. In: Proceedings of the 2005 ACM SIGCOMM workshop on Mining network data. New York: ACM: 2005. p. 197–202.
• Not checked yetcites [347] Park J, Tyan HR, Kuo CCJ. Internet traffic classification for scalable qos provision. In: Multimedia and Expo, vol 2006 IEEE International Conference on. IEEE: 2006. p. 1221–4. source

In the paper

“This is in contrast to NBKE, where the classification accuracy deteriorates dramatically with increasing number of features.”

What the sources say

• Not checked yetcites [347] Park J, Tyan HR, Kuo CCJ. Internet traffic classification for scalable qos provision. In: Multimedia and Expo, vol 2006 IEEE International Conference on. IEEE: 2006. p. 1221–4. source

In the paper

“Evidently, bulk data transfer applications exhibit higher burst than streaming applications.”

What the sources say

• Not checked yetcites [347] Park J, Tyan HR, Kuo CCJ. Internet traffic classification for scalable qos provision. In: Multimedia and Expo, vol 2006 IEEE International Conference on. IEEE: 2006. p. 1221–4. source

In the paper

“It also achieves temporal stability by classifying new flows collected twelve months later with an accuracy of 93% and TPR of 98%.”

What the sources say

• Not checked yetcites [321] Moore AW, Zuev D. Internet traffic classification using bayesian analysis techniques. In: ACM SIGMETRICS Performance Evaluation Review, ACM, vol 33.2005. p. 50–60. source

In the paper

“They notice that often streaming applications behave very similar to bulk data transfer applications.”

What the sources say

• Not checked yetcites [390] Roughan M, Sen S, Spatscheck O, Duffield N. Class-of-service mapping for qos: a statistical signature-based approach to ip traffic classification. In: Proceedings of the 4th ACM SIGCOMM conference on Internet measurement. ACM: 2004a. p. 135–148. source

In the paper

“In essence, flow feature-based traffic classification exploits the diversity and distinguishable characteristics of the traffic footprint generated by different applications [31,467].”

What the sources say

• Not checked yetcites [31] Bakhshi T, Ghita B. On internet traffic classification: A two-phased machine learning approach. J Comput Netw Commun. 2016; 2016:2016.
• Not checked yetcites [467] Wang Y, Xiang Y, Yu S. Internet traffic classification using machine learning: a token-based approach. IEEE: 2011. p. 285–9. source

In the paper

“However, the accuracy of the host behavior-based traffic classification strongly depends on the location of the monitoring system, especially since the observed communication pattern may be affected by routing asymmetries in the network core [229].”

What the sources say

• Not checked yetcites [229] Karagiannis T, Papagiannaki K, Faloutsos M. BLINC: multilevel traffic classification in the dark. ACM SIGCOMM Comput Commun Rev. 2005; 35(4):229–40. source

In the paper

“The F-measure, used to evaluate per class performance, of BoF-NB outperforms NB irrespective of the aggregation rule.”

What the sources say

• Not checked yetcites [496] Zhang J, Chen C, Xiang Y, Zhou W, Xiang Y. Internet traffic classification by aggregating correlated naive bayes predictions. IEEE Trans Inf Forensic Secur. 2013; 8(1):5–15. source

In the paper

“Furthermore, the classifier remains fairly robust over time for different traces.”

What the sources say

• Not checked yetcites [137] Erman J, Mahanti A, Arlitt M, Cohen I, Williamson C. Offline/realtime traffic classification using semi-supervised learning. Perform Eval. 2007a; 64(9):1194–213. source

In the paper

“[137] propose a semi-supervised TCP traffic classification technique that partially overcomes a limitation of [55] and [337]-a priori knowledge of application class or protocol.”

What the sources say

• Not checked yetcites [137] Erman J, Mahanti A, Arlitt M, Cohen I, Williamson C. Offline/realtime traffic classification using semi-supervised learning. Perform Eval. 2007a; 64(9):1194–213. source
• Not checked yetcites [55] Bernaille L, Teixeira R, Akodkenou I, Soule A, Salamatian K. Traffic classification on the fly. ACM SIGCOMM Comput Commun Rev. 2006a; 36(2):23–6. source
• Not checked yetcites [337] Nguyen TT, Armitage G, Branch P, Zander S. Timely and continuous machine-learning-based classification for interactive ip traffic. IEEE/ACM Trans Netw (TON). 2012; 20(6):1880–94. source

In the paper

“Similar to [137, 337], Li et al. [270] classify TCP traffic into application classes, including unknown application classes using a few packets in a flow.”

What the sources say

• Not checked yetcites [270] Li W, Moore AW. A machine learning approach for efficient traffic classification. In: Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, 2007. MASCOTS’07. 15th International, Symposium on. IEEE: 2007. p. 310–7. source

In the paper

“Both classifiers perform well when evaluated with missing flow start, missing directionality, or 5% independent random packet loss.”

What the sources say

• Not checked yetcites [337] Nguyen TT, Armitage G, Branch P, Zander S. Timely and continuous machine-learning-based classification for interactive ip traffic. IEEE/ACM Trans Netw (TON). 2012; 20(6):1880–94. source

In the paper

“In offline classification, the authors in [137] achieve over 94% flow accuracy with just two randomly labeled flows per cluster, amongst a mix of 64,000 unlabeled flows and k = 400.”

What the sources say

• Not checked yetcites [137] Erman J, Mahanti A, Arlitt M, Cohen I, Williamson C. Offline/realtime traffic classification using semi-supervised learning. Perform Eval. 2007a; 64(9):1194–213. source

In the paper

“However, it is infeasible to expect complete a priori information for applications, since often network operators do not even know all the applications that are running in the network.”

In the paper

“Despite the high accuracy, this technique lacks in flexibility, since the classifier can only recognize the application classes that were known a priori.”

What the sources say

• Not checked yetcites [337] Nguyen TT, Armitage G, Branch P, Zander S. Timely and continuous machine-learning-based classification for interactive ip traffic. IEEE/ACM Trans Netw (TON). 2012; 20(6):1880–94. source

In the paper

“Therefore, even with small labeled training datasets, RTC can achieve a higher flow and byte accuracy of up to 15% and 10%, respectively, in comparison to the second best technique.”

What the sources say

• Not checked yetcites [497] Zhang J, Chen X, Xiang Y, Zhou W, Wu J. Robust network traffic classification. IEEE/ACM Trans Netw (TON). 2015; 23(4):1257–70. source

In the paper

“The authors integrate their layered online classification in Bro and achieve byte accuracies in the 70-90% range.”

What the sources say

• Not checked yetcites [137] Erman J, Mahanti A, Arlitt M, Cohen I, Williamson C. Offline/realtime traffic classification using semi-supervised learning. Perform Eval. 2007a; 64(9):1194–213. source

In the paper

“DIFFUSE achieves a high accuracy of 98-99% for Enemy Territory (online game) and VoIP traffic replayed across the network, while monitoring one or more 1 Gb/s links.”

What the sources say

• Not checked yetcites [337] Nguyen TT, Armitage G, Branch P, Zander S. Timely and continuous machine-learning-based classification for interactive ip traffic. IEEE/ACM Trans Netw (TON). 2012; 20(6):1880–94. source

In the paper

“Furthermore, the accuracy of service classification is between 95-100% for majority of the providers.”

What the sources say

• Not checked yetcites [409] Shbair WM, Cholez T, Francois J, Chrisment I. A multi-level framework to identify https services. In: IEEE/IFIP Network Operations and Management Symposium (NOMS)2016. p. 240–8. source

In the paper

“Furthermore, when trained and tested across datasets (i.e.across networks), it achieves 83.7% DR and 1.5% FPR.”

What the sources say

• Not checked yetcites [17] Alshammari R, Zincir-Heywood AN. Machine learning based encrypted traffic classification: Identifying ssh and skype. In: Computational Intelligence for Security and Defense Applications, 2009. CISDA 2009. IEEE Symposium on. IEEE: 2009. p. 1–8. source

In the paper

“However, with a larger time difference between training and testing data collection, the error rates increase to 5.5 and 1.2% for TCP and UDP traffic, respectively.”

What the sources say

• Not checked yetcites [222] Jin Y, Duffield N, Erman J, Haffner P, Sen S, Zhang ZL. A modular machine learning system for flow-level traffic classification in large networks. ACM Trans Knowl Discov Data (TKDD). 2012; 6(1):4.

In the paper

“Unfortunately, Alshammari et al. [17] do not detect the precise applications within a secure session.”

What the sources say

• Not checked yetcites [17] Alshammari R, Zincir-Heywood AN. Machine learning based encrypted traffic classification: Identifying ssh and skype. In: Computational Intelligence for Security and Defense Applications, 2009. CISDA 2009. IEEE Symposium on. IEEE: 2009. p. 1–8. source

In the paper

“Hence, it generalizes well from one network to another.”

What the sources say

• Not checked yetcites [17] Alshammari R, Zincir-Heywood AN. Machine learning based encrypted traffic classification: Identifying ssh and skype. In: Computational Intelligence for Security and Defense Applications, 2009. CISDA 2009. IEEE Symposium on. IEEE: 2009. p. 1–8. source

In the paper

“In the evaluation, RF performs better in comparison to C4.5 DT with a precision of 92.6%, recall of 92.8%, and F-measure of 92.6%, to classify service providers with selected features.”

What the sources say

• Not checked yetcites [409] Shbair WM, Cholez T, Francois J, Chrisment I. A multi-level framework to identify https services. In: IEEE/IFIP Network Operations and Management Symposium (NOMS)2016. p. 240–8. source

In the paper

“Jointly, NB and χ 2 classifier outperform the classifiers in isolation by detecting Skype voice traffic over UDP and TCP with nearly zero FP.”

What the sources say

• Not checked yetcites [69] Bonfiglio D, Mellia M, Meo M, Rossi D, Tofanelli P. Revealing skype traffic: when randomness plays with you. In: ACM SIGCOMM Computer Communication Review. ACM: 2007. p. 37–48. source

In the paper

“The C4.5 DT classifier also performed well for Skype traffic with 98.4% DR and 7.8% FPR in the Dalhousie dataset.”

What the sources say

• Not checked yetcites [17] Alshammari R, Zincir-Heywood AN. Machine learning based encrypted traffic classification: Identifying ssh and skype. In: Computational Intelligence for Security and Defense Applications, 2009. CISDA 2009. IEEE Symposium on. IEEE: 2009. p. 1–8. source

In the paper

“In the evaluation, C4.5 DT outperform the other classifiers for the majority of datasets.”

What the sources say

• Not checked yetcites [17] Alshammari R, Zincir-Heywood AN. Machine learning based encrypted traffic classification: Identifying ssh and skype. In: Computational Intelligence for Security and Defense Applications, 2009. CISDA 2009. IEEE Symposium on. IEEE: 2009. p. 1–8. source