Predicting future traffic using Hidden Markov Models
Abstract
Network traffic volume estimation and prediction is an important research topic that attracts persistent attention from the networking community and the machine learning community. Although there has been extensive work on estimating or predicting the traffic matrix using time series models, low rank matrix decomposition et. al, to the best of our knowledge, there is few work investigating the problem whether we are able to estimate and predict the traffic volume based on some statistics of the traffic which are much less costly to collect, for example, the flow counts. In this paper, we propose to model the relationship between the traffic volume and simple statistics about flows using a Hidden Markov Model based on which we can avoid direct measurement of the traffic volume but instead we estimate and predict the hidden traffic volume based on those simple flow statistics which are collected by some sketch techniques. We demonstrate the feasibility and effectiveness of our proposed method using some semi-simulation and real data experimental results.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
They propose a HMM to describe the relationship between the flow count, flow volume and their temporal dynamic behavior, and use Kernel Bayes Rule and RNN with LSTM unit to predict future traffic volume based on current flow count.
“They propose a HMM to describe the relationship between the flow count, flow volume and their temporal dynamic behavior.The Kernel Bayes Rule (KBR) and RNN with LSTM unit is used to predict future traffic volume based on current flow count.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [94]
The RNN shows a prediction MSE of 0.3 at best, 0.05 higher than KBR and twice as much as the prediction error of an RNN fed with traffic volume instead of flow count.
“The RNN shows a prediction MSE of 0.3 at best, 0.05 higher than KBR and twice as much as the prediction error of an RNN fed with traffic volume instead of flow count.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [94]
Chen et al. investigate the possibility of reducing the cost of monitoring and collecting traffic volume by inferring future traffic volume based on flow count only.
“Chen et al. [94], investigate the possibility of reducing cost of monitoring and collecting traffic volume, by inferring future traffic volume based on flow count only.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [94]
Non-TSF approaches were investigated in [94,274,365] to infer traffic volumes from flow count and packet header fields.
“Non-TSF approaches were investigated in [94,274,365] to infer traffic volumes from flow count and packet header fields.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [94]
Cites
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,934 · see citing works
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Atiya, A. F. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.848998 · Cited by 2,288 · see citing works
Time-Varying Network Tomography: Router Link Data
Cao, J. L., Davis, D., Wiel, S. V., & Yu, B. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474303 · Cited by 322 · see citing works
Hilbert space embeddings of conditional distributions with applications to dynamical systems
Song, L., Huang, J., Smola, A. J., et al. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553497 · Cited by 261 · see citing works
Song, L., Fukumizu, K., & Gretton, A. (2013) · IEEE Signal Processing Magazine · doi:10.1109/msp.2013.2252713 · Cited by 200 · see citing works
Maximum pseudo likelihood estimation in network tomography
Liang, G., & Yu, B. (2003) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2003.814464 · Cited by 168 · see citing works
Time-Varying Network Tomography: Router Link Data
Cao, J. L., Davis, D., Wiel, S. V., et al. (2000) · Journal of the American Statistical Association · doi:10.2307/2669743 · Cited by 131 · see citing works
Wireless Traffic Modeling and Prediction Using Seasonal ARIMA Models
Shu, Y. (2005) · IEICE Transactions on Communications · doi:10.1093/ietcom/e88-b.10.3992 · Cited by 127 · see citing works
Wireless traffic modeling and prediction using seasonal ARIMA models
Shu, Y., Yu, M., Liu, J., et al. (2004) · International Conference on Communications · doi:10.1109/icc.2003.1203886 · Cited by 126 · see citing works
Identification and Prediction of Internet Traffic Using Artificial Neural Networks
Chabaa, S., Zeroual, A., & Antari, J. (2010) · Journal of Intelligent Learning Systems and Applications · doi:10.4236/jilsa.2010.23018 · Cited by 102 · see citing works
Traffic prediction using neural networks
Yu, E. S., & Chen, C. Y. R. (2002) · Proceedings of GLOBECOM '93. IEEE Global Telecommunications Conference · doi:10.1109/glocom.1993.318226 · Cited by 77 · see citing works
Network Tomography via Compressed Sensing
Firooz, M. H., & Roy, S. (2010) · Globecom · doi:10.1109/glocom.2010.5684036 · Cited by 60 · see citing works
Network Tomography: Identifiability and Fourier Domain Estimation
Chen, A., Cao, J. L., & Bu, T. (2010) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2010.2068294 · Cited by 51 · see citing works
Fukumizu, K., Song, L., & Gretton, A. (2011) · neural information processing systems · Cited by 46 · see citing works
BitTorrent Network Traffic Forecasting With ARMA
Hoong, P. K. (2012) · International journal of Computer Networks & Communications · doi:10.5121/ijcnc.2012.4409 · Cited by 46 · see citing works
Prediction of internet traffic based on Elman neural network
Junsong, W., Jiukun, W., Maohua, Z., et al. (2009) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc.2009.5192639 · Cited by 30 · see citing works
Prediction of Network Traffic Using Dynamic Bilinear Recurrent Neural Network
Park, D.-C., & Woo, D.-M. (2009) · International Conference on Natural Computation (ICNC) · doi:10.1109/icnc.2009.662 · Cited by 19 · see citing works
Impact of utilizing forecasted network traffic for data transfers
Hoong, N. K., Hoong, P. K., Tan, I. K. T., et al. (2011) · International Conference on Advanced Communication Technology · Cited by 18 · see citing works
Syed, A. R., Burney, S. M. A., & Sami, B. (2010) · International Journal of Computer and Electrical Engineering · doi:10.7763/ijcee.2010.v2.262 · Cited by 16 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
Congestion prediction for smart sustainable cities using IoT and machine learning approaches
Majumdar, S., Subhani, M. M., Roullier, B., Anjum, A., & Zhu, R. (2021) · Sustainable Cities and Society · doi:10.1016/j.scs.2020.102500 · Cited by 182 · see citing works
- Authors
- Chen, Z., Wen, J., & Geng, Y.
- Affiliations
- Huawei Technologies (China) (CN)
- University of Hong Kong (HK)
- Venue
- 2016 IEEE 24th International Conference on Network Protocols (ICNP)
- Publisher
- IEEE
- Published
- 2016-11
- Pages
- 1-6
- ISSN
- 1092-1648 (ISSN-L)
- OpenAlex
- W2566871274
- Cited by
- 72
- References
- 19
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 9 · 2023: 4 · 2022: 6 · 2021: 14 · 2020: 15 · 2019: 12 · 2018: 4 · 2017: 2
- FWCI
- 5.13
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Network Protocols (ICNP) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Traffic and Congestion Control
- Physical Sciences › Computer Science › Artificial Intelligence › Internet Traffic Analysis and Secure E-voting
- Physical Sciences › Engineering › Building and Construction › Traffic Prediction and Management Techniques
- Keywords
- network traffic prediction, traffic volume estimation, hidden Markov models, flow statistics, traffic matrix estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
