The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Experience-driven Networking: A Deep Reinforcement Learning based Approach
Open accessAbstract
Modern communication networks have become very complicated and highly dynamic, which makes them hard to model, predict and control. In this paper, we develop a novel experience-driven approach that can learn to well control a communication network from its own experience rather than an accurate mathematical model, just as a human learns a new skill (such as driving, swimming, etc). Specifically, we, for the first time, propose to leverage emerging Deep Reinforcement Learning (DRL) for enabling model-free control in communication networks; and present a novel and highly effective DRL-based control framework, DRL-TE, for a fundamental networking problem: Traffic Engineering (TE). The proposed framework maximizes a widely-used utility function by jointly learning network environment and its dynamics, and making decisions under the guidance of powerful Deep Neural Networks (DNNs). We propose two new techniques, TE-aware exploration and actor-critic-based prioritized experience replay, to optimize the general DRL framework particularly for TE. To validate and evaluate the proposed framework, we implemented it in ns-3, and tested it comprehensively with both representative and randomly generated network topologies. Extensive packet-level simulation results show that 1) compared to several widely-used baseline methods, DRL-TE significantly reduces end-to-end delay and consistently improves the network utility, while offering better or comparable throughput; 2) DRL-TE is robust to network changes; and 3) DRL-TE consistently outperforms a state-of-the-art DRL method (for continuous control), Deep Deterministic Policy Gradient (DDPG), which, however, does not offer satisfying performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(2016) · Cited by 11,679
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
McKeown (2008) · doi:10.1145/1355734.1355746 · Cited by 8,513
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
Optimization flow control. I. Basic algorithm and convergence
Low, S. H., & Lapsely, D. E. (1999) · IEEE/ACM Transactions on Networking · doi:10.1109/90.811451 · Cited by 1,972
B4: experience with a globally-deployed software defined wan
Jain, S., Kumar, A., Mandal, S., Ong, J., Poutievski, L., Singh, A., Venkata, S., Wanderer, J., Zhou, J., Zhu, M., et al. (2013) · Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM · doi:10.1145/2486001.2486019 · Cited by 1,771
A tutorial on decomposition methods for network utility maximization
Palomar, D. P., & Chiang, M. (2006) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2006.879350 · Cited by 1,647
BRITE: an approach to universal topology generation
Medina, A., Lakhina, A., Matta, I., & Byers, J. (2002) · MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascot.2001.948886 · Cited by 1,226
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217
The Mathematics of Internet Congestion Control
(2005) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2004.841398 · Cited by 821
Traffic engineering in software defined networks
Agarwal, S., Kodialam, M., & Lakshman, T. V. (2013) · 2013 Proceedings IEEE INFOCOM · doi:10.1109/infcom.2013.6567024 · Cited by 547
Determining the initial states in forward-backward filtering
Gustafsson, F. (1996) · IEEE Transactions on Signal Processing · doi:10.1109/78.492552 · Cited by 479
TCP ex machina: computer-generated congestion control
Winstein, K., & Balakrishnan, H. (2013) · Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM · doi:10.1145/2486001.2486020 · Cited by 332
Deep Reinforcement Learning in Large Discrete Action Spaces
Dulac-Arnold, G., Evans, R., van Hasselt, H., Sunehag, P., Lillicrap, T., Hunt, J., Mann, T., Weber, T., Degris, T., & Coppin, B. (2015) · arXiv · doi:10.48550/arxiv.1512.07679 · Cited by 290
Q-PrOP: Sample-efficient policy gradient with an off-policy critic
Gu, S., Lillicrap, T., Ghahramani, Z., Turner, R. E., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.17863/cam.21294 · Cited by 174
RLTE: Reinforcement Learning for Traffic-Engineering
Einhorn, E., & Mitschele‐Thiel, A. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-70587-1_10 · Cited by 11
Shortest Path Routing in Unknown Environments: Is the Adaptive Optimal Strategy Available?
Zhou, P., Cheng, L., & Wu, D. O. (2016) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/sahcn.2016.7733015 · Cited by 6
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Deep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
RouteNet: Leveraging Graph Neural Networks for Network Modeling and Optimization in SDN
Rusek, K., Suarez-Varela, J., Almasan, P., Barlet-Ros, P., & Cabellos-Aparicio, A. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3000405 · Cited by 366
Liu, M., Yu, F. R., Teng, Y., Leung, V. C. M., & Song, M. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2897805 · Cited by 365
Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case
Almasan, P., Suárez-Varela, J., Rusek, K., Barlet-Ros, P., & Cabellos-Aparicio, A. (2022) · Computer Communications · doi:10.1016/j.comcom.2022.09.029 · Cited by 290
Metadata
- Authors
- Xu, Z., Tang, J., Meng, J., Zhang, W., Wang, Y., Liu, C. H., & Yang, D.
- Venue
- IEEE INFOCOM 2018 - IEEE Conference on Computer Communications
- Publisher
- IEEE
- Published
- 2018-04
- Pages
- 1871-1879
- OpenAlex
- W2963549123
- Cited by
- 439
- References
- 24
- FWCI
- 41.56
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.