The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
AuTO: scaling deep reinforcement learning for datacenter-scale automatic traffic optimization
Open accessAbstract
Traffic optimizations (TO, e.g. flow scheduling, load balancing) in datacenters are difficult online decision-making problems. Previously, they are done with heuristics relying on operators' understanding of the workload and environment. Designing and implementing proper TO algorithms thus take at least weeks. Encouraged by recent successes in applying deep reinforcement learning (DRL) techniques to solve complex online control problems, we study if DRL can be used for automatic TO without human-intervention. However, our experiments show that the latency of current DRL systems cannot handle flow-level TO at the scale of current datacenters, because short flows (which constitute the majority of traffic) are usually gone before decisions can be made.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
(2017) · Cited by 11,625
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
McKeown (2008) · doi:10.1145/1355734.1355746 · Cited by 8,513
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091
(1999) · Cited by 5,147
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Network traffic characteristics of data centers in the wild
Benson, T., Akella, A., & Maltz, D. A. (2010) · Proceedings of the 10th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1879141.1879175 · Cited by 2,214
VL2: a scalable and flexible data center network
Greenberg, A., Hamilton, J. R., Jain, N., Kandula, S., Kim, C., Lahiri, P., Maltz, D. A., Patel, P., & Sengupta, S. (2009) · Proceedings of the ACM SIGCOMM 2009 conference on Data communication · doi:10.1145/1592568.1592576 · Cited by 2,111
Alizadeh, M., Greenberg, A., Maltz, D. A., Padhye, J., Patel, P., Prabhakar, B., Sengupta, S., & Sridharan, M. (2010) · Proceedings of the ACM SIGCOMM 2010 conference · doi:10.1145/1851182.1851192 · Cited by 1,877
Hedera: dynamic flow scheduling for data center networks
Al-Fares, M., Radhakrishnan, S., Raghavan, B., Huang, N., & Vahdat, A. M. (2010) · doi:10.5555/1855711.1855730 · Cited by 1,450
Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989385 · Cited by 1,424
Neural Adaptive Video Streaming with Pensieve
Mao, H., Netravali, R., & Alizadeh, M. (2017) · Proceedings of the Conference of the ACM Special Interest Group on Data Communication · doi:10.1145/3098822.3098843 · Cited by 1,357
Resource Management with Deep Reinforcement Learning
Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) · Proceedings of the 15th ACM Workshop on Hot Topics in Networks · doi:10.1145/3005745.3005750 · Cited by 1,353
Some Practical Guidelines for Effective Sample Size Determination
Lenth, R. V. (2001) · The American Statistician · doi:10.1198/000313001317098149 · Cited by 1,139
The nature of data center traffic: measurements & analysis
Kandula, S., Sengupta, S., Greenberg, A., Patel, P., & Chaiken, R. (2009) · Proceedings of the 9th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1644893.1644918 · Cited by 1,030
Analysis of an Equal-Cost Multi-Path Algorithm
Hopps, C. (2000) · doi:10.17487/rfc2992 · Cited by 965
Helios: a hybrid electrical/optical switch architecture for modular data centers
Farrington, N., Porter, G., Radhakrishnan, S., Bazzaz, H. H., Subramanya, V., Fainman, Y., Papen, G., & Vahdat, A. (2010) · Proceedings of the ACM SIGCOMM 2010 conference · doi:10.1145/1851182.1851223 · Cited by 918
Inside the Social Network's (Datacenter) Network
Roy, A., Zeng, H., Bagga, J., Porter, G., & Snoeren, A. C. (2015) · Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication · doi:10.1145/2785956.2787472 · Cited by 849
SEDA: an architecture for well-conditioned, scalable internet services
Welsh, M., Culler, D., & Brewer, E. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502057 · Cited by 834
Dell, R. B., Holleran, S., & Ramakrishnan, R. (2002) · ILAR Journal · doi:10.1093/ilar.43.4.207 · Cited by 741
Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach
Boyan, J. A., & Littman, M. L. (1994) · Cited by 738
Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network
Singh, A., Ong, J., Agarwal, A., Anderson, G., Armistead, A., Bannon, R., Boving, S., Desai, G., Felderman, B., Germano, P., et al. (2015) · Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication · doi:10.1145/2785956.2787508 · Cited by 556
Finishing flows quickly with preemptive scheduling
Hong, C.-Y., Caesar, M., & Godfrey, P. B. (2012) · Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication · doi:10.1145/2342356.2342389 · Cited by 451
Efficient coflow scheduling with Varys
Chowdhury, M., Zhong, Y., & Stoica, I. (2014) · Proceedings of the 2014 ACM conference on SIGCOMM · doi:10.1145/2619239.2626315 · Cited by 438
TCP ex machina: computer-generated congestion control
Winstein, K., & Balakrishnan, H. (2013) · Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM · doi:10.1145/2486001.2486020 · Cited by 332
Information-agnostic flow scheduling for commodity data centers
Bai, W., Chen, L., Chen, K., Han, D., Tian, C., & Wang, H. (2015) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 274
Self-citationEfficient Coflow Scheduling Without Prior Knowledge
Chowdhury, M., & Stoica, I. (2015) · Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication · doi:10.1145/2785956.2787480 · Cited by 250
Minimizing flow completion times in data centers
Munir, A., Qazi, I. A., Uzmi, Z. A., Mushtaq, A., Ismail, S. N., Iqbal, M. S., & Khan, B. (2013) · 2013 Proceedings IEEE INFOCOM · doi:10.1109/infcom.2013.6567018 · Cited by 200
CODA: Toward Automatically Identifying and Scheduling Coflows in the Dark
Zhang, H., Chen, L., Yi, B., Chen, K., Chowdhury, M., & Geng, Y. (2016) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2934872.2934880 · Cited by 188
Self-citationIncremental Natural Actor-Critic Algorithms
Bhatnagar, S., Ghavamzadeh, M., Lee, M., & Sutton, R. S. (2007) · Cited by 155
Scheduling Mix-flows in Commodity Datacenters with Karuna
Chen, L., Chen, K., Bai, W., & Alizadeh, M. (2016) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2934872.2934888 · Cited by 152
Self-citationWrangler: Predictable and Faster Jobs using Fewer Resources
Yadwadkar, N. J., Ananthanarayanan, G., & Katz, R. H. (2014) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2670979.2671005 · Cited by 142
Taking the Blame Game out of Data Centers Operations with NetPoirot
Arzani, B., Ciraci, S., Loo, B. T., Schuster, A., & Outhred, G. (2016) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2934872.2934884 · Cited by 111
PIAS: Practical Information-Agnostic Flow Scheduling for Commodity Data Centers
Bai, W., Chen, L., Chen, K., et al. (2017) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2017.2669216 · Cited by 109
Self-citationExplicit Path Control in Commodity Data Centers: Design and Applications
Hu, S., Chen, K., Wu, H., Bai, W., Lan, C., Wang, H., Zhao, H., & Guo, C. (2015) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2482988 · Cited by 91
Self-citation
Cited by
Deep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case
Almasan, P., Suárez-Varela, J., Rusek, K., Barlet-Ros, P., & Cabellos-Aparicio, A. (2022) · Computer Communications · doi:10.1016/j.comcom.2022.09.029 · Cited by 290
Deep reinforcement learning: a survey
Wang, H.-N., Liu, N., Zhang, Y.-Y., Feng, D.-W., Huang, F., Li, D.-S., & Zhang, Y.-M. (2020) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900533 · Cited by 274
NetLLM: adapting large language models for networking
Wu, D., Wang, X., Qiao, Y., Wang, Z., Jiang, J., Cui, S., & Wang, F. (2024) · Proceedings of the ACM SIGCOMM 2024 Conference · doi:10.1145/3651890.3672268 · Cited by 167
“Many networking tasks such as ABR [103] and CJS [78] employ RL to solve complex system optimization problems, which involve active interaction between the environments (e.g., network environments with varying band-width or workload patterns) to collect experiences for reward optimization.”
BackgroundMethodNetwork planning with deep reinforcement learning
Zhu, H., Gupta, V., Ahuja, S. S., Tian, Y., Zhang, Y., & Jin, X. (2021) · Proceedings of the 2021 ACM SIGCOMM 2021 Conference · doi:10.1145/3452296.3472902 · Cited by 112
Metadata
- Authors
- Chen, L., Lingys, J., Chen, K., & Liu, F.
- Venue
- Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication
- Publisher
- ACM
- Published
- 2018-08-07
- Pages
- 191-205
- OpenAlex
- W2824027552
- Cited by
- 282
- References
- 38
- FWCI
- 27.78
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.