CoChat
Sign Up
Cited by 112Software-Defined Networks and 5GCloud Computing and Resource ManagementSoftware System Performance and Reliability

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Network planning with deep reinforcement learning

Open access
Zhu, Hang, Gupta, Varun, Ahuja, Satyajeet Singh, Tian, Yuandong, Zhang, Ying, Jin, Xin·2021·Proceedings of the 2021 ACM SIGCOMM 2021 Conference
DOI 10.1145/3452296.3472902
Open this paper in CoChat — free accountOpen PDF

Abstract

Network planning is critical to the performance, reliability and cost of web services. This problem is typically formulated as an Integer Linear Programming (ILP) problem. Today's practice relies on hand-tuned heuristics from human experts to address the scalability challenge of ILP solvers.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3452296.3472902

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 70

  1. D. Bahdanau , P. Brakel , K. Xu , A. Goyal , R. Lowe , J. Pineau , A. Courville , and Y. Bengio . An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086 , 2016 . D. Bahdanau, P. Brakel, K. Xu, A. Goyal, R. Lowe, J. Pineau, A. Courville, and Y. Bengio. An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086, 2016. · arXiv
  2. I. Bello , H. Pham , Q. V. Le , M. Norouzi , and S. Bengio . Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940 , 2016 . I. Bello, H. Pham, Q. V. Le, M. Norouzi, and S. Bengio. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940, 2016. · arXiv
  3. Y. Bengio , A. Lodi , and A. Prouvost . Machine learning for combinatorial optimization: a methodological tour d'horizon. European Journal of Operational Research , 2020 . Y. Bengio, A. Lodi, and A. Prouvost. Machine learning for combinatorial optimization: a methodological tour d'horizon. European Journal of Operational Research, 2020.
  4. 10.5555/1097029 · DOI
  5. Q. Cappart , T. Moisan , L.-M. Rousseau , I. Prémont-Schwarz , and A. Cire . Combining reinforcement learning and constraint programming for combinatorial optimization. arXiv preprint arXiv:2006.01610 , 2020 . Q. Cappart, T. Moisan, L.-M. Rousseau, I. Prémont-Schwarz, and A. Cire. Combining reinforcement learning and constraint programming for combinatorial optimization. arXiv preprint arXiv:2006.01610, 2020. · arXiv
  6. Y. Chang , S. Rao , and M. Tawarmalani . Robust validation of network designs under uncertain demands and failures . In USENIX NSDI , 2017 . Y. Chang, S. Rao, and M. Tawarmalani. Robust validation of network designs under uncertain demands and failures. In USENIX NSDI, 2017.
  7. 10.1145/3230543.3230551 · DOI
  8. X. Chen and Y. Tian . Learning to perform local rewriting for combinatorial optimization . Advances in Neural Information Processing Systems , 2019 . X. Chen and Y. Tian. Learning to perform local rewriting for combinatorial optimization. Advances in Neural Information Processing Systems, 2019.
  9. D. Duvenaud , D. Maclaurin , J. Aguilera-Iparraguirre , R. Gómez-Bombarelli , T. Hirzel , A. Aspuru-Guzik , and R. P. Adams . Convolutional networks on graphs for learning molecular fingerprints. arXiv preprint arXiv:1509.09292 , 2015 . D. Duvenaud, D. Maclaurin, J. Aguilera-Iparraguirre, R. Gómez-Bombarelli, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams. Convolutional networks on graphs for learning molecular fingerprints. arXiv preprint arXiv:1509.09292, 2015. · arXiv
  10. M. Fey and J. E. Lenssen . Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 , 2019 . M. Fey and J. E. Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428, 2019. · arXiv
  11. 10.1109/INFCOM.2000.832225 · DOI
  12. 10.1109/MCOM.2014.6710063 · DOI
  13. A. Graves . Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 , 2013 . A. Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013. · arXiv
  14. L. Gu , D. Zeng , W. Li , S. Guo , A. Y. Zomaya , and H. Jin . Intelligent vnf orchestration and flow scheduling via model-assisted deep reinforcement learning. IEEE Journal on Selected Areas in Communications , 2019 . L. Gu, D. Zeng, W. Li, S. Guo, A. Y. Zomaya, and H. Jin. Intelligent vnf orchestration and flow scheduling via model-assisted deep reinforcement learning. IEEE Journal on Selected Areas in Communications, 2019.
  15. P. Gupta , M. Gasse , E. B. Khalil , M. P. Kumar , A. Lodi , and Y. Bengio . Hybrid models for learning to branch. arXiv preprint arXiv:2006.15212 , 2020 . P. Gupta, M. Gasse, E. B. Khalil, M. P. Kumar, A. Lodi, and Y. Bengio. Hybrid models for learning to branch. arXiv preprint arXiv:2006.15212, 2020. · arXiv
  16. 10.1145/2785956.2787495 · DOI
  17. Q. Huang , A. Haj-Ali , W. Moses , J. Xiang , I. Stoica , K. Asanovic , and J. Wawrzynek . Autophase: Juggling hls phase orderings in random forests with deep reinforcement learning. arXiv preprint arXiv:2003.00671 , 2020 . Q. Huang, A. Haj-Ali, W. Moses, J. Xiang, I. Stoica, K. Asanovic, and J. Wawrzynek. Autophase: Juggling hls phase orderings in random forests with deep reinforcement learning. arXiv preprint arXiv:2003.00671, 2020. · arXiv
  18. 10.1145/2486001.2486019 · DOI
  19. N. Jay , N. Rotman , B. Godfrey , M. Schapira , and A. Tamar . A deep reinforcement learning perspective on internet congestion control . In International Conference on Machine Learning , 2019 . N. Jay, N. Rotman, B. Godfrey, M. Schapira, and A. Tamar. A deep reinforcement learning perspective on internet congestion control. In International Conference on Machine Learning, 2019.
  20. Z. Jia , M. Zaharia , and A. Aiken . Beyond data and model parallelism for deep neural networks. arXiv preprint arXiv:1807.05358 , 2018 . Z. Jia, M. Zaharia, and A. Aiken. Beyond data and model parallelism for deep neural networks. arXiv preprint arXiv:1807.05358, 2018. · arXiv
  21. 10.1109/JSTQE.2004.826575 · DOI
  22. S. M. Kakade . A natural policy gradient . Advances in Neural Information Processing Systems , 2001 . S. M. Kakade. A natural policy gradient. Advances in Neural Information Processing Systems, 2001.
  23. E. Khalil , H. Dai , Y. Zhang , B. Dilkina , and L. Song . Learning combinatorial optimization algorithms over graphs . Advances in Neural Information Processing Systems , 2017 . E. Khalil, H. Dai, Y. Zhang, B. Dilkina, and L. Song. Learning combinatorial optimization algorithms over graphs. Advances in Neural Information Processing Systems, 2017.
  24. T. N. Kipf and M. Welling . Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016 . T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016. · arXiv
  25. V. Kirilin , A. Sundarrajan , S. Gorinsky , and R. K. Sitaraman . Rl-cache: Learning-based cache admission for content delivery. IEEE Journal on Selected Areas in Communications , 2020 . V. Kirilin, A. Sundarrajan, S. Gorinsky, and R. K. Sitaraman. Rl-cache: Learning-based cache admission for content delivery. IEEE Journal on Selected Areas in Communications, 2020.
  26. V. R. Konda and J. N. Tsitsiklis . Actor-critic algorithms . In Advances in Neural Information Processing Systems , 2000 . V. R. Konda and J. N. Tsitsiklis. Actor-critic algorithms. In Advances in Neural Information Processing Systems, 2000.
  27. W. Kool , H. Van Hoof , and M. Welling . Attention, learn to solve routing problems! arXiv preprint arXiv:1803.08475 , 2018 . W. Kool, H. Van Hoof, and M. Welling. Attention, learn to solve routing problems! arXiv preprint arXiv:1803.08475, 2018. · arXiv
  28. Y. Li , D. Tarlow , M. Brockschmidt , and R. Zemel . Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 , 2015 . Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493, 2015. · arXiv
  29. 10.1145/3341302.3342221 · DOI
  30. 10.1007/978-3-030-01246-5_2 · DOI
  31. 10.1145/3132747.3132759 · DOI
  32. Y. Liu , H. Zhang , W. Gongt , and D. Towsley . On the interaction between overlay routing and underlay routing . In IEEE INFOCOM , 2005 . Y. Liu, H. Zhang, W. Gongt, and D. Towsley. On the interaction between overlay routing and underlay routing. In IEEE INFOCOM, 2005.
  33. 10.1145/3098822.3098843 · DOI
  34. 10.1145/3341302.3342080 · DOI
  35. N. Mazyavkina , S. Sviridov , S. Ivanov , and E. Burnaev . Reinforcement learning for combinatorial optimization: A survey. arXiv preprint arXiv:2003.03600 , 2020 . N. Mazyavkina, S. Sviridov, S. Ivanov, and E. Burnaev. Reinforcement learning for combinatorial optimization: A survey. arXiv preprint arXiv:2003.03600, 2020. · arXiv
  36. 10.1145/3387514.3405859 · DOI
  37. A. Mirhoseini , A. Goldie , M. Yazgan , J. Jiang , E. Songhori , S. Wang , Y.-J. Lee , E. Johnson , O. Pathak , S. Bae , Chip placement with deep reinforcement learning. arXiv preprint arXiv:2004.10746 , 2020 . A. Mirhoseini, A. Goldie, M. Yazgan, J. Jiang, E. Songhori, S. Wang, Y.-J. Lee, E. Johnson, O. Pathak, S. Bae, et al. Chip placement with deep reinforcement learning. arXiv preprint arXiv:2004.10746, 2020. · arXiv
  38. J. E. Mitchell . Branch-and-cut algorithms for combinatorial optimization problems. Handbook of applied optimization , 2002 . J. E. Mitchell. Branch-and-cut algorithms for combinatorial optimization problems. Handbook of applied optimization, 2002.
  39. A. Mittal , A. Dhawan , S. Manchanda , S. Medya , S. Ranu , and A. Singh . Learning heuristics over large graphs via deep reinforcement learning. arXiv preprint arXiv:1903.03332 , 2019 . A. Mittal, A. Dhawan, S. Manchanda, S. Medya, S. Ranu, and A. Singh. Learning heuristics over large graphs via deep reinforcement learning. arXiv preprint arXiv:1903.03332, 2019. · arXiv
  40. V. Mnih , K. Kavukcuoglu , D. Silver , A. Graves , I. Antonoglou , D. Wierstra , and M. Riedmiller . Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 , 2013 . V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. · arXiv
  41. V. Mnih , K. Kavukcuoglu , D. Silver , A. A. Rusu , J. Veness , M. G. Bellemare , A. Graves , M. Riedmiller , A. K. Fidjeland , G. Ostrovski , S. Petersen , C. Beattie , A. Sadik , I. Antonoglou , H. King , D. Kumaran , D. Wierstra , S. Legg , and D. Hassabis . Human-level control through deep reinforcement learning . Nature , 2015 . V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis. Human-level control through deep reinforcement learning. Nature, 2015.
  42. T. Nishizeki and N. Chiba . Planar graphs: Theory and algorithms . Elsevier , 1988 . T. Nishizeki and N. Chiba. Planar graphs: Theory and algorithms. Elsevier, 1988.
  43. M. Padberg and G. Rinaldi . A branch-and-cut algorithm for the resolution of large-scale symmetric traveling salesman problems . SIAM review , 1991 . M. Padberg and G. Rinaldi. A branch-and-cut algorithm for the resolution of large-scale symmetric traveling salesman problems. SIAM review, 1991.
  44. H. Peng , J. Li , Y. He , Y. Liu , M. Bao , L. Wang , Y. Song , and Q. Yang . Large-scale hierarchical text classification with recursively regularized deep graph-cnn . In WWW , 2018 . H. Peng, J. Li, Y. He, Y. Liu, M. Bao, L. Wang, Y. Song, and Q. Yang. Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In WWW, 2018.
  45. F. Scarselli , M. Gori , A. C. Tsoi , M. Hagenbuchner , and G. Monfardini . The graph neural network model. IEEE Transactions on Neural Networks , 2008 . F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 2008.
  46. J. Schulman , P. Moritz , S. Levine , M. Jordan , and P. Abbeel . High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438 , 2015 . J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015. · arXiv
  47. 10.1364/JOCN.11.000547 · DOI
  48. 10.1109/SPAWC.2017.8227766 · DOI · Paper page
  49. Y. Tang , S. Agrawal , and Y. Faenza . Reinforcement learning for integer programming: Learning to cut . In International Conference on Machine Learning , 2020 . Y. Tang, S. Agrawal, and Y. Faenza. Reinforcement learning for integer programming: Learning to cut. In International Conference on Machine Learning, 2020.
  50. Y. Tian , J. Ma , Q. Gong , S. Sengupta , Z. Chen , J. Pinkerton , and C. L. Zitnick . Elf opengo: An analysis and open reimplementation of alphazero. arXiv preprint arXiv:1902.04522 , 2019 . Y. Tian, J. Ma, Q. Gong, S. Sengupta, Z. Chen, J. Pinkerton, and C. L. Zitnick. Elf opengo: An analysis and open reimplementation of alphazero. arXiv preprint arXiv:1902.04522, 2019. · arXiv
  51. M. Tornatore , G. Maier , and A. Pattavina . Wdm network design by ilp models based on flow aggregation. IEEE/ACM Transactions on Networking , 2007 . M. Tornatore, G. Maier, and A. Pattavina. Wdm network design by ilp models based on flow aggregation. IEEE/ACM Transactions on Networking, 2007.
  52. M. Trofin , Y. Qian , E. Brevdo , Z. Lin , K. Choromanski , and D. Li . Mlgo: a machine learning guided compiler optimizations framework. arXiv preprint arXiv:2101.04808 , 2021 . M. Trofin, Y. Qian, E. Brevdo, Z. Lin, K. Choromanski, and D. Li. Mlgo: a machine learning guided compiler optimizations framework. arXiv preprint arXiv:2101.04808, 2021. · arXiv
  53. A. Valadarsky , M. Schapira , D. Shahaf , and A. Tamar . Learning to route with deep rl . In NIPS Deep Reinforcement Learning Symposium , 2017 . A. Valadarsky, M. Schapira, D. Shahaf, and A. Tamar. Learning to route with deep rl. In NIPS Deep Reinforcement Learning Symposium, 2017.
  54. 10.1016/j.automatica.2010.02.018 · DOI
  55. P. Veličković , G. Cucurull , A. Casanova , A. Romero , P. Lio , and Y. Bengio . Graph attention networks. arXiv preprint arXiv:1710.10903 , 2017 . P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017. · arXiv
  56. 10.1109/TIT.2002.1003824 · DOI
  57. S. Verdú and S. Shamai . Spectral efficiency of cdma with random spreading. IEEE Transactions on Information Theory , 1999 . S. Verdú and S. Shamai. Spectral efficiency of cdma with random spreading. IEEE Transactions on Information Theory, 1999.
  58. J. Wang , D. Ding , H. Wang , C. Christensen , Z. Wang , H. Chen , and J. Li . Polyjuice: High-performance transactions via learned concurrency control . In USENIX OSDI , 2021 . J. Wang, D. Ding, H. Wang, C. Christensen, Z. Wang, H. Chen, and J. Li. Polyjuice: High-performance transactions via learned concurrency control. In USENIX OSDI, 2021.
  59. 10.1109/JLT.2012.2212180 · DOI
  60. Y. Wu and Y. Tian . Training agent for first-person shooter game with actor-critic curriculum learning . In ICLR , 2016 . Y. Wu and Y. Tian. Training agent for first-person shooter game with actor-critic curriculum learning. In ICLR, 2016.
  61. K. Xu , W. Hu , J. Leskovec , and S. Jegelka . How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 , 2018 . K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018. · arXiv
  62. 10.1145/3318464.3389770 · DOI
  63. J. Yao , R. Tao , R. Gu , J. Nieh , S. Jana , and G. Ryan . Distai: Data-driven automated invariant learning for distributed protocols . In USENIX OSDI , 2021 . J. Yao, R. Tao, R. Gu, J. Nieh, S. Jana, and G. Ryan. Distai: Data-driven automated invariant learning for distributed protocols. In USENIX OSDI, 2021.
  64. J. You , B. Liu , Z. Ying , V. Pande , and J. Leskovec . Graph convolutional policy network for goal-directed molecular graph generation . In Advances in Neural Information Processing Systems , 2018 . J. You, B. Liu, Z. Ying, V. Pande, and J. Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems, 2018.
  65. D. Zeng , L. Gu , S. Pan , J. Cai , and S. Guo . Resource management at the network edge: A deep reinforcement learning approach . IEEE Network , 2019 . D. Zeng, L. Gu, S. Pan, J. Cai, and S. Guo. Resource management at the network edge: A deep reinforcement learning approach. IEEE Network, 2019.
  66. 10.1109/COMST.2019.2904897 · DOI
  67. 10.1145/3292500.3330961 · DOI · Paper page
  68. J. Zhou , G. Cui , Z. Zhang , C. Yang , Z. Liu , and M. Sun . Graph neural networks: A review of methods and applications. arXiv preprint arXiv:1812.08434 , 2018 . J. Zhou, G. Cui, Z. Zhang, C. Yang, Z. Liu, and M. Sun. Graph neural networks: A review of methods and applications. arXiv preprint arXiv:1812.08434, 2018. · arXiv
  69. D. Zhuo , M. Ghobadi , R. Mahajan , A. Phanishayee , X. K. Zou , H. Guan , A. Krishnamurthy , and T. Anderson . RAIL: A case for redundant arrays of inexpensive links in data center networks . In USENIX NSDI , 2017 . D. Zhuo, M. Ghobadi, R. Mahajan, A. Phanishayee, X. K. Zou, H. Guan, A. Krishnamurthy, and T. Anderson. RAIL: A case for redundant arrays of inexpensive links in data center networks. In USENIX NSDI, 2017.
  70. B. Zoph and Q. V. Le . Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 , 2016 . B. Zoph and Q. V. Le. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578, 2016. · arXiv

Cited by

Sorted by citations

See all 112

Metadata

Authors
Zhu, H., Gupta, V., Ahuja, S. S., Tian, Y., Zhang, Y., & Jin, X.
Affiliations
  • Johns Hopkins University (US)
  • Meta (United States) (US)
  • Peking University (CN)
ORCID
Venue
Proceedings of the 2021 ACM SIGCOMM 2021 Conference
Publisher
ACM
Published
2021-08-09
Pages
258-271
DOI
10.1145/3452296.3472902
OpenAlex
W3191167321
Cited by
112
References
3
Citations by year
2026: 11 · 2025: 32 · 2024: 19 · 2023: 32 · 2022: 18
FWCI
15.94
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM SIGCOMM Conference (SIGCOMM) (publishedVersion)
Funders
NSF (National Science Foundation) (1813487,1918757)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software-Defined Networks and 5G
  • Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
Keywords
network planning, deep reinforcement learning, integer linear programming, web services, scalability, heuristic optimization, network reliability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review