The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Resource Management with Deep Reinforcement Learning
Abstract
Resource management problems in systems and networking often manifest as difficult online decision making tasks where appropriate solutions depend on understanding the workload and environment. Inspired by recent advances in deep reinforcement learning for AI problems, we consider building systems that learn to manage resources directly from experience. We present DeepRM, an example solution that translates the problem of packing tasks with multiple resource demands into a learning problem. Our initial results show that DeepRM performs comparably to state-of-the-art heuristics, adapts to different conditions, converges quickly, and learns strategies that are sensible in hindsight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resource management problems in practical settings.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [294]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. (2010) · Communications of the ACM · doi:10.1145/1721654.1721672 · Cited by 9,464
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
(1999) · Cited by 5,147
Hagan, M., Demuth, H. B., Beale, M. H., & Jes?s, O. D. (2016) · Institutional Repository of the National University Open and Distance UNAD (Universidad Nacional Abierta y a Distancia) · Cited by 5,133
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160
Dean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852
Apache Hadoop YARN: yet another resource negotiator
Vavilapalli, V. K., Murthy, A. C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Graves, T., Lowe, J., Shah, H., Seth, S., et al. (2013) · Proceedings of the 4th annual Symposium on Cloud Computing · doi:10.1145/2523616.2523633 · Cited by 1,848
Zaharia, M., Borthakur, D., Sarma, J. S., Elmeleegy, K., Shenker, S. J., & Stoica, I. G. (2010) · European Conference on Computer Systems · doi:10.1145/1755913.1755940 · Cited by 1,394
Dominant resource fairness: fair allocation of multiple resource types
Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S. J., & Stoica, I. G. (2011) · doi:10.5555/1972457.1972490 · Cited by 1,120
A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP
Yin, X., Jindal, A., Sekar, V., & Sinopoli, B. (2015) · ACM Special Interest Group on Data Communication · doi:10.1145/2785956.2787486 · Cited by 1,030
(2010) · doi:10.5555/1855711.1855728 · Cited by 970
Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., & Goldberg, A. V. (2009) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/1629575.1629601 · Cited by 856
Monahan, G. (1982) · Management Science · doi:10.1287/mnsc.28.1.1 · Cited by 736
Delimitrou, C., & Kozyrakis, C. (2014) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/2541940.2541941 · Cited by 704
Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach
Boyan, J. A., & Littman, M. L. (1994) · Cited by 694
Reining in the outliers in map-reduce clusters using Mantri
Ananthanarayanan, G., Kandula, S., Greenberg, A. G., Stoica, I. G., Lu, Y., Saha, B., & Harris, E. F. (2010) · Operating Systems Design and Implementation · doi:10.5555/1924943.1924962 · Cited by 660
Self-citationLearning policies for partially observable environments: Scaling up
Littman, M. L., Cassandra, A. R., & Kaelbling, L. P. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50052-9 · Cited by 659
Project Adam: building an efficient and scalable deep learning training system
Chilimbi, T. M., Suzue, Y., Apacible, J., & Kalyanaraman, K. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685094 · Cited by 614
An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0006 · Cited by 551
Neuro-dynamic programming: an overview
Bertsekas, D. P., & Tsitsiklis, J. N. (2002) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.1995.478953 · Cited by 435
Multi-resource packing for cluster schedulers
Grandl, R., Ananthanarayanan, G., Kandula, S., Rao, S., & Akella, A. (2014) · Proceedings of the 2014 ACM conference on SIGCOMM · doi:10.1145/2619239.2626334 · Cited by 421
Self-citationCS2P: Improving Video Bitrate Selection and Adaptation with Data-Driven Throughput Prediction
Sun, Y., Yin, X., Jiang, J., Sekar, V., Lin, F., Wang, N., Liu, T., & Sinopoli, B. (2016) · Proceedings of the 2016 ACM SIGCOMM Conference · doi:10.1145/2934872.2934898 · Cited by 407
A reinforcement learning approach to job-shop scheduling
Zhang, W., & Dietterich, T. G. (1995) · Cited by 378
Winstein, K., & Balakrishnan, H. (2013) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2486001.2486020 · Cited by 332
PCC: Re-architecting Congestion Control for Consistent High PerformancePreprint
Dong, M., Li, Q., Zarchy, D., Godfrey, B., & Schapira, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7092 · Cited by 298
Ferguson, A. D., Bodík, P., Kandula, S., Boutin, E., & Fonseca, R. (2012) · European Conference on Computer Systems · doi:10.1145/2168836.2168847 · Cited by 286
Self-citationStochastic forecasts achieve high throughput and low delay over cellular networks
Winstein, K., Sivaraman, A., & Balakrishnan, H. (2013) · Cited by 249
Machine Learning Proceedings 1995
(1995) · Elsevier eBooks · doi:10.1016/c2009-0-27705-1 · Cited by 226
Basis Function Adaptation in Temporal Difference Reinforcement Learning
Menache, I., Mannor, S., & Shimkin, N. (2005) · Annals of Operations Research · doi:10.1007/s10479-005-5732-z · Cited by 186
Self-citationRe-optimizing data-parallel computing
Agarwal, S., Kandula, S., Bruno, N., Wu, M.-C., Stoica, I. G., & Zhou, J. (2012) · Cited by 174
Self-citationJiang, J., Das, R., Ananthanarayanan, G., Chou, P. A., Padmanabhan, V. N., Sekar, V., Dominique, E., Goliszewski, M., Kukoleca, D., Vafin, R., et al. (2016) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/2934872.2934907 · Cited by 97
Optimizing Production Manufacturing Using Reinforcement Learning
Mahadevan, S., & Theocharous, G. (1998) · Cited by 64
Research works printed but not linked: 20
- Agarwal S., 2012, NSDI, 281
- Ananthanarayanan G., 2010, In OSDI, number, 1, 24
- Bertsekas D. P., 1995, Decision and Control
- Chilimbi T., 2014, OSDI, 571
- Dong M., 2015, NSDI, 395
- Ghodsi A., 2011, NSDI'11, 323
- Heller B., 2010, NSDI'10
- Junchen J., 2015, SIGCOMM '15
- Kaelbling L. P., 1996, Journal of artificial intelligence research
- Mahadevan S., 1998, FLAIRS Conference
- Mnih V., 2016, CoRR
- Mnih V., 2013, CoRR
- Mnih V., 2015, Nature
- J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. · arXiv
- 10.5555/551283 · DOI
- Sutton R. S., 1999, NIPS
- Winstein K., 2013, NSDI, 459
- Yi S., 2016, SIGCOMM
- Yin X., 2016, SIGCOMM '16
- Zhang W., 1995, IJCAI. Citeseer
Cited by
Sorted by citations
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D. T., Wang, P., Liang, Y., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,181
“The authors in [163] propose a DRL-based solution, DeepRM, by employing policy gradient methods [148] to manage resources in computer systems directly from experience.”
Method6G and Beyond: The Future of Wireless Communications Systems
Akyildiz, I. F., Kak, A., & Nie, S. (2020) · IEEE Access · doi:10.1109/access.2020.3010896 · Cited by 1,472
“Reinforcement Learning: Reinforcement learning typically finds use in problems associated with resource management [97], [98].”
BackgroundNeural Adaptive Video Streaming with Pensieve
Mao, H., Netravali, R., & Alizadeh, M. (2017) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3098822.3098843 · Cited by 1,434
“Pensieve uses modern reinforcement learning (RL) techniques [27, 29, 42] to learn a control policy for bitrate adaptation purely through experience.”
MethodSelf-citationOptimizing Federated Learning on Non-IID Data with Reinforcement Learning
Wang, H., Kaplan, Z., Niu, D., & Li, B. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155494 · Cited by 1,054
“The recent success of RL [12, 13] comes from the capability that RL agents learn from the interaction with a dynamic environment.”
BackgroundBoutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resourcemanagement problems in practical settings.”
BackgroundMethod
Metadata
- Authors
- Mao, H., Alizadeh, M., Menache, I., & Kandula, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 15th ACM Workshop on Hot Topics in Networks
- Publisher
- ACM
- Published
- 2016-11-09
- Pages
- 50-56
- OpenAlex
- W2546571074
- Cited by
- 1,353
- References
- 41
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 98 · 2023: 140 · 2022: 151 · 2021: 191 · 2020: 180 · 2019: 155 · 2018: 62 · 2017: 24
- FWCI
- 68.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Hot Topics in Networks (HotNets) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- resource management, deep reinforcement learning, task scheduling, resource allocation, workload modeling, online decision making, experiential learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
