The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Resource Management with Deep Reinforcement Learning
Abstract
Resource management problems in systems and networking often manifest as difficult online decision making tasks where appropriate solutions depend on understanding the workload and environment. Inspired by recent advances in deep reinforcement learning for AI problems, we consider building systems that learn to manage resources directly from experience. We present DeepRM, an example solution that translates the problem of packing tasks with multiple resource demands into a learning problem. Our initial results show that DeepRM performs comparably to state-of-the-art heuristics, adapts to different conditions, converges quickly, and learns strategies that are sensible in hindsight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resource management problems in practical settings.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [294]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. (2010) · Communications of the ACM · doi:10.1145/1721654.1721672 · Cited by 9,464
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Research works printed but not linked: 20
- Agarwal S., 2012, NSDI, 281
- Ananthanarayanan G., 2010, In OSDI, number, 1, 24
- Bertsekas D. P., 1995, Decision and Control
- Chilimbi T., 2014, OSDI, 571
- Dong M., 2015, NSDI, 395
- Ghodsi A., 2011, NSDI'11, 323
- Heller B., 2010, NSDI'10
- Junchen J., 2015, SIGCOMM '15
- Kaelbling L. P., 1996, Journal of artificial intelligence research
- Mahadevan S., 1998, FLAIRS Conference
- Mnih V., 2016, CoRR
- Mnih V., 2013, CoRR
- Mnih V., 2015, Nature
- J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. · arXiv
- 10.5555/551283 · DOI
- Sutton R. S., 1999, NIPS
- Winstein K., 2013, NSDI, 459
- Yi S., 2016, SIGCOMM
- Yin X., 2016, SIGCOMM '16
- Zhang W., 1995, IJCAI. Citeseer
Cited by
Sorted by citations
Items 201–300 of 1,000
A dynamic clustering technique based on deep reinforcement learning for Internet of vehicles
Sharif, A., Li, J., Saleem, M. A., Manogran, G., Kadry, S., Basit, A., & Khan, M. A. (2021) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-020-01722-7 · Cited by 43
“For instance, a solution is presented in Mao et al. (2016) through which the tasks have multiple resource requirements are translated into a reinforcement learning problem.”
BackgroundMethodZhao, J. W., Rodriguez, M. A., & Buyya, R. (2021) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud53861.2021.00037 · Cited by 45
“Recently, using DRL as a resource management and task scheduling technique was sparked by Mao et al. in their paper titled DeepRM [13], they formulated task packing problem into a learning problem and used the policy gradient method to achieve good performance.”
BackgroundMethodToward Decentralized and Collaborative Deep Learning Inference for Intelligent IoT Devices
Huang, Y.-K., Qiao, X.-Q., Dustdar, S., Zhang, J., & Li, J. (2022) · IEEE Network · doi:10.1109/mnet.011.2000639 · Cited by 38
“To guide the agent to minimize the average slowdown, we employ the similar defi nition of the reward R = Sj∈nt(–1/Tj), which has been illustrated to be eff ective in [13].”
MethodTapFinger: Task Placement and Fine-Grained Resource Allocation for Edge Machine Learning
Li, Y., Zeng, T., Zhang, X., Duan, J., & Wu, C. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10229031 · Cited by 42
“Deep Reinforcement learning (DRL) has demonstrated its effectiveness in online decision making including task scheduling and resource allocation [20], [21].”
BackgroundDRS: A deep reinforcement learning enhanced Kubernetes scheduler for microservice‐based system
Jian, Z., Xie, X., Fang, Y., Jiang, Y., Lu, Y., Dash, A., Li, T., & Wang, G. (2023) · Software Practice and Experience · doi:10.1002/spe.3284 · Cited by 45
“Specifically, we use the reward settings of three previous works (RLSK, 37 DeepRM, 22 and Multi DeepRM 38 ) respectively under the aforementioned experimental environment and workloads.”
Certified reinforcement learning with logic guidance
Hasanbeig, H., Kroening, D., & Abate, A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103949 · Cited by 67
Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation
Ahn, B. H., Pilligundla, P., Yazdanbakhsh, A., & Esmaeilzadeh, H. (2020) · PubMed · doi:10.48550/arxiv.2001.08743 · Cited by 56
“There are a growing body of studies on using reinforcement learning to perform various optimizations (Gao et al., 2018; Mirhoseini et al., 2017; Nareyek, 2003; Mao et al., 2016; Xu et al., 2018; Mao et al., 2019) for a variety of objectives including hyper-parameter optimization for neural networks.”
BackgroundRuntime Task Scheduling Using Imitation Learning for Heterogeneous Many-Core Systems
Krishnakumar, A., Arda, S. E., Goksoy, A. A., Mandal, S. K., Ogras, U. Y., Sartor, A. L., & Mărculescu, R. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012861 · Cited by 42
“shown promise when applied to the scheduling problem [20], [21], [37], but it suffers from slow convergence and sensitivity of the reward function [15], [18].”
BackgroundMethodInfluentialJob scheduling for large-scale machine learning clusters
Wang, H., Liu, Z., & Shen, H. (2020) · ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) · doi:10.1145/3386367.3432588 · Cited by 42
“MLF-RL is novel compared with the previous RL-based job schedulers [35, 37, 39] in that it additionally considers ML features to improve both JCT and accuracy while previous RL-based job schedulers do not aim to improve accuracy or consider ML features.”
BackgroundMethodInfluentialLearning to schedule multi-NUMA virtual machines via reinforcement learning
Sheng, J., Hu, Y., Zhou, W., Zhu, L., Jin, B., Wang, J., & Wang, X. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108254 · Cited by 42
Domain-Specific Architectures: Research Problems and Promising Approaches
Krishnakumar, A., Ogras, U. Y., Mărculescu, R., Kishinevsky, M., & Mudge, T. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3563946 · Cited by 48
“Task scheduling algorithms are broadly classiied into: 1) optimization-based approaches, 2) heuristic techniques, and 3) machine learning-based schedulers [120, 167, 193].”
Topology-Aware Scheduling Framework for Microservice Applications in Cloud
Li, X., Zhou, J., Wei, X., Li, D., Qian, Z., Wu, J., Qin, X., & Lu, S. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3238751 · Cited by 41
“[50] applied reinforcement learning to the problem of multi-resource packing for batch jobs and achieved good results in improving resource utilization.”
BackgroundDethise, A., Canini, M., & Kandula, S. (2019) · ACM SIGCOMM Workshop on Network Meets AI & ML (NetAI) · doi:10.1145/3341216.3342210 · Cited by 40
“Machine learning (ML) based solutions have recently been developed for several problems in networked systems such as resource allocation [10], routing [16], video rate selection [11] and congestion control [3].”
BackgroundSelf-citationPaeng, B., Park, I.-B., & Park, J. (2021) · IEEE Access · doi:10.1109/access.2021.3097254 · Cited by 40
“Until recently, a lot of studies widened the scope of DRL to scheduling problems in computer resource management [47], [48] and distributed system [49], [50].”
BackgroundMulti-resource constrained dynamic workshop scheduling based on proximal policy optimisation
Luo, P. C., Xiong, H. Q., Zhang, B. W., Peng, J. Y., & Xiong, Z. F. (2021) · International Journal of Production Research · doi:10.1080/00207543.2021.1975057 · Cited by 40
“The DeepRM2 model that only uses imitation training is called DMR2_WCO, and the two-stage trained model is called DRM2.”
BackgroundMethodInfluentialMachine Learning for Underwater Acoustic Communications
Huang, L., Wang, Y., Zhang, Q., Han, J., Tan, W., & Tian, Z. (2022) · IEEE Wireless Communications · doi:10.1109/mwc.2020.2000284 · Cited by 39
Solving Combinatorial Optimization Problems with Deep Neural Network: A Survey
Wang, F., He, Q., & Li, S. (2024) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010076 · Cited by 39
Grid-Wise Control for Multi-Agent Reinforcement Learning in Video Game AI.
Han, L., Sun, P., Du, Y., Xiong, J., Wang, Q., Sun, X., Liu, H., & Zhang, T. (2019) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 37
A Learning-Based Data Placement Framework for Low Latency in Data Center Networks
Liu, K., Peng, J., Wang, J., Yu, B., Liao, Z., Huang, Z., & Pan, J. (2019) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2019.2940953 · Cited by 39
“Mao et al. [23] used deep reinforcement learning to solve the resource management problem.”
BackgroundDeep Smart Scheduling: A Deep Learning Approach for Automated Big Data Scheduling Over the Cloud
Rjoub, G., Bentahar, J., Wahab, O. A., & Bataineh, A. S. (2019) · International Conference on Future Internet of Things and Cloud (FiCloud) · doi:10.1109/ficloud.2019.00034 · Cited by 39
“Reinforcement learning [19] is an active research and application area of machine learning that has been applied to solve problems in many disciplines, especially problems that are characterized by uncertainty where exact models are often infeasible.”
Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning
You, X., Li, X., Xu, Y., Feng, H., & Zhao, J. (2019) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.23919/wiopt47501.2019.9144110 · Cited by 39
“Recent successful applications include cloud resource allocation [20], adaptive bitrate video streaming”
BackgroundTopology Aware Deep Learning for Wireless Network Optimization
Zhang, S., Yin, B., Zhang, W., & Cheng, Y. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3179352 · Cited by 44
“This line of research is shown by a series of online network control problems [16], [33], [34].”
Online Task Scheduling for Fog Computing with Multi-Resource Fairness
Bian, S., Huang, X., & Shao, Z. (2019) · IEEE Vehicular Technology Conference · doi:10.1109/vtcfall.2019.8891573 · Cited by 38
“Algorithm Design: By adopting the idea of dominant resource fairness (DRF) [5] and applying DRL [6] [7], we propose FairTS, an efficient online task scheduling scheme that learns directly from experience to achieve ultra-low average task slowdown and ensure the multiresource fairness among tasks.”
MethodZhou, P., Xie, Y., Niu, B., Pu, L., Xu, Z., Jiang, H., & Huang, H. (2020) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2020.3038998 · Cited by 38
“DRL-based solutions have already intelligently solved problems from many aspects, such as resource allocation [30], adaptive streaming [31] and human-level game control [32], etc.”
BackgroundAccelerating Serverless Computing by Harvesting Idle Resources
Yu, H., Wang, H., Li, J., Yuan, X., & Park, S. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511979 · Cited by 42
“Similar to [24], we set the discount factor γ in Equation 4 to be 1.”
ResultNeuroViNE: A Neural Preprocessor for Your Virtual Network Embedding Algorithm
Blenk, A., Kalmbach, P., Zerwas, J., Jarschel, M., Schmid, S., & Kellerer, W. (2018) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2018.8486263 · Cited by 37
“Recent work has successfully applied deep neural networks [27] for cloud resource management.”
BackgroundPeng, Z., Zhang, Y., Feng, Y., Zhang, T., Wu, Z., & Su, H. (2019) · Chinese Automation Congress · doi:10.1109/cac48633.2019.8997498 · Cited by 37
“In recent years, motivated by the great success of Deep Reinforcement Learning, several DRL-based methods have been proposed to solve resource allocation problems and scheduling problems, such as resource management in cluster [7], device placement optimization [8], power allocation in satellites [9], railway lines sc…”
BackgroundA Machine Learning Approach to RoutingPreprint
Valadarsky, A., Schapira, M., Shahaf, D., & Tamar, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.03074 · Cited by 47
“, selecting a single action from a fairly small set [32, 33]).”
BackgroundMethodInfluentialRLSchert: An HPC Job Scheduler Using Deep Reinforcement Learning and Remaining Time Prediction
Wang, Q., Zhang, H., Qu, C., Shen, Y., Liu, X., & Li, J. (2021) · Applied Sciences · doi:10.3390/app11209448 · Cited by 36
“Combining RL with resource scheduling is also a new trend in recent years, such as DeepRM [15,16], CuSH [17], Decima [33], and RLScheuler [34].”
MethodInfluentialLearning-Based Resource Allocation in Cloud Data Center using Advantage Actor-Critic
Chen, Z., Hu, J. W., & Min, G. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761309 · Cited by 36
People-Centric Evolutionary System for Dynamic Production Scheduling
Nguyen, S., Zhang, M., Alahakoon, D., & Tan, K. C. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2936001 · Cited by 36
“There have been several studies on adaptive scheduling methods based on ML and online optimization but these methods are restricted by many assumptions and not flexible enough to cope with the complexity of smart factories [3], [4].”
BackgroundA Multi-Agent Reinforcement Learning Framework for Lithium-ion Battery Scheduling Problems
Sui, Y. F., & Song, S. (2020) · Energies · doi:10.3390/en13081982 · Cited by 38
“Reinforcement learning (RL) has b en a novel method in scheduling for applications such as job scheduling i clusters [8] and smart grids [9].”
MethodEnabling DNN Acceleration With Data and Model Parallelization Over Ubiquitous End Devices
Huang, Y., Qiao, X., Lai, W., Dustdar, S., Zhang, J., & Li, J. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3112715 · Cited by 36
“( i.e., at each time interval [18], [23].”
BackgroundMethodBoB: Bandwidth Prediction for Real-Time Communications Using Heuristic and Reinforcement Learning
Bentaleb, A., Akcay, M. N., Lim, M. L., Begen, A. C., & Zimmermann, R. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3216456 · Cited by 39
LIFT: Reinforcement Learning in Computer Systems by Learning From DemonstrationsPreprint
Schaarschmidt, M., Kuhnle, A., Ellis, B., Fricke, K., Gessert, F., & Yoneki, E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07903 · Cited by 51
“More recent examples include automated TensorFlow device placements [46, 47], client-side bit-rate selection for video streaming [43], and simplified cluster scheduling [42].”
BackgroundMethodQoS-Aware Scheduling of Heterogeneous Servers for Inference in Deep Neural Networks
Fang, Z., Yu, T., Mengshoel, O. J., & Gupta, R. K. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133045 · Cited by 35
“Reinforcement learning typically computes actions at regular time intervals ( e.g. , [6]).”
BackgroundMethodRadio Resource Scheduling for 5G NR via Deep Deterministic Policy Gradient
Tseng, S.-C., Liu, Z., Chou, Y.-C., & Huang, C. (2019) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2019.8757174 · Cited by 35
“In our design, we adopt a negative reward [10], since it is easier to measure the progress of transmission.”
MethodAlphaR: Learning-Powered Resource Management for Irregular, Dynamic Microservice Graph
Hou, X., Li, C., Liu, J., Zhang, L., Ren, S., Leng, J., Chen, Q., & Guo, M. (2021) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps49936.2021.00089 · Cited by 35
“DeepRM [18] uses RL to train a neural network for multidimensional resource packing.”
MethodLiu, H., Chen, P., Ouyang, X., Gao, H., Yan, B. C., Grosso, P., & Zhao, Z. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.03.029 · Cited by 39
“To achieve the goal of deadline guarantee, we define an execution slowdown value Q T j ( t ) [56] for each task, whose calculation is as follows: The f ( Q T j ( t )) function is meant to check whether execution slow-down of a task exceed the expected deadline or not.”
BackgroundMethodInfluentialDeep Reinforcement Learning based Distributed Resource Allocation for V2V Broadcasting
Ye, H., & Li, G. Y. (2018) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc.2018.8450518 · Cited by 36
“By combining the advanced deep learning techniques with reinforcement learning, impressive improvement has been shown in many applications, including playing Go games [17], playing video games [18], and job scheduling in computing clusters [19].”
BackgroundReinforcement learning for resource management in multi-tenant serverless platforms
Qiu, H., Mao, W., Patke, A., Wang, C., Franke, H., Kalbarczyk, Z., Başar, T., & Iyer, R. K. (2022) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3517207.3526971 · Cited by 35
“Since Mao et al. proposed a reinforcement learning (RL)- based solution for scheduling cluster resources [12], there has been a line of work using RL for resource management [4, 8, 11, 16, 18, 19, 32, 34].”
BackgroundYe, M., Hu, Y., Zhang, J., Guo, Z., & Chao, H. J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3191337 · Cited by 34
“Inspired by [41] and [42], we define the action space as {0, 1, …, N ∗ ( N − 1) − 1 } and allow each sub-model p j to sample K j different actions for each instance s (i.e., a 1 , a 2 , . . . , a K j ). c) Reward: For each sub-model p j , K j different router-destination pairs would be sampled for a…”
MethodWang, Y., & Yang, X. L. (2025) · International Conference on Advanced Algorithms and Control Engineering (ICAACE) · doi:10.1109/icaace65325.2025.11019615 · Cited by 34
“Mao et al. [13] found that DQN achieved optimal balance between performance and computational overhead for discrete allocation problems in vehicular edge computing networks.”
SCARL: Attentive Reinforcement Learning-Based Scheduling in a Multi-Resource Heterogeneous Cluster
Cheong, M., Lee, H., Yeom, I., & Woo, H. (2019) · IEEE Access · doi:10.1109/access.2019.2948150 · Cited by 34
“This simulation result is consistent with [6], demonstrating the feasibility of RL-based schedulers in the domain of cluster resource management.”
BackgroundMethodResultInfluentialResource Management in Cloud Computing Using Machine Learning: A Survey
Goodarzy, S., Nazari, M., Han, R., Keller, E., & Rozner, E. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00132 · Cited by 27
“In [31], the authors proposed a multi-resource cluster scheduler that schedules jobs online.”
BackgroundLi, P., Song, M., Xing, M., Xiao, Z., Ding, Q., Guan, S., & Long, J. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645386 · Cited by 34
“Deep Reinforcement Learning (DRL) [25] combines the strengths of deep learning [22] and RL and has been widely applied in sequential decision problems [28, 29, 44, 45].”
BackgroundImproving Resource Utilization in Data Centers using an LSTM-based Prediction Model
Thonglek, K., Ichikawa, K., Takahashi, K., Iida, H., & Nakasan, C. (2019) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/cluster.2019.8891022 · Cited by 36
“improved resource management with deep reinforcement learning [15].”
BackgroundREADYS: A Reinforcement Learning Based Strategy for Heterogeneous Dynamic Scheduling
Grinsztajn, N., Beaumont, O., Jeannot, E., & Preux, P. (2021) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/cluster48925.2021.00031 · Cited by 33
“In [35] the authors present a reinforcement learning approach to map independent parallel jobs onto a parallel machine.”
Bao, Y., Peng, Y., & Wu, C. (2022) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2022.3202529 · Cited by 35
“Our proposed ML cluster scheduler, Harmony, schedules jobs in a batch processing manner, similar to [25], [49], and [19].”
MethodDeep Reinforcement Learning Based VNF Management in Geo-distributed Edge Computing
Gu, L., Zeng, D., Li, W., Guo, S., Zomaya, A. Y., & Jin, H. (2019) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00097 · Cited by 32
“In order to make each agent observe a sufficient number of samples, the rates of the flows requesting the 10 services are randomly generated in range [0 , 11] during each episode.”
BackgroundMethodFraternali, F., Balaji, B., Sengupta, D., Hong, D., & Gupta, R. K. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3430734 · Cited by 34
“Studies show that RL agent is comparable and sometimes better than ad-hoc heuristics [48]; (ii) delayed reward – RL learns by maximizing cumulative reward in the long term: it can learn to sacrifice detection of a few events to spare energy for capturing more events later; (iii) pattern recognition – whenever present,…”
BackgroundAn Actor-Critic-Based Transfer Learning Framework for Experience-Driven Networking
Xu, Z., Yang, D., Tang, J., Tang, Y., Yuan, T., Wang, Y., & Xue, G. (2020) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2020.3037231 · Cited by 32
“As the first work to apply DRL for cluster resource scheduling, Mao et al. [34] proposed DeepRM that enables a cluster system to gradually learn the optimal resource scheduling policy from previous experiences.”
BackgroundMethodAn Edge Server Placement Method Based on Reinforcement Learning
Luo, F., Zheng, S., Ding, W., Fuentes, J., & Li, Y. (2022) · Entropy · doi:10.3390/e24030317 · Cited by 32
“Reinforcement learning is commonly used to solve sequence decision-making problems and find the optimal strategy by maximizing long-term rewards [32].”
BackgroundLi, J., Zhang, X., Wei, J., Ji, Z., & Wei, Z. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.04.032 · Cited by 33
Ye, M., Zhang, J., Guo, Z., & Chao, H. J. (2022) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2022.3217083 · Cited by 32
“Inspired by [29] and [30], we define the action space as { 0 , 1 , . . ., N ∗ ( N − 1) − 1 } and allow the agent to sample K different actions in each time step t (i.e., a 1 t , a 2 t , . . . , a Kt ).”
MethodResource Constrained Deep Reinforcement Learning
Bhatia, A., Varakantham, P. R., & Kumar, A. (2019) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v29i1.3528 · Cited by 31
Tang, F., Mao, B., Fadlullah, Z. M., Liu, J., & Kato, N. (2019) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2019.2929935 · Cited by 32
“The researchers in [19], [20], [21] proposed methods of using deep learning in resources allocation in network includes load off-loading, power management and channel allocation.”
BackgroundCost-Effective and Interpretable Job Skill Recommendation with Deep Reinforcement Learning
Sun, Y., Zhuang, F., Zhu, H., He, Q., & Xiong, H. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3449985 · Cited by 31
“In recent years, deep RL has been widely applied in many application scenarios, including games [17, 18, 23], traffic control [27], resources management [14], robotics [12], natural scientific researches [36] and recommendation [35].”
BackgroundIqbal, N., Khan, A. N., Imran, I., Rizwan, A., Qayyum, F., Malik, S., Ahmad, R., & Kim, D. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.05.015 · Cited by 31
EdgeTuner: Fast Scheduling Algorithm Tuning for Dynamic Edge-Cloud Workloads and Resources
Han, R., Wen, S., Liu, C. H., Yuan, Y., Wang, G., & Chen, L. Y. (2022) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom48880.2022.9796792 · Cited by 31
“Early work in this area applies reinforcement learning (RL) techniques to schedule jobs at particular time slots and minimize their latencies [20].”
BackgroundMethodLou, J., Tang, Z., & Jia, W. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3210204 · Cited by 31
“…work based on DRL technologies for resource allocation in the data center mainly needs heavy offline training of Deep Neural Network (DNN) [6], [15], [16], [17], which has the following shortcomings: (1) It requires a large amount of workload traces and elaborately generated state transition…”
BackgroundMethodInfluentialDesign and planning of flexible mobile Micro-Grids using Deep Reinforcement Learning
Caputo, C., Cardin, M., Ge, P., Teng, F., Korre, A., & del Rio‐Chanona, E. A. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.120707 · Cited by 31
An Empirical Analysis of Deep Learning for Cardinality EstimationPreprint
Ortíz, J., Bałazińska, M., Gehrke, J. E., & Keerthi, S. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06425 · Cited by 31
SpotDAG: An RL-Based Algorithm for DAG Workflow Scheduling in Heterogeneous Cloud Environments
Lin, L., Pan, L., & Liu, S. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3422828 · Cited by 31
Pang, H., Zhang, C., Wang, F., Liu, J., & Sun, L. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737395 · Cited by 30
Toward Greater Intelligence in Route Planning: A Graph-Aware Deep Learning Approach
Zhuang, Z., Wang, J., Qi, Q., Sun, H., & Liao, J. (2019) · IEEE Systems Journal · doi:10.1109/jsyst.2019.2922217 · Cited by 30
“ment [12].”
BackgroundTarget-based Resource Allocation for Deep Learning Applications in a Multi-tenancy System
Zheng, W., Song, Y. S., Guo, Z., Cui, Y., Gu, S., Mao, Y., & Cheng, L. (2019) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2019.8916403 · Cited by 30
“Besides human’s efforts, DeepRM [19] shows that machine learning itself can provide a viable alternative to human-generated heuristics for resource management.”
BackgroundData Centers Job Scheduling with Deep Reinforcement Learning
Liang, S., Yang, Z., Jin, F., & Chen, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-47436-2_68 · Cited by 30
“Specifically, a series of policy based deep reinforcement learning approaches have been proposed to manage CPU and memory for incoming jobs [10], schedule time-critical workloads [8], handle jobs with dependency [9], and schedule data centers with hundreds of nodes [2].”
BackgroundPrivacy-Preserving Energy Sharing Among Cloud Service Providers via Collaborative Job Scheduling
Sun, Y., Ding, Z., Yan, Y., Wang, Z., Dehghanian, P., & Lee, W. (2024) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2024.3482390 · Cited by 31
“Consequently, data-driven approaches, such as machine learning, have been widely adopted to solve such types of problems in both academic [24], [25], [26], [28], [29], [30] and industrial [38], [39], [40], [41], [42] sectors.”
Adaptive DAG Tasks Scheduling with Deep Reinforcement Learning
Wu, Q., Wu, Z. S., Yuehui, Z., & Cheng, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-05054-2_37 · Cited by 29
HQTimer: A Hybrid ${Q}$ -Learning-Based Timeout Mechanism in Software-Defined Networks
Li, Q., Huang, N., Wang, D., Li, X., Jiang, Y., & Song, Z. (2019) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2018.2890754 · Cited by 29
“resource management [21], video delivery [22], routing [23], etc.”
BackgroundRjoub, G., Wahab, O. A., Bentahar, J., & Bataineh, A. S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-65310-1_23 · Cited by 29
On the Future of Cloud Engineering
Bermbach, D., Chandra, A., Krintz, C., Gokhale, A. S., Slominski, A., Thamsen, L., Cavalcante, E., Guo, T., Brandić, I., & Wolski, R. (2021) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e52221.2021.00044 · Cited by 29
Li, J., Zhang, X., Wei, Z., Wei, J., & Ji, Z. (2021) · CCF Transactions on High Performance Computing · doi:10.1007/s42514-021-00083-8 · Cited by 29
“However, none of thees studies (Mao et al. 2016; Li and Hu 2019; Liang et al. 2020) considered the difference and exogeneity of jobs.”
BackgroundMethodDeep Reinforcement Learning with the Random Neural Network
Serrano, W. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104751 · Cited by 29
Zhu, M., Chen, Q., Gu, J., & Gu, P. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3172344 · Cited by 29
“Recently, DRL methods have been successfully applied to play games [25], realize human-level control [26], and solve multi-resource management problems [27].”
MethodNi, Y., Issa, M., Abraham, D., Imani, M., Yin, X., & Imani, M. (2022) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3489517.3530668 · Cited by 34
“Using DNN in the policy network of RL agents brings increased power to discover the optimal action decision, which helps shape the recent success of policy-based RL, including Atari games [12, 13], system resource management [14] and robot control [15].”
Huang, Y., Wu, S., Zhankui, Z. E. N. G., Kang, Z., Zhongcheng, M. U., & HUANG, H. (2022) · Chinese Journal of Aeronautics · doi:10.1016/j.cja.2022.12.012 · Cited by 29
A bibliometric analysis and review on reinforcement learning for transportation applications
Li, C., Bai, L., Yao, L., Waller, S. T., & Liu, W. (2023) · Transportmetrica B Transport Dynamics · doi:10.1080/21680566.2023.2179461 · Cited by 34
“In addition, RL can be used to improve communication network resource allocation efficiency (Mao et al. 2016), where the energy usage for data centres can be reduced.”
BackgroundInSS: An Intelligent Scheduling Orchestrator for Multi-GPU Inference With Spatio-Temporal Sharing
Han, Z., Zhou, R., Xu, C., Zeng, Y., & Zhang, R. (2024) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2024.3430063 · Cited by 29
A Slice Admission Policy Based on Reinforcement Learning for a 5G Flexible RAN
Raza, M. R., Natalino, C., Öhlén, P., Wosinska, L., & Paolo Monti, A. (2018) · European Conference on Optical Communication · doi:10.1109/ecoc.2018.8535483 · Cited by 28
An End-to-End Deep RL Framework for Task Arrangement in Crowdsourcing Platforms
Shan, C., Mamoulis, N., Cheng, R. C. K., Li, G., Li, X., & Qian, Y. (2020) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde48307.2020.00012 · Cited by 30
“Deep RL has experienced dramatic growth recently in multiple fields, such as games (AlphaGo) [20], [28], [30], robotics [9], natural language processing [21], computer vision [18], finance [7], computer systems [16], [19], [36] and recommender systems [37], [38].”
Review of Applications of Artificial Intelligence Algorithms in Pavement Management
Yang, X., & Zhang, Z. (2022) · Journal of Transportation Engineering Part B Pavements · doi:10.1061/jpeodx.0000373 · Cited by 34
CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing
Liu, Y., Ding, J., Zhang, Z.-L., & Liu, X. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671840 · Cited by 29
Policy Distillation and Value Matching in Multiagent Reinforcement Learning
Wadhwania, S., Kim, D.-K., Omidshafiei, S., & How, J. P. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967849 · Cited by 28
“Recent advances in reinforcement learning (RL) algorithms have led to novel solutions in challenging domains that range from resource management [1] to traffic control [2] to game playing [3], [4].”
BackgroundAutomated Cloud Provisioning on AWS using Deep Reinforcement LearningPreprint
Wang, Z., Gwon, C. S., Oates, T., & Iezzi, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.04305 · Cited by 28
“Others have used a feed forward neural network (Xu, Rao, and Bu 2011; Mao et al. 2016), but these studies predominantly used a simplified state space and have not explored deep reinforcement learning methods on continuous time series.”
A View on Deep Reinforcement Learning in System OptimizationPreprint
Haj-Ali, A., Ahmed, N. K., Willke, T. L., Gonzalez, J. E., Asanović, K., & Stoica, I. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01275 · Cited by 43
“This might explain why most system optimization challenges tackled with RL were in the cloud (Mao et al., 2016; He et al., 2017a;b; Tesauro et al., 2006; Xu et al., 2017; Liu et al., 2017; Xu et al., 2012; Rao et al., 2009).”
BackgroundInfluentialEnergy-aware scheduling for spark job based on deep reinforcement learning in cloud
Li, H., Lu, L., Shi, W., Tan, G., & Luo, H. (2023) · Computing · doi:10.1007/s00607-023-01171-z · Cited by 27
MRLCC: an adaptive cloud task scheduling method based on meta reinforcement learning
Xiu, X., Li, J., Long, Y., & Wu, W. (2023) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-023-00440-8 · Cited by 30
“For example, DeepRM [11] uses deep reinforcement learning to schedule tasks by representing the states as images, GoSu [12] applies graph convolution network to task scheduling, and Peng et al. [13] solved the problem of task scheduling using Q-learning.”
MethodModel-Based Transfer Reinforcement Learning Based on Graphical Model Representations
Sun, Y., Zhang, K., & Sun, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3107375 · Cited by 28
“It has been successful in various sequential decision-making problems and has rapidly expanded to real-world applications such as robotics [1]–[4], games [5]–[7], recommendation sys-tem [8], traffic signal control [9], resources management [10], and so on [11]–[13].”
BackgroundEnabling Robust DRL-Driven Networking Systems via Teacher-Student Learning
Zheng, Y., Lin, L., Zhang, T., Chen, H., Duan, Q., Xu, Y., & Wang, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126085 · Cited by 27
A Survey on Trusted Distributed Artificial Intelligence
Ağca, M. A., Faye, S., & Khadraoui, D. (2022) · IEEE Access · doi:10.1109/access.2022.3176385 · Cited by 30
“Therefore, system resource modeling and management paradigms can be improved with novel learning heuristics [59].”
BackgroundLearning to Solve Network Flow Problems via Neural DecodingPreprint
Chen, Y., & Zhang, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04091 · Cited by 27
“In the cases of [3], [23], [24], the authors describe reinforcement learning formulation to model the interaction between learners’ decisions and rewards, but there is no guarantee the solutions satisfy all constraints.”
BackgroundBlenk, A., Kalmbach, P., Kellerer, W., & Schmid, S. (2017) · doi:10.1145/3098593.3098597 · Cited by 26
“Just recently, the concept of deep reinforcement learning has been applied to resource management of cloud network resources [16].”
BackgroundLiu, L., & Xu, H. L. (2018) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3267809.3267818 · Cited by 26
Zhu, X. X., Montazeri, S., Ali, M., Hua, Y., Wang, Y., Mou, L., Shi, Y., Xu, F., & Bamler, R. (2020) · elib (German Aerospace Center) · doi:10.48550/arxiv.2006.10027 · Cited by 26
“DeepRL uses different types of neural networks to create these functions [41][42].”
MethodDing, L., Zhao, D. Y., Cao, M., & Ma, H. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3062569 · Cited by 25
DATE: Disturbance-Aware Traffic Engineering with Reinforcement Learning in Software-Defined Networks
Ye, M., Zhang, J., Guo, Z., & Chao, H. J. (2021) · International Workshop on Quality of Service · doi:10.1109/iwqos52092.2021.9521343 · Cited by 26
Knowledge-defined networking: Applications, challenges and future work
Ashtari, S., Zhou, I., Abolhasan, M., Shariati, N., Lipman, J., & Ni, W. (2022) · Array · doi:10.1016/j.array.2022.100136 · Cited by 29
Zhang, D., Dai, D., & Xie, B. (2022) · ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) · doi:10.1145/3502181.3531470 · Cited by 26
“Such a strategy is commonly used in other RL-based system optimization work [21, 39] as they expect the deep neural network to learn the useful features out of raw inputs automatically.”
Method
Metadata
- Authors
- Mao, H., Alizadeh, M., Menache, I., & Kandula, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 15th ACM Workshop on Hot Topics in Networks
- Publisher
- ACM
- Published
- 2016-11-09
- Pages
- 50-56
- OpenAlex
- W2546571074
- Cited by
- 1,353
- References
- 41
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 98 · 2023: 140 · 2022: 151 · 2021: 191 · 2020: 180 · 2019: 155 · 2018: 62 · 2017: 24
- FWCI
- 68.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Hot Topics in Networks (HotNets) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- resource management, deep reinforcement learning, task scheduling, resource allocation, workload modeling, online decision making, experiential learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
