The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Resource Management with Deep Reinforcement Learning
Abstract
Resource management problems in systems and networking often manifest as difficult online decision making tasks where appropriate solutions depend on understanding the workload and environment. Inspired by recent advances in deep reinforcement learning for AI problems, we consider building systems that learn to manage resources directly from experience. We present DeepRM, an example solution that translates the problem of packing tasks with multiple resource demands into a learning problem. Our initial results show that DeepRM performs comparably to state-of-the-art heuristics, adapts to different conditions, converges quickly, and learns strategies that are sensible in hindsight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resource management problems in practical settings.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [294]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. (2010) · Communications of the ACM · doi:10.1145/1721654.1721672 · Cited by 9,464
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Research works printed but not linked: 20
- Agarwal S., 2012, NSDI, 281
- Ananthanarayanan G., 2010, In OSDI, number, 1, 24
- Bertsekas D. P., 1995, Decision and Control
- Chilimbi T., 2014, OSDI, 571
- Dong M., 2015, NSDI, 395
- Ghodsi A., 2011, NSDI'11, 323
- Heller B., 2010, NSDI'10
- Junchen J., 2015, SIGCOMM '15
- Kaelbling L. P., 1996, Journal of artificial intelligence research
- Mahadevan S., 1998, FLAIRS Conference
- Mnih V., 2016, CoRR
- Mnih V., 2013, CoRR
- Mnih V., 2015, Nature
- J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. · arXiv
- 10.5555/551283 · DOI
- Sutton R. S., 1999, NIPS
- Winstein K., 2013, NSDI, 459
- Yi S., 2016, SIGCOMM
- Yin X., 2016, SIGCOMM '16
- Zhang W., 1995, IJCAI. Citeseer
Cited by
Sorted by citations
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D. T., Wang, P., Liang, Y., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,181
“The authors in [163] propose a DRL-based solution, DeepRM, by employing policy gradient methods [148] to manage resources in computer systems directly from experience.”
Method6G and Beyond: The Future of Wireless Communications Systems
Akyildiz, I. F., Kak, A., & Nie, S. (2020) · IEEE Access · doi:10.1109/access.2020.3010896 · Cited by 1,472
“Reinforcement Learning: Reinforcement learning typically finds use in problems associated with resource management [97], [98].”
BackgroundNeural Adaptive Video Streaming with Pensieve
Mao, H., Netravali, R., & Alizadeh, M. (2017) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3098822.3098843 · Cited by 1,434
“Pensieve uses modern reinforcement learning (RL) techniques [27, 29, 42] to learn a control policy for bitrate adaptation purely through experience.”
MethodSelf-citationOptimizing Federated Learning on Non-IID Data with Reinforcement Learning
Wang, H., Kaplan, Z., Niu, D., & Li, B. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155494 · Cited by 1,054
“The recent success of RL [12, 13] comes from the capability that RL agents learn from the interaction with a dynamic environment.”
BackgroundBoutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resourcemanagement problems in practical settings.”
BackgroundMethodDeep Reinforcement Learning Based Resource Allocation for V2V Communications
Ye, H., Li, G. Y., & Juang, B.-H. F. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2897134 · Cited by 870
Layer-Wise Relevance Propagation: An Overview
Montavon, G., Binder, A., Lapuschkin, S., Samek, W., & Müller, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_10 · Cited by 883
Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
“[90] proposed to use reinforcement learning in”
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
Learning scheduling algorithms for data processing clusters
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019) · ACM Special Interest Group on Data Communication · doi:10.1145/3341302.3342080 · Cited by 831
“For example, if the objective is to minimize average JCT, Decima penalizes the agent −τ × J at each time step, where τ is the absolute time (in seconds) since last action and J is the number of jobs in the system [28].”
BackgroundMethodInfluentialSelf-citationAntonopoulos, I., Robu, V., Couraud, B., Kirli, D., Norbu, S., Kiprakis, A., Flynn, D., Elizondo-Gonzalez, S., & Wattam, S. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109899 · Cited by 656
“The RL framework has been applied to a number of domains with the most important being robotics [102,103], resource management in computer clusters [104], playing video games from pixel input [105], automated ML frameworks [106].”
Cellular, Wide-Area, and Non-Terrestrial IoT: A Survey on 5G Advances and the Road Toward 6G
Vaezi, M., Azari, A., Khosravirad, S. R., Shirvanimoghaddam, M., Azari, M. M., Chasaki, D., & Popovski, P. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3151028 · Cited by 685
Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning
Liang, L., Ye, H., & Li, G. Y. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933962 · Cited by 559
“In particular, recent success of deep RL in humanlevel video game play [7] and AlphaGo [8] has sparked a flurry of interest in applying RL techniques to solve problems from a wide variety of areas and remarkable progress has been made ever since [9], [10], [11].”
BackgroundApplications of artificial intelligence for disaster management
Sun, W., Bocchini, P., & Davison, B. D. (2020) · Natural Hazards · doi:10.1007/s11069-020-04124-3 · Cited by 491
Machine Learning for Networking: Workflow, Advances and Opportunities
Wang, M., Cui, Y., Wang, X., Xiao, S., & Jiang, J. (2017) · IEEE Network · doi:10.1109/mnet.2017.1700200 · Cited by 448
“[13] H.”
BackgroundInfluentialSemisupervised Deep Reinforcement Learning in Support of IoT and Smart City Services
Mohammadi, M., Al‐Fuqaha, A., Guizani, M., & Oh, J.-S. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2712560 · Cited by 441
“[22] formulated the problem of job scheduling with multiple resource demands as a DRL process.”
BackgroundMestres, A., Rodriguez-Natal, A., Carner, J., Barlet‐Ros, P., Alarcón, E., Solé, M., Muntés-Mulero, V., Meyer, D., Barkai, S., Hibbett, M. J., et al. (2017) · ACM SIGCOMM Computer Communication Review · doi:10.1145/3138808.3138810 · Cited by 430
Hussain, F. A., Hassan, S. A., Hussain, R., & Hossain, E. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2964534 · Cited by 411
“allocation in wireless systems is presented in [78] where the authors used DRL algorithms to manage resources efficiently in the presence of interference and noise.”
MethodOptimization of global production scheduling with deep reinforcement learning
Waschneck, B., Reichstaller, A., Belzner, L., Altenmüller, T., Bauernhansl, T., Knapp, A., & Kyek, A. (2018) · Procedia CIRP · doi:10.1016/j.procir.2018.03.212 · Cited by 388
Deep Learning Meets SAR: Concepts, models, pitfalls, and perspectives
Zhu, X. X., Montazeri, S., Ali, M., Hua, Y., Wang, Y., Mou, L., Shi, Y., Xu, F., & Bamler, R. (2021) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2020.3046356 · Cited by 422
Device Placement Optimization with Reinforcement LearningPreprint
Mirhoseini, A., Pham, H., Le, Q. V., Steiner, B., Larsen, R., Zhou, Y., Kumar, N., Norouzi, M., Samy Bengio, A., & Dean, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04972 · Cited by 367
“For example, Mao et al. (2016) pro-pose to train a resource management algorithm with policy gradients.”
BackgroundExplainability in deep reinforcement learning
Heuillet, A., Couthouis, F., & Díaz-Rodríguez, N. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106685 · Cited by 374
Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey
Chen, W., Qiu, X., Cai, T., Dai, H., Zheng, Z., & Zhang, Y. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3073036 · Cited by 334
“With one step further, Mao et al. in [130] considered dispatching tasks with multiple resource demands and proposed a novel policy gradient method that promotes the monotonic improvement of policy iteration.”
BackgroundResource Allocation Based on Deep Reinforcement Learning in IoT Edge Computing
Xiong, X., Zheng, K., Lei, L., & Hou, L. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.2986615 · Cited by 301
“time [35].”
Multi-Objective Workflow Scheduling With Deep-Q-Network-Based Multi-Agent Reinforcement Learning
Wang, Y., Liu, H., Zheng, W., Xia, Y., Li, Y., Chen, P., Guo, K., & Xie, H. L. (2019) · IEEE Access · doi:10.1109/access.2019.2902846 · Cited by 297
Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement LearningPreprint
Zhang, C., Song, W. W., Cao, Z., Zhang, J., Tan, P. S., & Xu, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12367 · Cited by 305
“In [23], a DRL agent is proposed to learn job scheduling policies for a compute cluster.”
BackgroundChen, L., Lingys, J., Chen, K., & Liu, F. (2018) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3230543.3230551 · Cited by 289
Deep reinforcement learning: a survey
Wang, H.-N., Liu, N., Zhang, Y., Feng, D., Huang, F., Li, D., & Zhang, Y. (2020) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900533 · Cited by 297
“Tackling multi-resource cluster scheduling with a PG algorithm (Mao et al., 2016) optimizes various objectives like average job slowdown or completion time in an online manner with dynamic job arrivals, and validates the approach via simulation.”
MethodQi, Q., Wang, J., Ma, Z., Sun, H., Cao, Y., Zhang, L., & Liao, J. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2894437 · Cited by 268
“Mao et al. [22] find the optimal resource management policy by DRL.”
BackgroundTuli, S., Ilager, S., Ramamohanarao, K., & Buyya, R. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3017079 · Cited by 276
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 290
“Reading maps [329], face hallucination [330], view planning [331] and semantic parsing [332] are other applications of RL in computer vision.”
MethodDirect shape optimization through deep reinforcement learning
Viquerat, J., Rabault, J., Kuhnle, A., Ghraieb, H., Larcher, A., & Hachem, E. (2020) · Journal of Computational Physics · doi:10.1016/j.jcp.2020.110080 · Cited by 262
Deep Reinforcement Learning for Resource Allocation in V2V Communications
Ye, H., & Li, G. Y. (2018) · 2018 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2018.8422586 · Cited by 222
Li, C. (2019) · Chapman & Hall/CRC eBooks · doi:10.1201/9781351006620-6 · Cited by 257
Migration Modeling and Learning Algorithms for Containers in Fog Computing
Tang, Z., Zhou, X., Zhang, F., Weijia Jia, B., & Zhao, W. (2018) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2018.2827070 · Cited by 229
“[58] use the deep reinforcement learning in network resource scheduling.”
MethodUsing GANs for Sharing Networked Time Series Data
Lin, Z., Jain, A., Wang, C., Fanti, G. C., & Sekar, V. (2020) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3419394.3423643 · Cited by 278
“Data-driven techniques [51] are central to networking and systems research (e.g., [11, 16, 23, 48, 60, 60, 71, 74, 83, 104]).”
BackgroundCellular Network Traffic Scheduling With Deep Reinforcement Learning
Chinchali, S., Hu, P., Chu, T., Sharma, M., Bansal, M., Misra, R., Pavone, M., & Katti, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11339 · Cited by 218
“A recent application of RL to job scheduling in internet data centers is deepRM (Mao et al. 2016).”
BackgroundMethodValadarsky, A., Schapira, M., Shahaf, D., & Tamar, A. (2017) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3152434.3152441 · Cited by 238
Routing optimization meets Machine Intelligence: A perspective for the future network
Dai, B., Cao, Y., Wu, Z., Dai, Z., Yao, R., & Xu, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.06.093 · Cited by 215
Reinforcement learning for iot security: A comprehensive survey
Uprety, A., & Rawat, D. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3040957 · Cited by 214
“Researchers have proposed many DRL approaches with its application ranging from control [15], resource management [16], [17], robotics [18], [19] and many more.”
BackgroundWang, Y., Wang, K., Huang, H., Miyazaki, T., & Guo, S. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2883991 · Cited by 213
“Many existing studies focused on only a particular type of network or type of task [21], [22].”
BackgroundA Graph Neural Network-Based Digital Twin for Network Slicing Management
Wang, H., Wu, Y., Min, G., & Miao, W. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3047843 · Cited by 216
“Researchers are using Deep Neural Networks (DNN) to model complex network behaviours [9] and develop decision-making strategies based on Deep Reinforcement Learning [10].”
MethodLearning to Perform Local Rewriting for Combinatorial OptimizationPreprint
Chen, X., & Tian, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00337 · Cited by 254
“Following DeepRM [33], we use the average job slowdown ηj ≡ (Cj −Aj)/Tj ≥ 1 as our evaluation metric.”
BackgroundMethodInfluentialDeep multiagent reinforcement learning: challenges and directions
Wong, A., Bäck, T., Kononova, A. V., & Plaat, A. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10299-x · Cited by 248
“2015), resource management (Wen et al. 2015; Mao et al. 2016), indoor robot navigation (Zhu et al.”
BackgroundCFR-RL: Traffic Engineering With Reinforcement Learning in SDN
Zhang, J., Ye, M., Guo, Z., Yen, C.-Y., & Chao, H. J. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3000371 · Cited by 200
“ach state s t, CFR-RL would select K critical flows. Given that there are total N „N 1”flows in a network with N nodes, this RL problem would require a large action space of size CK N„ 1” . Inspired by [27], we define the action space as {0, 1, ..., „N „N 1”” 1} and allow the agent to sample K different actions in each t…”
BackgroundA deep reinforcement learning approach for chemical production scheduling
Hubbs, C. D., Li, C., Sahinidis, N. V., Grossmann, I. E., & Wassick, J. M. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.106982 · Cited by 211
Jamil, B., Ijaz, H., Shojafar, M., Munir, K., & Buyya, R. (2022) · ACM Computing Surveys · doi:10.1145/3513002 · Cited by 205
“In 2016, Mao et al. [124] are the first ones who claim that a system itself can learn to manage resources.”
BackgroundMachine learning (ML)-centric resource management in cloud computing: A review and future directions
Khan, T., Tian, W., Zhou, G., Ilager, S., Gong, M., & Buyya, R. (2022) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2022.103405 · Cited by 216
Zhao, Y., Li, Y., Zhang, X., Geng, G., Zhang, W., & Sun, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2928564 · Cited by 176
“DeepRM [52] was presented as a solution that translates the problem of packing tasks with multiple resource demands into a learning problem.”
BackgroundA Learning-based Iterative Method for Solving Vehicle Routing Problems
Hao, L., Zhang, X., & Yang, S. (2020) · International Conference on Learning Representations · Cited by 175
Lee, S.-S., & Lee, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2996213 · Cited by 174
“Mao et al. [28] employed the REINFORCE algorithm [36] for a resource allocation problem with a discrete action space.”
MethodGawłowicz, P., & Zubow, A. (2019) · ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM) · doi:10.1145/3345768.3355908 · Cited by 176
“…a boom in the usage of machine learning in general and reinforcement learning (RL) in particular for the optimization of communication and networking systems ranging from scheduling [2, 6], resource management [18], congestion control [14, 15, 29], routing [1] and adaptive video streaming [19].”
BackgroundDeep reinforcement learning based control for Autonomous Vehicles in CARLA
Pérez-Gil, Ó., Barea, R., López-Guillén, E., Bergasa, L. M., Gómez-Huélamo, C., Gutiérrez, R., & Diaz-Diaz, A. (2022) · Multimedia tools and applications · doi:10.1007/s11042-021-11437-3 · Cited by 169
“…through AV architecture, dealing with different processes such as detection, Multi-Object Tracking (MOT) and environment prediction, or evaluating the current situation of the ego-vehicle to conduct the safest decision, for example making use of DRL algorithms for behavioural driving [31].”
MethodDeep and reinforcement learning for automated task scheduling in large‐scale cloud computing systems
Rjoub, G., Bentahar, J., Wahab, O. A., & Bataineh, A. S. (2020) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.5919 · Cited by 156
Chen, Z., Hu, J. W., Min, G., Luo, C., & El-Ghazawi, T. A. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3132422 · Cited by 171
“Different from these work using value-based DRL algorithms, Mao et al. [20] leveraged a policy-based DRL algorithm (i.e., policy gradient (PG)) to handle the resource allocation problem in cloud datacenters.”
MethodDeep Learning-based Job Placement in Distributed Machine Learning Clusters
Bao, Y., Peng, Y., & Wu, C. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737460 · Cited by 153
“In this way, we use multiple inferences to effectively reduce action space [23] [22].”
MethodLiang, E., Zhu, H., Jin, X., & Stoica, I. G. (2019) · ACM Special Interest Group on Data Communication · doi:10.1145/3341302.3342221 · Cited by 154
“There are also many solutions that apply deep reinforcement learning to congestion control [6, 16, 64] and resource management [34].”
BackgroundInfluentialInternet of Things Connected Wireless Sensor Networks for Smart Cities
Ghazal, T. M., Hasan, M. K., Alzoubi, H. M., Alshurideh, M. T., Ahmad, M., & Akbar, S. S. (2023) · Studies in computational intelligence · doi:10.1007/978-3-031-12382-5_107 · Cited by 151
Liang, H., Wen, X., Liu, Y., Zhang, H., Li, Z., & Wang, L. (2020) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2020.101991 · Cited by 147
Reinforcement learning-assisted evolutionary algorithm: A survey and research opportunities
Song, Y., Wu, Y., Guo, Y., Yan, R., Suganthan, P. N., Zhang, Y., Witold Pedrycz, C., Das, S., Mallipeddi, R., Ajani, O. S., et al. (2024) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2024.101517 · Cited by 155
Peekaboo: Learning-Based Multipath Scheduling for Dynamic Heterogeneous Environments
Wu, H., Alay, Ö., Brunström, A., Ferlin, S., & Caso, G. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3000365 · Cited by 143
“Previous work has also considered using Deep RL (DRL) methods for decision-making problems, such as DeepRM [9] and Pensieve [10].”
Hu, X., Liu, S., Chen, R., Wang, W., & Wang, C. (2018) · IEEE Communications Letters · doi:10.1109/lcomm.2018.2844243 · Cited by 142
“DRL as an effective online method have achieved many success in game playing [5] and resource management [6].”
MethodDeep reinforcement learning for semiconductor production scheduling
Waschneck, B., Reichstaller, A., Belzner, L., Altenmüller, T., Bauernhansl, T., Knapp, A., & Kyek, A. (2018) · ASMC proceedings · doi:10.1109/asmc.2018.8373191 · Cited by 150
Jayanetti, A., Halgamuge, S. K., & Buyya, R. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.06.012 · Cited by 143
“Therefore, in each actual time-step, we use the DRL agent multiple times to determine the nodes in which all tasks in RTQ are to be scheduled [11].”
MethodEfficient Data Structures and Algorithms for Cloud Computing Platforms
Aladiyan, A. (2024) · International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) · doi:10.1109/icacite60783.2024.10617203 · Cited by 135
“[9] Bloom Filters Space-efficient data structures that tell you whether an element might be in a set.”
BackgroundQian, B., Su, J., Wen, Z., Jha, D. N., Li, Y., Guan, Y., Puthal, D., James, P., Yang, R., Zomaya, A. Y., et al. (2021) · ACM Computing Surveys · doi:10.1145/3398020 · Cited by 132
“Another task formulated in discrete action space can be found in Reference [182], where the action space is the “schedule of the job at i-th slot.”
BackgroundInfluentialZhan, Y., Li, P., & Guo, S. (2020) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps47924.2020.00033 · Cited by 136
“Towards this direction, DeepRM [46] leveraged DRL to solve the online multi-resource job placement problem and can achieve better performance as compared with the heuristic algorithms.”
MethodLeveraging Deep Reinforcement Learning for Traffic Engineering: A Survey
Xiao, Y., Liu, J., Wu, J., & Ansari, N. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3102580 · Cited by 146
Gazori, P., Rahbari, D., & Nickray, M. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.09.060 · Cited by 130
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Hutsebaut-Buysse, M., Mets, K., & Latré, S. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010009 · Cited by 129
“autonomous vehicles [27,28] smart grids [29], fleet management [30], resource management [31,32] and recommender systems [33].”
BackgroundLiu, S., Hu, X., & Wang, W. (2018) · IEEE Access · doi:10.1109/access.2018.2809581 · Cited by 128
“The emerging DRL method shows great prospect in sequential decision making problems [12]–[15] and has been applied in many dynamic resource management problems [16]–[20].”
MethodDeep Reinforcement Learning Based Optimization Algorithm for Permutation Flow-Shop Scheduling
Pan, Z., Wang, L., Wang, J., & Lu, J. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3098354 · Cited by 127
“…improvement heuristics for TSP and VRP. Xin et al. [39] presented a novel DRL method by using multi-decoder attention model with embedding glimpse to solve VRP. Mao et al. [40] built management systems with a designed DeepRM based on deep RL (DRL) to learn resource managementdirectlyfromexperience.”
MethodTran‐Dang, H., Bhardwaj, S., Rahim, T., Musaddiq, A., & Kim, D. (2022) · Journal of Communications and Networks · doi:10.23919/jcn.2021.000041 · Cited by 134
Ng, J. S., Lim, W. Y. B., Luong, N. C., Xiong, Z., Asheralieva, A., Niyato, D. T., Leung, C., & Miao, C. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3091684 · Cited by 143
“Moreover, for the future works, the dynamics of the system may be captured using Deep Reinforcement Learning based resource allocation [229].”
Islam, M. T., Karunasekera, S. A., & Buyya, R. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3124670 · Cited by 133
“There are lots of existing works [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19] that focused on various SLA objectives.”
BackgroundMethodAdaptable and Data-Driven Softwarized Networks: Review, Opportunities, and Challenges
Kellerer, W., Kalmbach, P., Blenk, A., Basta, A., Martin Reisslein, B., & Schmid, S. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2895553 · Cited by 123
“[102] H.”
Cloud Resource Scheduling With Deep Reinforcement Learning and Imitation Learning
Guo, W., Tian, W., Ye, Y., Xu, L., & Wu, K. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3025015 · Cited by 128
“Nevertheless, depending on different machine learning techniques used in the solution, research in this domain can be classified in different ways, including: 1) bioinspired solutions [28]; 2) Q -learning-based solutions [29]–[31]; and 3) deep RL-based solutions [17], [32].”
BackgroundMethodInfluentialDeep Reinforcement Scheduling for Mobile Crowdsensing in Fog Computing
Li, H., Ota, K., & Dong, M. (2019) · ACM Transactions on Internet Technology · doi:10.1145/3234463 · Cited by 120
“—DeepRM: a resource management method with deep reinforcement learning [31].”
BackgroundMethodSellami, B., Hakiri, A., Yahia, S. B., & Berthou, P. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.108957 · Cited by 120
“[30] introduced a DeepRM framework to build autonomous and intelligent systems that learn to manage resources from their own experience directly.”
BackgroundLiu, W., Cai, J., Chen, Q. C., & Wang, Y. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102865 · Cited by 126
AI-Enhanced Cloud-Edge-Terminal Collaborative Network: Survey, Applications, and Future Directions
Gu, H., Zhao, L., Han, Z., Zheng, G., & Song, S. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3338153 · Cited by 124
AI-Based Resource Provisioning of IoE Services in 6G: A Deep Reinforcement Learning Approach
Sami, H., Otrok, H., Bentahar, J., & Mourad, A. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3066625 · Cited by 114
“Aware of the above limitations and cooping with 1) the Deep Reinforcement Learning (DRL) advancements in the resource management field [21], and 2) the potential of having an AI fostered environment in 6G [22], exploring the use of DRL as a resource management solution is promising.”
BackgroundDDPG-Based Joint Time and Energy Management in Ambient Backscatter-Assisted Hybrid Underlay CRNs
Zheng, K., Jia, X., Chi, K., & Liu, X. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3221422 · Cited by 113
“Secondly, utilizing DRL can address complex tasks, and keeps graceful performance under varying conditions by continuing learning [26], [27].”
MethodThermal Prediction for Efficient Energy Management of Clouds Using Machine Learning
Ilager, S., Ramamohanarao, K., & Buyya, R. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3040800 · Cited by 111
Toward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning
You, X., Li, X., Xu, Y., Feng, H., Zhao, J., & Yan, H. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3012832 · Cited by 114
“Recent successful applications include cloud resource allocation [21], adaptive bitrate video streaming [22], and cellular scheduling [23].”
BackgroundMachine-Learning-Assisted Security and Privacy Provisioning for Edge Computing: A Survey
Singh, S., Sulthana, A. R., Shewale, T., Chamola, V., Benslimane, A., & Sikdar, B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3098051 · Cited by 113
“Reinforcement algorithms are capable of handling certain problems that require heuristic human intelligence and can take the place of human intelligence [39].”
BackgroundDL2: A Deep Learning-Driven Scheduler for Deep Learning Clusters
Peng, Y., Bao, Y., Chen, Y., Wu, C., Meng, C., & Lin, W. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3052895 · Cited by 120
“, experience replay, actor-critic) as [14], [16], and also uses the training data generated by an analytical performancemodel [12] in a simulation environment.”
BackgroundMethodInfluentialHossain, M. A., Noor, R. M., Yau, K. A., Azzuhri, S. R., Z’aba, M. R., & Ahmedy, I. (2020) · IEEE Access · doi:10.1109/access.2020.2989870 · Cited by 111
“Concerning the research question, ‘‘Can systems learn to manage resources on their own?’’, the answer can be found in [292], in which an experiment was conducted using deep RL and DNN.”
BackgroundLee, J., Lee, Y. C., & Kim, J. T. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.10.013 · Cited by 106
RLScheduler: An Automated HPC Batch Job Scheduler Using Reinforcement Learning
Zhang, D., Dai, D., He, Y., Bao, F. S., & Xie, B. (2020) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41405.2020.00035 · Cited by 116
“Examples include resource scheduling and task provisioning, such as DeepRM [34, 39], Decima [28], and RRL [40]; resource configuration tuning [41–43]; file system tuning [44] and performance prediction [45, 46].”
BackgroundKabir, H., Tham, M., & Chang, Y. C. (2023) · Digital Communications and Networks · doi:10.1016/j.dcan.2023.05.006 · Cited by 126
Yao, H., Mai, T., Xu, X., Zhang, P., Li, M., & Liu, Y. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2859480 · Cited by 101
“There is a large amount of literature on applying supervised learning to solve specific problems for a variety of network applications, such as resource allocations [13], Web services [14], [15], Internet telephony [16], and video streaming [17].”
BackgroundMulti-agent reinforcement learning for resources allocation optimization: a survey
Hady, M. A., Hu, S., Pratama, M., Cao, Z., & Kowalczyk, R. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11340-5 · Cited by 131
“In RAO, this allows the agent to dynamically allocate resources based on the current system state and its expected impact on future performance Mao et al. (2016).”
MLCAD: A Survey of Research in Machine Learning for CAD Keynote Paper
Rapp, M., Amrouch, H., Lin, Y., Yu, B., Pan, D. Z., Wolf, M., & Henkel, J. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2021.3124762 · Cited by 116
“Mao et al. [121] used policy-based RL to decide when to schedule jobs with the goal of maximizing the performance.”
BackgroundLearning and Generating Distributed Routing Protocols Using Graph-Based Deep Learning
Geyer, F., & Carle, G. (2018) · doi:10.1145/3229607.3229610 · Cited by 96
“In this context, machine learning has often been proposed and applied to various tasks such as routing [3, 25, 28], computing resource management [16] or packet scheduling [4].”
MethodReinforcement Learning for Slicing in a 5G Flexible RAN
Raza, M. R., Natalino, C., Öhlén, P., Wosinska, L., & Monti, P. (2019) · Journal of Lightwave Technology · doi:10.1109/jlt.2019.2924345 · Cited by 97
“The RL agent is modeled as a stochastic policy network (PN) [Mao16], which, in turn, uses an artificial neural network (ANN) to represent its stochastic policy.”
Intelligent VNF Orchestration and Flow Scheduling via Model-Assisted Deep Reinforcement Learning
Gu, L., Zeng, D., Li, W., Guo, S., Zomaya, A. Y., & Jin, H. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2959182 · Cited by 98
“…learning, including DRL, has been successfully applied as an alternative promising means to address many issues in computer networks, e.g., TCP congestion control [8], workload balancing [9], SDN routing [10], resource allocation [11], wireless signal calibration [12], and traffic optimization [13].”
MethodDynamic Jobshop Scheduling Algorithm Based on Deep Q Network
Zhao, Y., Wang, Y., Tan, Y., Zhang, J., & Yu, H. (2021) · IEEE Access · doi:10.1109/access.2021.3110242 · Cited by 97
“Mao et al. [29] proposed a multi-resource cluster scheduler called DeepRM for scheduling problems, and built a system that can learn scheduling strategies directly from previous scheduling experience.”
BackgroundYang, Z., Nguyen, P., Jin, H., & Nahrstedt, K. (2019) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00021 · Cited by 94
“problem of packing tasks with multiple resource demands into a deep reinforcement learning problem [22].”
BackgroundEnergy-Efficient Artificial Intelligence of Things With Intelligent Edge
Zhu, S., Ota, K., & Dong, M. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3143722 · Cited by 97
Metadata
- Authors
- Mao, H., Alizadeh, M., Menache, I., & Kandula, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 15th ACM Workshop on Hot Topics in Networks
- Publisher
- ACM
- Published
- 2016-11-09
- Pages
- 50-56
- OpenAlex
- W2546571074
- Cited by
- 1,353
- References
- 41
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 98 · 2023: 140 · 2022: 151 · 2021: 191 · 2020: 180 · 2019: 155 · 2018: 62 · 2017: 24
- FWCI
- 68.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Hot Topics in Networks (HotNets) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- resource management, deep reinforcement learning, task scheduling, resource allocation, workload modeling, online decision making, experiential learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
