The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Resource Management with Deep Reinforcement Learning
Abstract
Resource management problems in systems and networking often manifest as difficult online decision making tasks where appropriate solutions depend on understanding the workload and environment. Inspired by recent advances in deep reinforcement learning for AI problems, we consider building systems that learn to manage resources directly from experience. We present DeepRM, an example solution that translates the problem of packing tasks with multiple resource demands into a learning problem. Our initial results show that DeepRM performs comparably to state-of-the-art heuristics, adapts to different conditions, converges quickly, and learns strategies that are sensible in hindsight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resource management problems in practical settings.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [294]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. (2010) · Communications of the ACM · doi:10.1145/1721654.1721672 · Cited by 9,464
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Research works printed but not linked: 20
- Agarwal S., 2012, NSDI, 281
- Ananthanarayanan G., 2010, In OSDI, number, 1, 24
- Bertsekas D. P., 1995, Decision and Control
- Chilimbi T., 2014, OSDI, 571
- Dong M., 2015, NSDI, 395
- Ghodsi A., 2011, NSDI'11, 323
- Heller B., 2010, NSDI'10
- Junchen J., 2015, SIGCOMM '15
- Kaelbling L. P., 1996, Journal of artificial intelligence research
- Mahadevan S., 1998, FLAIRS Conference
- Mnih V., 2016, CoRR
- Mnih V., 2013, CoRR
- Mnih V., 2015, Nature
- J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. · arXiv
- 10.5555/551283 · DOI
- Sutton R. S., 1999, NIPS
- Winstein K., 2013, NSDI, 459
- Yi S., 2016, SIGCOMM
- Yin X., 2016, SIGCOMM '16
- Zhang W., 1995, IJCAI. Citeseer
Cited by
Sorted by citations
Items 101–200 of 1,000
A deep learning approach for optimizing content delivering in cache-enabled HetNet
Lei, L., You, L., Dai, G., Vu, T. X., Yuan, D., & Chatzinotas, S. (2017) · International Symposium on Wireless Communication Systems · doi:10.1109/iswcs.2017.8108157 · Cited by 91
“In [7], the authors applied a reinforcement learning approach for solving a classical job scheduling problem.”
MethodAL-Tam, F., Correia, N., & Rodrı́guez, J. (2020) · IEEE Access · doi:10.1109/access.2020.3000893 · Cited by 93
“A general resource management problem is handled in [36] by a policy gradient DRL agent.”
BackgroundQi, Q., Zhang, L., Wang, J., Sun, H., Zhuang, Z., Liao, J., & Yu, F. R. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3029864 · Cited by 91
“The basic value of input data for each child task are [51, 37, 61, 86, 15, 71, 37, 61, 116, 35]Mbits.”
BackgroundLearning to Coordinate Video Codec with Transport Protocol for Mobile Video Telephony
Zhou, A., Zhang, H., Su, G., Wu, L., Ma, R., Meng, Z., Zhang, X., Xie, X., Ma, H., & Chen, X. (2019) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3300061.3345430 · Cited by 90
“Examples include Remy [56], PCC [18, 19] Custard [27], QTCP [35], RL-TCP [32] for congestion control, DeepRM [37] for resource scheduling or routing, and also Pensieve [38] for VoD.”
BackgroundSun, G., Boateng, G. O., Ayepah-Mensah, D., Liu, G., & Jiang, W. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.2982857 · Cited by 90
“Reinforcement learning has shown impressive improvement in many applications such as job scheduling with multiple resource demands in the computing clusters [23].”
BackgroundPeng, Z., Lin, J., Cui, D., Li, Q., & He, J. (2020) · Cluster Computing · doi:10.1007/s10586-019-03042-9 · Cited by 88
“For example, Mao et al. [34] first proposed a cloud resource management model based on DRL, which transformed the multi-resource job scheduling problem into a task-boxing problem.”
MethodJazayeri, F., Shahidinejad, A., & Ghobaei‐Arani, M. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02561-3 · Cited by 88
“In a study (Mao et al. 2016), for the first time, the DRL was used for job scheduling in the Cloud.”
MethodTrust-driven reinforcement selection strategy for federated learning on IoT devices
Rjoub, G., Wahab, O. A., Bentahar, J., & Bataineh, A. S. (2022) · Computing · doi:10.1007/s00607-022-01078-1 · Cited by 88
“Reinforcement learning [22, 27] is an active research and application area of machine learning that has been applied to solve uncertainty-driven problems wherein exact models are often infeasible.”
Enabling technologies for AI empowered 6G massive radio access networks
Shahjalal, M. A., Kim, W., Khalid, W., Moon, S., Khan, M. A., Liu, S., Lim, S., Kim, E., Yun, D.-W., Lee, J., et al. (2022) · ICT Express · doi:10.1016/j.icte.2022.07.002 · Cited by 92
Explainable reinforcement learning in production control of job shop manufacturing system
Kuhnle, A., May, M. C., Schäfer, L., & Lanza, G. (2021) · International Journal of Production Research · doi:10.1080/00207543.2021.1972179 · Cited by 92
“Researchers demonstrate how RL can be used to optimally allocate computer resources to waiting tasks (Mao et al. 2016).”
BackgroundIntelligent Edge-Assisted Crowdcast with Deep Reinforcement Learning for Personalized QoE
Wang, F., Zhang, C., wang, F., Liu, J., Zhu, Y., Pang, H., & Sun, L. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737456 · Cited by 87
“effective use of the advanced deep reinforcement learning (DRL) [4]–[8] to tackle the key challenge therein.”
BackgroundMethodImproving TCP Congestion Control with Machine Intelligence
Kong, Y., Zang, H., & Ma, X. (2018) · ACM SIGCOMM Workshop on Network Meets AI & ML (NetAI) · doi:10.1145/3229543.3229550 · Cited by 83
Li, F., & Hu, B. (2019) · International Conference on Big Data and Computing (ICBDC) · doi:10.1145/3335484.3335513 · Cited by 87
“Instead of directly using deep reinforcement learning model for job scheduling[4][11], DeepJS is embedded in the framework of the bin packing problem.”
BackgroundMethodInfluentialLiu, Y., Lu, H., Li, X., Zhang, Y., Xi, L., & Zhao, D. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3038793 · Cited by 83
“Recently, with great success in DRL, DRL has been recognized as a promising method to deal with the workload balancing [44], network resource management [45], routing [46], etc.”
MethodDeepViNE: Virtual Network Embedding with Deep Reinforcement Learning
Dolati, M., Hassanpour, S. B., Ghaderi, M., & Khonsari, A. (2019) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infcomw.2019.8845171 · Cited by 82
“However, since the number of ways a VN can be embedded in the physical network is combinatorial, the DRL approach potentially requires a huge number of actions to derive an appropriate solution, which could lead to prohibitively long convergence time for the learning process [17].”
BackgroundAn End-to-End Load Balancer Based on Deep Learning for Vehicular Network Traffic Control
Li, J., Luo, G., Cheng, N., Yuan, Q., Wu, Z., Gao, S., & Liu, Z. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2866435 · Cited by 80
“[34] adopted deep reinforcement learning (DRL) for the task of packing tasks with multiple resource demands, which is formu-”
Sellami, B., Hakiri, A., & Yahia, S. B. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.07.024 · Cited by 85
Thein, T., Myo, M. M., Parvin, S., & Gawanmeh, A. (2018) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2018.11.005 · Cited by 79
Task allocation algorithm and optimization model on edge collaboration
Deng, X., Li, J., Liu, E., & Zhang, H. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2020.101778 · Cited by 78
Li, Y., Zhang, X., Zeng, T., Duan, J., Wu, C., Wu, D., & Chen, X. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3313779 · Cited by 81
“Similar approaches are adopted in [16], [24] for the single agent DRL.”
Spotlight: Optimizing Device Placement for Training Deep Neural Networks
Gao, Y., Chen, L., & Li, B. (2018) · International Conference on Machine Learning · Cited by 75
Improving Cognitive Ability of Edge Intelligent IIoT through Machine Learning
Chen, B., Wan, J., Lan, Y., Imran, M. A., Li, D., & Guizani, N. (2019) · IEEE Network · doi:10.1109/mnet.001.1800505 · Cited by 77
“In [11], Mao et al.”
BackgroundInfluentialRevising the Observation Satellite Scheduling Problem Based on Deep Reinforcement Learning
Huang, Y., Mu, Z., Wu, S., Cui, B., & Duan, Y. (2021) · Remote Sensing · doi:10.3390/rs13122377 · Cited by 75
“Besides the TSP and VRP, another type of optimization problem, resource allocation [25], also has been solved by RL. Khadilkar et al. [31] adopted RL to schedule time resourced for a railway system, and it was found that the Q-learning algorithm is superior to heuristic approaches in effectiveness.”
Task Scheduling in Cloud Using Deep Reinforcement Learning
Swarup, S., Shakshuki, E., & Yasar, A.-U.-H. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.03.016 · Cited by 74
Spear: Optimized Dependency-Aware Task Scheduling with Deep Reinforcement Learning
Hu, Z., Tu, J., & Li, B. (2019) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00201 · Cited by 73
“Reinforcement learning was adopted in DeepRM [27] to minimize the average job slow down for a group of independent jobs where dependencies do not exist.”
MethodZhao, J., Kong, M., Li, Q., & Sun, X. (2019) · IEEE Access · doi:10.1109/access.2019.2963051 · Cited by 73
“In [24], the method of DRL is used to meet different resource requirements.”
MethodBlack or White? How to Develop an AutoTuner for Memory-based Analytics
Kunjir, M., & Babu, S. (2020) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3318464.3380591 · Cited by 73
“Most of the literature has focussed on training ML-based performance models using either a small-scale benchmark test bed, historical performance data, or from application performance under low workload [3, 19, 36, 60– 62, 69].”
BackgroundA reinforcement learning based approach for multi-projects scheduling in cloud manufacturing
Chen, S., Fang, S., & Tang, R. (2018) · International Journal of Production Research · doi:10.1080/00207543.2018.1535205 · Cited by 73
“Mao et al. (2016) applied Deep-RL techniques to solve the multiple services cluster scheduling problem.”
MethodFlexibility in Softwarized Networks: Classifications and Research Challenges
He, M., Alba, A. M., Basta, A., Blenk, A., & Kellerer, W. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2892806 · Cited by 72
“Nonetheless, the goals of exploiting this aspect are different: maximize traffic delivered ([91], [167]), minimize link utilization ([92], [99]), balance the load ([93], [106]–[108]), dynamically adapt network policies ([94], [101], [104]–[106]),”
BackgroundDeep reinforcement learning‐based beam Hopping algorithm in multibeam satellite systems
Hu, X., Liu, S., Wang, Y., Xu, L., Zhang, Y., Wang, C., & Wang, W. (2019) · IET Communications · doi:10.1049/iet-com.2018.5774 · Cited by 71
“Here, the state reformulation concept is used in designing the state [25].”
MethodToward ML-centric cloud platforms
Bianchini, R., da Fontoura, M. F. M. C., Cortez, E., Bonde, A., Muzio, A. F. V., Constantin, A.-M., Moscibroda, T., Magalhães, G., Bablani, G., & Russinovich, M. (2020) · Communications of the ACM · doi:10.1145/3364684 · Cited by 71
“20,26 For the same example, each server can learn different thresholds for its own resource management.”
BackgroundHou, H., Jawaddi, S. N. A., & Ismail, A. B. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.10.002 · Cited by 77
Deep Reinforcement Learning: A Chronological Overview and Methods
Terven, J. R. (2025) · AI · doi:10.3390/ai6030046 · Cited by 98
“In resource management scenarios, RL is used in distributed systems and cloud infrastructures [24–26].”
Applications of Machine Learning in Networking: A Survey of Current Issues and Future Challenges
Ridwan, M. A., Radzi, N. A. M., Abdullah, F., & Jalil, Y. E. (2021) · IEEE Access · doi:10.1109/access.2021.3069210 · Cited by 76
“The combination of RL and DL has recently been proven as a promising alternative solution to various resource management problems in practical settings [126].”
BackgroundLiu, Y., Liang, H., Xiao, Y., Zhang, H., Zhang, J., Li, Z., & Wang, L. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102323 · Cited by 71
Koo, J., Mendiratta, V., Rahman, M. R., & Walid, A. (2019) · International Conference on Network and Service Management (CNSM) · doi:10.23919/cnsm46954.2019.9012702 · Cited by 67
A Computational Approach to Packet Classification
Rashelbach, A., Rottenstreich, O., & Silberstein, M. (2020) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3387514.3405886 · Cited by 67
Park: An Open Platform for Learning-Augmented Computer Systems
Mao, H., Negi, P., Narayan, A., Wang, H., Yang, J., Wang, H., Marcus, R., Addanki, R., Shirkoohi, M. K., He, S., et al. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 67
Self-citationA Survey on Slice Admission Control Strategies and Optimization Schemes in 5G Network
Ojijo, M. O., & Falowo, O. E. (2020) · IEEE Access · doi:10.1109/access.2020.2967626 · Cited by 66
Zhao, B., Liu, J., Wei, Z., & You, I. (2020) · IEEE Access · doi:10.1109/access.2020.2983437 · Cited by 66
A Survey on Scheduling Techniques in Computing and Network Convergence
Tang, S., Yu, Y., Wang, H., Wang, G., Chen, W., Xu, Z., Guo, S., & Gao, W. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3329027 · Cited by 77
“Furthermore, it tries to solve the reward interference of the job arrival sequence using sequences separately trained [178].”
InfluentialTowards Lightweight and Robust Machine Learning for CDN Caching
Berger, D. S. (2018) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3286062.3286082 · Cited by 65
“Finally, there is an emerging literature on successful machine learning applications in networking research such as traffic classification [52], video rate adaptation [48], resource allocation [36], and congestion control [47].”
BackgroundResource Allocation based on Graph Neural Networks in Vehicular Communications
He, Z., Wang, L., Ye, H., Li, G. Y., & Juang, B.-H. F. (2020) · Globecom · doi:10.1109/globecom42002.2020.9322537 · Cited by 65
“The work in [10] shows that the deep RL framework can address the resource management problems and is comparable and some-times better than heuristic based approaches for a multi-resource cluster scheduling problem.”
Ma, X., Li, Q., Zou, L., Peng, J., Zhou, J., Chai, J., Jiang, Y., & Muntean, G. (2022) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2022.3171131 · Cited by 65
“Mao et al. [38] uses DRL to design a multi-resource management scheduler to minimize average job slowdown or completion time.”
MethodSalman, S., Streiffer, C., Chen, H., Benson, T., & Kadav, A. (2018) · ACM SIGCOMM Workshop on Network Meets AI & ML (NetAI) · doi:10.1145/3229543.3229554 · Cited by 66
“, cluster scheduling [23] or routing [9], in this paper, we present a general framework, called DeepConf, that simplifies the process of designing ML models for a broad range of DCN topol-”
BackgroundMethodEdgeSlice: Slicing Wireless Edge Computing Network with Decentralized Deep Reinforcement Learning
Liu, Q., Han, T., & Moges, E. (2020) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00028 · Cited by 63
“Second, the existing DRL models are unable to adjust their policies based on coordinating information from an external control [20].”
BackgroundMethodDeep Reinforcement Agent for Scheduling in HPC
Fan, Y., Lan, Z., Childers, J. T., Rich, P., Allcock, W. E., & Papka, M. E. (2021) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps49936.2021.00090 · Cited by 73
DDQP: A Double Deep Q-Learning Approach to Online Fault-Tolerant SFC Placement
Wang, L., Mao, W., Zhao, J., & Xu, Y. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3049298 · Cited by 65
“…learning (DRL) shows greater magic skills: it beats the top human players in video games [14], [15], controls complex machinery for operation [16], helps to allocate network resources [17], saves energy greatly for the data center, and even automatic adjusts machine learning algorithms [18].”
BackgroundDQRA: Deep Quantum Routing Agent for Entanglement Routing in Quantum Networks
Le, L., & Nguyen, T. N. (2022) · IEEE Transactions on Quantum Engineering · doi:10.1109/tqe.2022.3148667 · Cited by 72
Energy-Efficient Ultra-Dense Network With Deep Reinforcement Learning
Ju, H., Kim, S., Kim, Y., & Shim, B. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3150425 · Cited by 61
“In recent years, we have witnessed great success of DRL in various applications such as Go game [13], natural language processing [14], and resource management in computer systems and networks [15].”
A Particle Filter-Based Reinforcement Learning Approach for Reliable Wireless Indoor Positioning
Villacrés, J. L. C., Zhao, Z., Braun, T. I., & Li, Z. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933886 · Cited by 61
“In [28], authors applied RL to”
Verifying Deep-RL-Driven Systems
Kazak, Y., Barrett, C., Katz, G., & Schapira, M. (2019) · ACM SIGCOMM Workshop on Network Meets AI & ML (NetAI) · doi:10.1145/3341216.3342218 · Cited by 62
“We evaluate Verily on three deep-RL-driven systems: the Pensieve adaptive video streaming scheme [24], the DeepRM scheduler for cloud resource management [23], and the Custard Internet con-”
BackgroundMethodInfluentialMulti-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning
Wei, W.-T., Gu, H., Wang, K., Li, J., Zhang, X., & Wang, N. (2023) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3213575 · Cited by 56
Nikita, S., Tiwari, A., Sonawat, D., Kodamana, H., & Rathore, A. S. (2020) · Chemical Engineering Science · doi:10.1016/j.ces.2020.116171 · Cited by 60
FaaSRank: Learning to Schedule Functions in Serverless Platforms
Yu, H., Irissappane, A. A., Wang, H., & Lloyd, W. (2021) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS) · doi:10.1109/acsos52086.2021.00023 · Cited by 62
“DeepRM is an RL-based scheduler in a simulation system [15], which achieves scheduling fairness by assuming jobs execute identically regardless of the state of resource contention in the system.”
BackgroundMethodDeep Reinforcement Learning Techniques in Diversified Domains: A Survey
Gupta, S., Singal, G., & Garg, D. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-021-09552-3 · Cited by 58
“The recent advancements [131, 136, 178] that amalgamate the RL with DL scale their applicability to the domain in which features are not handcrafted, and state or action space is large or continuous [136].”
BackgroundCluster resource scheduling in cloud computing: literature review and research challenges
Khallouli, W., & Huang, J. (2021) · The Journal of Supercomputing · doi:10.1007/s11227-021-04138-z · Cited by 58
“Mao et al. [49] was the first work to explore the feasibility of using DRL structure for multi-resource task scheduling in distributed computing systems.”
BackgroundMAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores
Kao, S.-C., & Krishna, T. (2022) · 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00065 · Cited by 58
“Some works leverage RL for jobs ordering over clusters such as DeepRM [59], Decima [60] and Thamsen et al.”
Liu, J., Wu, Z., Feng, D., Zhang, M., Wu, X., Yao, X., Yu, D., Ma, Y., Zhao, F., & Dou, D. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.05.032 · Cited by 63
Efficient Compute-Intensive Job Allocation in Data Centers via Deep Reinforcement Learning
Yi, D., Zhou, X., Wen, Y., & Tan, R. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2968427 · Cited by 57
“Another recent study [28] applies online DRL to allocate computing resources to arriving jobs such that the job latency is reduced.”
BackgroundDECCO: Deep-Learning Enabled Coverage and Capacity Optimization for Massive MIMO Systems
Yang, Y., Li, Y., Li, K., Zhao, S., Chen, R., Wang, J., & Ci, S. (2018) · IEEE Access · doi:10.1109/access.2018.2828859 · Cited by 56
“Figure 4 shows a general setting of reinforcement learning where an agent interacts with an environment [14].”
BackgroundDeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing
Liu, Q., Han, T., Zhang, N., & Wang, Y. (2020) · Globecom · doi:10.1109/globecom42002.2020.9322106 · Cited by 56
“imizing the resource allocation in network slicing. Exploiting deep learning and deep reinforcement learning (DRL) for resource management in mobile networks has gained increasing research attentions [7], [8], [9]. These works formulate the network utility maximization problem as a reinforcement learning problem and a…”
MethodWang, Y., Arya, K., Kogias, M., Vanga, M., Bhandari, A., Yadwadkar, N. J., Sen, S., Elnikety, S., Kozyrakis, C., & Bianchini, R. (2021) · European Conference on Computer Systems · doi:10.1145/3447786.3456225 · Cited by 55
Adaptive DRL-Based Task Scheduling for Energy-Efficient Cloud Computing
Kang, K., Ding, D., Xie, H., Yin, Q., & Zeng, J. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3137926 · Cited by 55
“…presented in this paper consists of M heterogeneous machines (servers) and each machine (server), denoted as a vector m i = { m i , 1 , m i , 2 , . . . , m i , d } , offers d types of resources (e.g., CPU, run-time memory, disk storage) which are corresponding to task resource requirements [25].”
BackgroundKosanoglu, F., Atmis, M., & Turan, H. H. (2022) · Annals of Operations Research · doi:10.1007/s10479-022-04612-8 · Cited by 55
“Some recent works use DRL to solve some of the most prominent complex optimization problems such as resource management problems (Mao et al. 2016), job scheduling (Chen et al. 2017; Liu et al. 2020; Liang et al. 2020), VRP (Nazari et al. 2018; Lin et al. 2020; Zhao et al. 2020; Yu et al. 2019), and…”
Scaling Geo-Distributed Network Function Chains: A Prediction and Learning Framework
Luo, Z., Wu, C., Li, Z., & Zhou, W. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2927068 · Cited by 54
“Different from existing DL-based schedulers [15], [16], our scheduler does not rely on pure offline training.”
BackgroundMethodFraternali, F., Balaji, B., Agarwal, Y., & Gupta, R. K. (2020) · ACM Transactions on Sensor Networks · doi:10.1145/3404191 · Cited by 55
“[40] show that RL is comparable or better than ad hoc heuristics.”
RLSK: A Job Scheduler for Federated Kubernetes Clusters based on Reinforcement Learning
Huang, J., Xiao, C., & Wu, W. (2020) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e48712.2020.00019 · Cited by 55
“W. Chen [20] improves DeepRM [8] and realizes the management of resources in a multi-cluster and multi-resource environment, but with the same state representation with DeepRM [8].”
BackgroundMethodResultInfluentialNaeem, M., Pietro, G. D., & Coronato, A. (2021) · Sensors · doi:10.3390/s22010309 · Cited by 56
Artificial Intelligence Application in Demand Response: Advantages, Issues, Status, and Challenges
Ali, A. N. F., Sulaima, M. F., Razak, I. A. W. A., Kadir, A. F. A., & Mokhlis, H. (2023) · IEEE Access · doi:10.1109/access.2023.3237737 · Cited by 57
“The RL framework has been utilized in a variety of fields, with robotics [74], resource management in computer clusters [75], playing video games via pixel input [76], and automated ML frameworks [77] being among the most significant.”
MethodCao, Y., Zhao, Y., Li, J., Lin, R., Zhang, J., & Chen, J. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.2964003 · Cited by 55
“NN is trained by using a variant of the REINFORCE algorithm introduced in [58].”
BackgroundMethodInfluentialXu, Y., Zhu, K., Xu, H., & Ji, J. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3240425 · Cited by 54
“vehicular networks [19], [20], [21], [22], [23].”
Cai, H., Bian, Y., & Liu, L. (2023) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2023.102628 · Cited by 67
Empowering Self-Driving Networks
Kalmbach, P., Zerwas, J., Babarczi, P., Blenk, A., Kellerer, W., & Schmid, S. (2018) · doi:10.1145/3229584.3229587 · Cited by 52
“Especially the last point sets empowerment apart from traditional Reinforcement Learning (RL) approaches [16, 17] and mathematical optimization [2, 14, 21].”
BackgroundToward Edge-Assisted Video Content Intelligent Caching With Long Short-Term Memory Learning
Zhang, C., Pang, H., Liu, J., Tang, S., Zhang, R.-X., Wang, D., & Sun, L. (2019) · IEEE Access · doi:10.1109/access.2019.2947067 · Cited by 56
“Existing works already showed the superior of deep learning-based design in term of resource management [24], adaptive quality selection [25], viewers’ behavior prediction [26], etc.”
BackgroundPer-Host DDoS Mitigation by Direct-Control Reinforcement Learning
Simpson, K. A., Rogers, S., & Pezaros, D. P. (2019) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2019.2960202 · Cited by 52
“data-driven networking, such as for intra-AS route optimisation [46] and resource-constrained process allocation [47].”
A UAV-based moving 5G RAN for massive connectivity of mobile users and IoT devices
Νομικός, Ν., Michailidis, E. T., Trakadas, P. T., Vouyioukas, D., Karl, H., Martrat, J., Zahariadis, T. B., Papadopoulos, K. A., & Voliotis, S. (2020) · Vehicular Communications · doi:10.1016/j.vehcom.2020.100250 · Cited by 52
Liu, Z., Wang, Y., Liang, X., Ma, Y., Feng, Y., Cheng, G., & Liu, Z. (2022) · Information Sciences · doi:10.1016/j.ins.2022.06.017 · Cited by 50
Tshakwanda, P. M., Arzo, S. T., & Devetsikiotis, M. (2024) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2024.3398540 · Cited by 60
“With RL, we integrate current network data with future predictions to develop comprehensive routing strategies, enhancing performance and reducing latency [10].”
MethodRepresentation and Reinforcement Learning for Task Scheduling in Edge Computing
Tang, Z., Weijia Jia, B., Zhou, X., Yang, W., & You, Y. (2020) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2020.2990558 · Cited by 53
“Compared with traditional optimization algorithms, DRL-based algorithms can get a more efficient scheduling strategy [11], [12].”
MethodAdmission Control for 5G Core Network Slicing Based on Deep Reinforcement Learning
Jácome, W. F. V., Rendón, O. M. C., & da Fonseca, N. L. S. (2022) · IEEE Systems Journal · doi:10.1109/jsyst.2022.3172658 · Cited by 49
“Moreover, the Network Slicing process involves repetitive decisions and produces a large quantity of data useful to train RL and DRL algorithms [18], [24].”
MethodLearning Correlated Communication Topology in Multi-Agent Reinforcement learning
Du, Y., Liu, B., Moens, V., Liu, Z., Ren, Z., Wang, Y.-X., Chen, X., & Zhang, H. (2021) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/fxam2191 · Cited by 49
Caching in Dynamic IoT Networks by Deep Reinforcement Learning
Yao, J., & Ansari, N. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3004394 · Cited by 48
“Cao and ZhiMin [17] sorted out the development of DRL. Mao et al. [18] built a DeepRM system to manage resources and schedule jobs by utilizing DRL. Qi et al. [19] proposed a knowledge-driven service offloading decision framework for the Internet of Vehicles and designed a DRL approach to obtain…”
Metis: Learning to Schedule Long-Running Applications in Shared Container Clusters at Scale
Wang, L., Weng, Q., Wang, W., Chen, C., & Li, B. (2020) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41405.2020.00072 · Cited by 48
Scheduling of Time-Varying Workloads Using Reinforcement Learning
Mondal, S. S., Sheoran, N., & Mitra, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17088 · Cited by 47
“DeepRM (Mao et al. 2016) used DRL to schedule jobs that use constant amount of resources, on a single monolithic machine.”
MethodInfluentialDA-DRLS: Drift adaptive deep reinforcement learning based scheduling for IoT resource management
Chowdhury, A., Raut, S., & Narman, H. S. (2019) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.04.010 · Cited by 47
A Lightweight Collaborative Deep Neural Network for the Mobile Web in Edge Cloud
Huang, Y., Qiao, X., Ren, P., Liu, L., Pu, C., Dustdar, S., & Chen, J. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3043051 · Cited by 46
“To updates the parameters of deep neural networks for policy distribution, gradient-descent is a popular way [32], [34], which can be described as”
BackgroundInfluentialVerifying learning-augmented systems
Eliyahu, T., Kazak, Y., Katz, G., & Schapira, M. (2021) · Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication · doi:10.1145/3452296.3472936 · Cited by 99
“Examples include compilers [30], databases [42], compute resource scheduling [55], congestion control on the Internet [1, 35], video streaming [57], and many more.”
BackgroundMethodInfluentialDRAS: Deep Reinforcement Learning for Cluster Scheduling in High Performance Computing
Fan, Y., Li, B., Favorite, D., Singh, N., Childers, J. T., Rich, P., Allcock, W. E., Papka, M. E., & Lan, Z. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3205325 · Cited by 49
“Several recent studies have shown that RL driven scheduling is a promising approach for cluster scheduling [6], [16], [17], [18], [19], [20].”
BackgroundLazebnik, T. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106783 · Cited by 49
DAG-based workflows scheduling using Actor–Critic Deep Reinforcement Learning
Koslovski, G., Pereira, K., & Albuquerque, P. R. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.09.018 · Cited by 45
Deep Reinforcement Learning for IoT Network Dynamic Clustering in Edge Computing
Liu, Q., Cheng, L., Özçelebi, T., Murphy, J., & Lukkien, J. J. (2019) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid.2019.00077 · Cited by 45
“For example, [6] presents a solution that translates the tasks with multiple resource demands into a RL problem.”
BackgroundYi, D., Zhou, X., Wen, Y., & Tan, R. (2019) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00069 · Cited by 45
“Another recent study [26] applies online DRL to allocate computing resources to arriving jobs such that the job latency is reduced.”
BackgroundReview of Clustering Techniques in Control System
Singh, S., & Srivastava, S. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.06.032 · Cited by 45
Ghannad, P., Lee, Y.-C., & Choi, J. O. (2020) · Journal of Management in Engineering · doi:10.1061/(asce)me.1943-5479.0000868 · Cited by 45
Feng, X., Li, Y., & Xu, M. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102362 · Cited by 44
Chi, C., Ji, K., Song, P., Marahatta, A., Zhang, S., Zhang, F., Qiu, D., & Liu, Z. (2021) · Energies · doi:10.3390/en14082071 · Cited by 44
“In [20], a DRL-based task scheduling was proposed aiming at reducing the average job slowdown for data centers, which transformed the problem of packing tasks with multiple resource demands into a learning problem.”
MethodFedPA: An adaptively partial model aggregation strategy in Federated Learning
Liu, J., Wang, J. H., Rong, C., Xu, Y., Yu, T., & Wang, J. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.108468 · Cited by 44
“The recent success of RL [36, 37, 38] shows that RL agent can learn from the interaction with a complex and dynamic environment.”
BackgroundEfficient Provision of Service Function Chains in Overlay Networks Using Reinforcement Learning
Li, G., Zhou, H., Feng, B., Zhang, Y., & Yu, S. (2019) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2019.2961093 · Cited by 43
On deep reinforcement learning security for Industrial Internet of Things
Liu, X., Yu, W., Liang, F., GRIFFITH, D. W. T., & Golmie, N. (2021) · Computer Communications · doi:10.1016/j.comcom.2020.12.013 · Cited by 43
Metadata
- Authors
- Mao, H., Alizadeh, M., Menache, I., & Kandula, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 15th ACM Workshop on Hot Topics in Networks
- Publisher
- ACM
- Published
- 2016-11-09
- Pages
- 50-56
- OpenAlex
- W2546571074
- Cited by
- 1,353
- References
- 41
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 98 · 2023: 140 · 2022: 151 · 2021: 191 · 2020: 180 · 2019: 155 · 2018: 62 · 2017: 24
- FWCI
- 68.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Hot Topics in Networks (HotNets) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- resource management, deep reinforcement learning, task scheduling, resource allocation, workload modeling, online decision making, experiential learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
