The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Resource Management with Deep Reinforcement Learning
Abstract
Resource management problems in systems and networking often manifest as difficult online decision making tasks where appropriate solutions depend on understanding the workload and environment. Inspired by recent advances in deep reinforcement learning for AI problems, we consider building systems that learn to manage resources directly from experience. We present DeepRM, an example solution that translates the problem of packing tasks with multiple resource demands into a learning problem. Our initial results show that DeepRM performs comparably to state-of-the-art heuristics, adapts to different conditions, converges quickly, and learns strategies that are sensible in hindsight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resource management problems in practical settings.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [294]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. (2010) · Communications of the ACM · doi:10.1145/1721654.1721672 · Cited by 9,464
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Research works printed but not linked: 20
- Agarwal S., 2012, NSDI, 281
- Ananthanarayanan G., 2010, In OSDI, number, 1, 24
- Bertsekas D. P., 1995, Decision and Control
- Chilimbi T., 2014, OSDI, 571
- Dong M., 2015, NSDI, 395
- Ghodsi A., 2011, NSDI'11, 323
- Heller B., 2010, NSDI'10
- Junchen J., 2015, SIGCOMM '15
- Kaelbling L. P., 1996, Journal of artificial intelligence research
- Mahadevan S., 1998, FLAIRS Conference
- Mnih V., 2016, CoRR
- Mnih V., 2013, CoRR
- Mnih V., 2015, Nature
- J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. · arXiv
- 10.5555/551283 · DOI
- Sutton R. S., 1999, NIPS
- Winstein K., 2013, NSDI, 459
- Yi S., 2016, SIGCOMM
- Yin X., 2016, SIGCOMM '16
- Zhang W., 1995, IJCAI. Citeseer
Cited by
Sorted by citations
Items 401–500 of 1,000
A Deep Reinforcement Learning-based Task Scheduling Algorithm for Energy Efficiency in Data Centers
Song, P., Chi, C., Ji, K., Liu, Z., Zhang, F., Zhang, S., Qiu, D., & Wan, X. (2021) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn52240.2021.9522309 · Cited by 17
Chen, Z. (2021) · Software Practice and Experience · doi:10.1002/spe.3066 · Cited by 17
“Prior work 21,22 pays more attention to introducing deep reinforcement learning (Deep RL) into resource management of large-scale and complex systems.”
BackgroundLearning A Continuous and Reconstructible Latent Space for Hardware Accelerator Design
Huang, Q., Hong, C., Wawrzynek, J., Subedar, M. M., & Shao, Y. S. (2022) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass55109.2022.00041 · Cited by 17
“…of machine learning (ML) algorithms in performing difficult tasks such as image recognition [12]–[14], natural language processing [15], [16], and system optimization [17], [18], ML-based algorithms have also been used to search for optimal solutions in the discrete hardware design space [4]– [11].”
MethodCSDLEEG: Identifying Confused Students Based on EEG Using Multi-View Deep Learning
Abu-gellban, H., Zhuang, Y., Nguyen, L., Zhang, Z., & Imhmed, E. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00192 · Cited by 17
Cornalba, F., Disselkamp, C., Scassola, D., & Helf, C. (2022) · Neural computing & applications (Print) · doi:10.1007/s00521-023-09033-7 · Cited by 14
“RL techniques have initially been deployed – among many – in the field of gaming [31], robotics [25], personalized recommendations [48], and resource management [30].”
MethodLiu, J., Wang, Q., Zhang, S., Hu, L., & Silva, D. D. (2023) · International Middleware Conference · doi:10.1145/3590140.3592851 · Cited by 17
“These techniques can be further categorized into four groups: (a) threshold-based (or rule-based) [31, 42, 44], (b) statistical profilingbased [17, 54], (c) analytical model-based [8, 13, 40, 76], and (d) machine learning-based [23, 39, 51, 66, 71].”
BackgroundJin, J., Xing, S., Ji, E., & Liu, W. (2025) · Sensors · doi:10.3390/s25072183 · Cited by 19
“DeepRM [5] employs deep RL for resource management in datacenters, while Pensieve [24] uses RL to optimize adaptive video streaming.”
Machine Learning Aided Orchestration in Multi-tenant Networks
Natalino, C., Raza, M. R., Rostami, A., Öhlén, P., Wosinska, L., & Monti, P. (2018) · IEEE/LEOS Summer Topical Meetings · doi:10.1109/phosst.2018.8456735 · Cited by 17
“Results have been obtained by using a modified version of the simulator presented in [2].”
BackgroundMethodInfluentialReal-Time Decision Making for a Car Manufacturing Process Using Deep Reinforcement Learning
Gros, T. P., Groß, J., & Wolf, V. (2020) · Winter Simulation Conference · doi:10.1109/wsc48552.2020.9383884 · Cited by 19
“Deep reinforcement learning has also been applied successfully to scheduling problems, such as resource management (Mao et al. 2016; Chen et al. 2017) or global production scheduling (Waschneck et al. 2018).”
BackgroundAutonomous Cache Resource Slicing and Content Placement at Virtualized Mobile Edge Network
Sun, G., Al-Ward, H., Boateng, G. O., & Liu, G. (2019) · IEEE Access · doi:10.1109/access.2019.2923021 · Cited by 17
“The deep Q-network (DQN) algorithm is chosen in our solution for the following reasons: 1) it is a model-free algorithm, which allows us to model such a complex problem that would otherwise be very challenging to build with handcrafted features, 2) DQN can be adaptive to varying conditions as long as it is allowed to…”
MethodReinforcement Learning and Adaptive Sampling for Optimized DNN CompilationPreprint
Ahn, B. H., Pilligundla, P., & Esmaeilzadeh, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12799 · Cited by 25
“There are a growing body of studies on using reinforcement learning to perform various optimizations (Gao et al., 2018; Mirhoseini et al., 2017; Mao et al., 2016; Ye & Li, 2018; Henderson et al., 2017; Dong et al., 2018; Xu et al., 2018; Jaderberg et al., 2017) for a variety of objectives including…”
BackgroundPan, J., Wei, Y., Meng, L., & Meng, X. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00092-5 · Cited by 16
“Mao et al. (2016) designed DeepRM, a multi-resource cluster scheduler based on reinforcement learning, which optimizes resources and adjusts them across multiple objectives by assigning different reinforcement rewards to various resource types.”
Motor Load Balancing with Roll Force Prediction for a Cold-Rolling Setup with Neural Networks
Lee, S., & Son, Y. (2021) · Mathematics · doi:10.3390/math9121367 · Cited by 16
“Other applications include managing supply chains by predicting the demand of the participants in the supply chain, sharing information [20], and applying reinforcement learning to resource management problems, such as job scheduling or relay selection [21].”
BackgroundMethodReinforcement learning for energy-efficient control of parallel and identical machines
Loffredo, A., May, M. C., Schäfer, L., Matta, A., & Lanza, G. (2023) · CIRP journal of manufacturing science and technology · doi:10.1016/j.cirpj.2023.05.007 · Cited by 16
A Self-Supervised Learning Approach for Accelerating Wireless Network Optimization
Zhang, S., Ajayi, O. T., & Cheng, Y. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3244043 · Cited by 16
“These tend to focus on a purely data-driven end-to-end approach [5], [41], [42].”
BackgroundZou, Y., Cai, X., Zhu, M., Gu, J., Wang, Y., Guo, Z. Y., Shi, C., Zhang, J., Cai, Y., & Lei, M. (2023) · Journal of Lightwave Technology · doi:10.1109/jlt.2023.3299272 · Cited by 16
“To avoid this, we reduce the action space by allowing the agent to execute multiple actions within a single time slot, where each action step is referred to as ‘time-frozen step’ [26].”
BackgroundAn Efficient Reinforcement Learning Based Framework for Exploring Logic Synthesis
Qian, Y., Zhou, X., Zhou, H., & Wang, L. (2023) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3632174 · Cited by 18
“With the development of deep learning, reinforcement learning has shown its great power in solving the complex problems, such as games [ 37 ] and computer systems [ 20 ].”
BackgroundWu, W., Gao, Y., Zhou, T., Jia, Y., Zhang, H., Wei, T., & Sun, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2960191 · Cited by 16
“The update of the parameters of the neural network, 1θ , can be calculated as [27]”
BackgroundMethodKey-Size-Driven Wavelength Resource Sharing Scheme for QKD and the Time-Varying Data Services
Niu, J., Sun, Y., Jia, X., & Ji, Y. (2021) · Journal of Lightwave Technology · doi:10.1109/jlt.2021.3056109 · Cited by 16
“A similar training algorithm is also explained in detail in [59].”
BackgroundMethodKiangala, K. S., & Wang, Z. (2022) · Sensors · doi:10.3390/s22030941 · Cited by 16
“Over the years, RL has been the subject of many kinds of research in various applications such as chemical reaction [36], resource management [37], traffic-light management [38], autonomous driving [39], dam management [40], surgery [41] and robotics [42].”
BackgroundSuzuki, A., Kawahara, R., & Harada, S. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3149243 · Cited by 18
“Therefore, the RL approach potentially requires a huge number of actions to derive an appropriate solution, which could lead to a prohibitively long convergence time for the learning process [17].”
BackgroundBiases in Data-Driven Networking, and What to Do About Them
Bartulović, M., Jiang, J., Balakrishnan, S., Sekar, V., & Sinopoli, B. (2017) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3152434.3152448 · Cited by 16
“For instance, recent work [1, 3, 4, 15, 16, 29, 30] demonstrates the promise in several applications such as optimizing quality in video and VoIP as well as classical problems in routing and traffic engineering.”
BackgroundA Reinforcement Learning Based Network Scheduler for Deadline-Driven Data Transfers
Ghosal, D., Shukla, S. K., Sim, A., Thakur, A., & Wu, K. J. (2019) · Globecom · doi:10.1109/globecom38437.2019.9013255 · Cited by 17
“As mentioned in [16] decisions made by these systems are often highly repetitive they provide the required training data for RL algorithms.”
BackgroundDynamic Channel Allocation for Satellite Internet of Things via Deep Reinforcement Learning
Liu, J., Zhao, B., Xin, Q., & Liu, H. (2020) · International Conference on Information Networking · doi:10.1109/icoin48656.2020.9016474 · Cited by 18
DeepSoCS: A Neural Scheduler for Heterogeneous System-on-Chip (SoC) Resource Scheduling
Sung, T. T., Ha, J., Kim, J., Yahja, A., Sohn, C.-B., & Ryu, B. (2020) · Electronics · doi:10.3390/electronics9060936 · Cited by 17
“rning on scheduling or resource allocation problem. DRM first employs deep reinforcement learning to schedule a simple job resource allocation which does not have job hierarchy and homogeneous setting [24]. A consecutive work, Decima [25], schedules randomly injecting jobs to homogeneous resources and uses message pass…”
BackgroundBen-Ameur, A., Araldo, A., Chahed, T., & Dán, G. (2025) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2025.3538158 · Cited by 15
Algorithmic Techniques for GPU Scheduling: A Comprehensive Survey
Chab, R., Setia, S. K., & Li, F. (2025) · Algorithms · doi:10.3390/a18070385 · Cited by 15
A Deep Deterministic Policy Gradient Based Network Scheduler For Deadline-Driven Data Transfers
Ghosal, G. R., Ghosal, D., Sim, A., Thakur, A., & Wu, K. J. (2020) · eScholarship (California Digital Library) · Cited by 15
MRSch: Multi-Resource Scheduling for HPC
Li, B., Fan, Y., Dearing, M. T., Lan, Z., Rich, P., Allcock, W. E., & Papka, M. E. (2022) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/cluster51413.2022.00020 · Cited by 15
“Previous scheduling studies for data centers also used a fixed-size 2D image for encoding job and system information (i.e., one dimension for resource availability and the other for time duration) [14].”
BackgroundMethodAn Alternative to Cognitivism: Computational Phenomenology for Deep Learning
Beckmann, P., Köstner, G., & Hipólito, I. (2023) · Minds and Machines · doi:10.1007/s11023-023-09638-w · Cited by 20
A Dual-Agent Scheduler for Distributed Deep Learning Jobs on Public Cloud via Reinforcement Learning
Xing, M., Mao, H., Yin, S., Pan, L., Zhang, Z., Xiao, Z., & Long, J. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599241 · Cited by 15
“DeepRM [39] uses a bitmap to model cluster state and employs REINFORCE algorithm [66] to decide the time slot when a job should be executed.”
MethodSelf-citationEfficiency traps beyond the climate crisis: exploration–exploitation trade-offs and rebound effects
Segovia‐Martín, J., Creutzig, F., & Winters, J. (2023) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2022.0405 · Cited by 16
“These models can help balance the trade-off between trying out new methods to conserve resources and using currently effective methods [48].”
MethodCollaborative Multi-Agent Reinforcement Learning Approach for Elastic Cloud Resource Scaling
Fang, B., & Gao, D. (2025) · International Conference on Artificial Intelligence Technologies and Applications (ICAITA) · doi:10.1109/icaita67588.2025.11137847 · Cited by 15
“DeepRM[27] 72.4 3.85 11.9 AutoScale[28] 74.7 2.41 9.7 CloudSim-AutoML[25] 70.2 4.12 14.3 DRL-Scheduler[16] 76 Experimental results demonstrate that the proposed multi-agent elastic scaling strategy achieves superior performance in resource utilization, with an average CPU usage of 78.6%,…”
InfluentialDeep reinforcement learning for quantum Szilard engine optimization
Sørdal, V. B., & Bergli, J. (2019) · Physical Review A · doi:10.1103/physreva.100.042314 · Cited by 15
“[28] Hongzi Mao, Mohammad Alizadeh, Ishai Menache, and Srikanth Kandula, “Resource management with deep reinforcement learning,” in Proceedings of the 15th ACM Workshop on Hot Topics in Networks (ACM, 2016) pp.”
BackgroundDeep reinforcement learning for scheduling in large-scale networked control systems
Redder, A., Ramaswamy, A., & Quevedo, D. E. (2019) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2019.12.177 · Cited by 15
Delay-sensitive Task Scheduling with Deep Reinforcement Learning in Mobile-edge Computing Systems
Meng, H., Chao, D., Guo, Q., & Li, X. (2019) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1229/1/012059 · Cited by 15
“…tasks in the queue and the resource allocation status of execution unit and transmission unit of mobile device in the current and future T timesteps[11], as described in Figure 3, the horizontal axis indicates time, the rightmost column of the vertical axis shows the allocation status of CPU in…”
MethodDynamic Colocation Policies with Reinforcement Learning
Li, Y., Sun, D., & Lee, B. C. (2020) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3375714 · Cited by 15
“In addition, reinforcement learning has also been applied to datacenter resource allocation [19, 27, 28, 39, 43].”
MethodRaven: Scheduling Virtual Machine Migration During Datacenter Upgrades with Reinforcement Learning
Ying, C., Li, B., Ke, X., & Guo, L. (2020) · Mobile Networks and Applications · doi:10.1007/s11036-020-01632-1 · Cited by 15
“Since the number of stateaction pairs in complex decision-making problems would be too large to store in tabular form, it is common to use function approximators, such as deep neural networks [6] [7] [8] [9].”
BackgroundToward Efficient Online Scheduling for Distributed Machine Learning Systems
Yu, M., Liu, J., Wu, C., Ji, B., & Bentley, E. S. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3104513 · Cited by 15
Towards High-Quality CGRA Mapping with Graph Neural Networks and Reinforcement Learning
Zhuang, Y., Zhang, Z., & Liu, D. (2022) · doi:10.1145/3508352.3549458 · Cited by 15
He, B., Wang, J., Qi, Q., Sun, H., & Liao, J. (2020) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2020.2985651 · Cited by 15
A Reinforcement Learning Based System for Minimizing Cloud Storage Service Cost
Wang, H., Shen, H., Liu, Q., Zheng, K., & Xu, J. (2020) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3404397.3404466 · Cited by 16
“DQN has been successfully used to solve large-scale RL tasks [26, 27, 30, 36].”
MethodAdaptive ABAC Policy Learning: A Reinforcement Learning ApproachPreprint
Karimi, L., Abdelhakim, M., & Joshi, J. B. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08587 · Cited by 23
“Recently, RL is used in designing protocols for controlling congestion [9] as well as resource management in networks [24] and resource allocation for vehicle-to-vehicle communications [38].”
MethodIntent-based networking for the enterprise: a modern network architecture
Falkner, M., & Apostolopoulos, J. (2022) · Communications of the ACM · doi:10.1145/3538513 · Cited by 14
Noah: Reinforcement-Learning-Based Rate Limiter for Microservices in Large-Scale E-Commerce Services
Li, Z., Sun, H., Xiong, Z., Huang, Q., Hu, Z., Li, D., Ruan, S., Hong, H., Gui, J., He, J., et al. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264038 · Cited by 15
E2EMap: End-to-End Reinforcement Learning for CGRA Compilation via Reverse Mapping
Liu, D., Xia, Y., Shang, J., Zhong, J., Ouyang, P., & Yin, S. (2024) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca57654.2024.00015 · Cited by 14
“Recently, inspired by the great processes [5], [36], [39], [47], [48], [54] made by artificial intelligence, Machine Learning (ML) has also been introduced to CGRA mapping.”
BackgroundRjoub, G., Elmekki, H., Bentahar, J., Pedrycz, W., Kassaymeh, S., Almobydeen, S. B., & Rachida Dssouli, A. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113574 · Cited by 14
Neural Task Scheduling with Reinforcement Learning for Fog Computing Systems
Bian, S., Huang, X., Shao, Z., & Yang, Y. (2019) · Globecom · doi:10.1109/globecom38437.2019.9014045 · Cited by 14
“Recently, a line of works have applied deep reinforcement learning (DRL) [5] techniques to solve online task scheduling problems [6] [7].”
BackgroundMethodQian, B., Su, J., Wen, Z., Jha, D. N., Li, Y., Guan, Y., Puthal, D., James, P., Yang, R., Zomaya, A. Y., et al. (2019) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.1910.05433 · Cited by 15
“forcement Learning Paradigm Discussion.In IoT environments, a variety of problems can be modelled by using the aforementioned three approaches. The applications range from system and networking [168] [167], smart city [310] [147], to smart grid [283] [223], etc. To begin with modelling, it is essential for users to ch…”
BackgroundInfluentialWill, J., Thamsen, L., Scheinert, D., Bader, J., & Kao, O. (2021) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e52221.2021.00018 · Cited by 14
“This data is gained either from dedicated profiling or previous full executions [3], [12], [15]– [20].”
MethodInfluentialNICE: Robust Scheduling through Reinforcement Learning-Guided Integer Programming
Kenworthy, L., Nayak, S., Chin, C., & Balakrishnan, H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21218 · Cited by 14
Interpretable Feedback for AutoML and a Proposal for Domain-customized AutoML for Networking
Arzani, B., Hsieh, K., & Chen, H. (2021) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3484266.3487373 · Cited by 14
“Today, this requires someone with expertise in both domains (networking and ML) to select the model, engineer features, identify corner cases where the model can fail, and optimize the model for performance and scale [1, 6, 15, 26, 34, 36, 51, 52, 55].”
BackgroundZhu, M., Gu, J., Shen, T., Shi, C., & Ren, X. (2021) · Journal of Lightwave Technology · doi:10.1109/jlt.2021.3120874 · Cited by 14
Cache Allocation in Multi-Tenant Edge Computing via online Reinforcement Learning
Ben-Ameur, A., Araldo, A., & Chahed, T. (2022) · International Conference on Communications · doi:10.1109/icc45855.2022.9838489 · Cited by 14
Zhu, L., Wu, F., Hu, Y., Huang, K., & Tian, X. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08208-6 · Cited by 16
“Most of the existing related methods tend to devise the cluster scheduling from the perspective of the balance between task and resource, such as cluster energy consumption [30], resource allocation [11, 12] and utilization [13, 31, 32, 34].”
BackgroundLiu, H., Chen, P., & Zhao, Z. (2021) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud53861.2021.00082 · Cited by 14
“…in red depicts the system time t cur ; Each task is with its arrival time T arrj and execution time T lenj , then we propose the execution slowdown [18] value of each task Q T j ( t ) as follows: Based on this, we formulate the function f ( Q T j ( t ) to judge if execution slowdown of a task is…”
BackgroundMethodInfluentialAuto-tuning Distributed Stream Processing Systems using Reinforcement LearningPreprint
Vaquero, L. M., & Cuadrado, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05495 · Cited by 17
“Algorithm 1 Adapted REINFORCE algorithm, based on [31, 50]”
BackgroundMethodInfluentialConstrained Combinatorial Optimization with Reinforcement LearningPreprint
Solozabal, R., Ceberio, J., & Takáč, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11984 · Cited by 14
“E.g., [28, 29, 26, 27] address scheduling problems relying on masking schemes to avoid unfeasible actions during the resolution process.”
Xiang, X., Foo, S. Y., & Zang, H. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3040043 · Cited by 18
“[95] presented DeepRM in 2016, which translates the problem with multiple resource demands into a learning problem.”
BackgroundFrom Design to Deployment of Zero Touch Deep Reinforcement Learning WLANs
Iacoboaiea, O., Krolikowski, J., Houidi, Z. B., & Rossi, D. (2022) · IEEE Communications Magazine · doi:10.1109/mcom.002.2200318 · Cited by 13
“Following a similar path, recent attempts to use DRL instead of heuristics for automating network resource allocation [4], routing [5], WLAN configuration [6], and more [7–9] have shown promising results.”
MethodInfluentialPunitha, P., Sivaparthipan, C. B., Muthu, B. A., & Kumar, R. L. (2024) · International Journal of Communication Systems · doi:10.1002/dac.5850 · Cited by 13
“23 Utilizing motivation from advances in AI, DeepRM transforms the problem of packaging employment with varying resource needs into a learning problem.”
Demeter: Fine-grained Function Orchestration for Geo-distributed Serverless Analytics
Yue, X., Yang, S., Zhu, L., Trajanovski, S., & Fu, X. (2024) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom52122.2024.10621303 · Cited by 13
“We set the discount factor γ to be 1 as these actions are of equal importance over time [26].”
BackgroundAI-enabled routing in next generation networks: A survey
Aktas, F., Shayea, I. A. M., Ergen, M., Saoud, B., Yahya, A. E., & Aldasheva, L. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.01.095 · Cited by 21
Harvesting Randomness to Optimize Distributed Systems
Lécuyer, M., Lockerman, J., Nelson, L., Sen, S., Sharma, A. K., & Slivkins, A. (2017) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3152434.3152435 · Cited by 13
“Other applications do not use a model, but rely on continuous interactions with the environment ( i.e., invasive deployments) to learn a good policy [3, 6, 10–12, 24, 26, 32].”
BackgroundMarVeLScaler: A Multi-View Learning-Based Auto-Scaling System for MapReduce
Li, Y., Liu, F., Chen, Q., Sheng, Y., Zhao, M., & Wang, J. (2019) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2019.2944916 · Cited by 13
“Reinforcement learning based models like [41] can transform the problem of packing tasks with multiple resource demands into an optimization problem, demonstrating good adaption, quick convergence, and learning of smart strategies.”
BackgroundPnP-DRL: A Plug-and-Play Deep Reinforcement Learning Approach for Experience-Driven Networking
Xu, Z., Wu, K., Zhang, W., Tang, J., Wang, Y., & Xue, G. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087270 · Cited by 13
“Mao et al. [39] presented DeepRM, which enables a cluster system to learn the best resource scheduling policy from its control experience.”
BackgroundDynamic Intelligent Supply-Demand Adaptation Model Towards Intelligent Cloud Manufacturing
Sun, Y., Qiao, F., Wang, W., Xu, B., Zhu, J., Mansour, R. F., & Qi, J. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.026574 · Cited by 13
Lin, M., Чен, Т., Ren, B., Chen, H., Zhang, M., & Guo, D. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3236104 · Cited by 12
“Alizadeh translates the packing task problem into a learning problem and presents a model called DeepRM to solve it [42].”
Woo, S., Yeon, J., Ji, M., Moon, I., & Park, J. (2018) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2018.00717 · Cited by 13
“The most well-known real-world applications are from game algorithms [1]–[3], but other domains such as finance [4], resource management [5], [6] or generic object control [7]– [9] are also active areas.”
BackgroundLi, B., Wang, Y., Wang, R., Tai, C., Iyer, R., Zhu, Z., Herdrich, A., Zhang, T., Haj-Ali, A., Stoica, I. G., et al. (2020) · IEEE International Conference on Network Softwarization (NetSoft) · doi:10.1109/netsoft48620.2020.9165471 · Cited by 14
“Recent advancement in deep RL provides new opportunities to optimize resource management and scheduling [33], [48]– [53].”
BackgroundMeta-Scheduling for the Wireless Downlink Through Learning With Bandit Feedback
Song, J., Veciana, G. A. D., & Shakkottai, S. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3117783 · Cited by 12
“Not surprisingly recently, reinforcement learning (RL) approaches have been proposed to address complex scheduling problems, including job scheduling for data centers [17] and wireless scheduling in various settings [6], [9], [19], [30].”
BackgroundA design methodology for deep reinforcement learning in autonomous systems
Hillebrand, M., Lakhani, M., & Dumitrescu, R. (2020) · Procedia Manufacturing · doi:10.1016/j.promfg.2020.11.044 · Cited by 14
Supporting Sustainable Virtual Network Mutations With Mystique
Sacco, A., Flocco, M., Esposito, F., & Marchetto, G. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3059647 · Cited by 13
“Although reinforcement learning is emerging as a valuable technique to solve many networking problems, as in [7], [8], there is no solution incorporating network information to automatically and efficiently orchestrate network resources in a decentralized manner.”
BackgroundKarthik, P., & Sekhar, K. (2021) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12041 · Cited by 13
“Among the recent algorithms used for resource scheduling is the DeepRM [14].”
BackgroundMethodInfluentialToward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement LearningPreprint
You, X., Li, X., Xu, Y., Feng, H., Zhao, J., & Yan, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03494 · Cited by 13
“Recent successful applications include cloud resource allocation [20], adaptive bitrate video streaming”
BackgroundMultiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches.Preprint
Wong, A., Bäck, T., Kononova, A. V., & Plaat, A. (2021) · arXiv (Cornell University) · Cited by 13
DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning
Hasanbeig, M., Jeppu, N. Y., Abate, A., Melham, T., & Kroening, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16935 · Cited by 17
“Reinforcement Learning (RL) is the key enabling technique for a variety of applications of artificial intelligence, including advanced robotics (Polydoros and Nalpantidis 2017), resource and traffic management (Mao et al. 2016; Sadigh et al. 2014), drone control (Abbeel et al.”
BackgroundShao, X., & Kim, C. S. (2022) · IEEE Access · doi:10.1109/access.2022.3188765 · Cited by 12
“[31] and Luo [32] applied deep reinforcement learning (DRL) to process JSSP.”
Joint Routing and Scheduling for CQF
Liu, Y., Cheng, Z., Ren, J., & Yang, D. (2022) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs55155.2022.9846349 · Cited by 12
PHH: Policy-Based Hyper-Heuristic With Reinforcement Learning
Udomkasemsub, O., Sirinaovakul, B., & Achalakul, T. (2023) · IEEE Access · doi:10.1109/access.2023.3277953 · Cited by 12
“[21] designed a way to apply reinforcement learning to a multi-resource cluster scheduling problem.”
MethodOn Optimizing Traffic Scheduling for Multi-replica Containerized Microservices
Zhu, X., Li, Y., Yao, L., Qi, Z., Xu, Y., Wang, P., Wang, W., & Zhu, X. (2023) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3605573.3605646 · Cited by 12
Hwamei: A Learning-Based Synchronization Scheme for Hierarchical Federated Learning
Qi, T., Zhan, Y., Li, P., Guo, J., & Xia, Y. (2023) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs57875.2023.00047 · Cited by 15
“Reinforcement learning [15] is a method for maximizing rewards through continuous interaction between the agent and the environment.”
MethodKarma: Adaptive Video Streaming via Causal Sequence Modeling
Xu, B., Chen, H., & Ma, Z. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612177 · Cited by 12
“Although learning-based ABR approaches (e.g., imitation learning (IL) [17, 35] and reinforcement learning (RL) [22, 31, 32, 43] algorithms) have shown superior performances, they solely stack past observations to decide the next action, as illustrated in Figure 1(a).”
BackgroundSmart-Kube: Energy-Aware and Fair Kubernetes Job Scheduler Using Deep Reinforcement Learning
Ghafouri, S., Abdipoor, S., & Doyle, J. C. (2023) · IEEE International Conference on Smart Cloud (SmartCloud) · doi:10.1109/smartcloud58862.2023.00035 · Cited by 12
“Our representations are inspired by the previous work in this field [1], [7], [8], [16], [20].”
MethodCrowdsourcing Live High Definition Map via Collaborative Computation in Automotive Edge Computing
Zhang, Y., Liu, Q., Wang, H., Chen, D., & Han, K. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3393728 · Cited by 12
Xing, S., Wang, Y., & Liu, W. (2025) · Symmetry · doi:10.3390/sym17071109 · Cited by 12
Learning Workflow Scheduling on Multi-Resource Clusters
Hu, Y., de Laat, C. T. A. M., & Zhao, Z. (2019) · IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas.2019.8834720 · Cited by 13
“Scheduling with Reinforcement Learning DeepRM [29] is the first example solution that applies deep reinforcement learning to cluster scheduling problem.”
Zhou, G., Wu, R., Hu, M., Zhou, Y., Fu, T. Z. J., & Wu, D. (2021) · International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV) · doi:10.1145/3458306.3460993 · Cited by 12
“Reinforcement learning (RL) is a kind of machine learning framework [16, 18], in which an agent learns how to act by interacting with the environment.”
BackgroundReinforcement Learning: A Friendly Introduction
Daoun, D., Ibnat, F., Alom, Z., Aung, Z., & Azim, M. A. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-84337-3_11 · Cited by 13
Fast and Fine-grained Autoscaler for Streaming Jobs with Reinforcement Learning
Xing, M., Mao, H., & Xiao, Z. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/80 · Cited by 12
“To remedy these drawbacks, another line of work [Mao et al. , 2016] formulates the autoscaling process as Markov Decision Process (MDP), and adopt Reinforcement Learning (RL) to train the autoscalers, namely RL-based autoscalers .”
MethodSelf-citationZhang, K., Wang, P., Gu, N., & Nguyen, T. D. (2022) · Symposium on Cloud Computing · doi:10.1145/3542929.3563501 · Cited by 13
“Therefore, unlike previous work [4, 27], our CA’s policy network structure is independent of the problem size.”
BackgroundMethodDeep Reinforcement Learning for Multi-resource Cloud Job Scheduling
Lin, J., Peng, Z., & Cui, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04182-3_26 · Cited by 12
Meng, H., Chao, D., Huo, R., Guo, Q., Li, X., & Huang, T. (2019) · International Conference on Mathematics and Artificial Intelligence (ICMAI) · doi:10.1145/3325730.3325731 · Cited by 12
A Deep-Reinforcement-Learning-Based Scheduler for FPGA HLS
Chen, H., & Shen, M. (2019) · 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) · doi:10.1109/iccad45719.2019.8942126 · Cited by 14
“The biggest difference between our problem and the resource management problem described in [20] is the dependency.”
BackgroundMethodInfluentialZhang, J., Guo, Z., Ye, M., & Chao, H. J. (2020) · doi:10.1145/3405671.3405809 · Cited by 12
“Inspired by [20][17], we define the action space as {0, 1, ..., N ∗ ( N − 1 ) − 1} and allow the RL agent to sample K different actions for each instance s (i.e., a 1 , a 2 , ..., a K ).”
MethodA Hybrid Learning Framework for Service Function Chaining Across Geo-Distributed Data Centers
Tang, T., Wu, B., & Hu, G. (2020) · IEEE Access · doi:10.1109/access.2020.3024135 · Cited by 12
5G Scheduling using Reinforcement Learning
Zavyalova, D. V., & Геннадьевна, Д. В. (2020) · International Multi-Conference on Industrial Engineering and Modern Technologies (FarEastCon) · doi:10.1109/fareastcon50210.2020.9271421 · Cited by 12
“In [5] authors discuss the feasibility to use state-of-the-art Deep Reinforcement Learning (RL) methods to large-scale systems, showing that the RL agent is comparable and in some cases even better than ad-hoc heuristics for a multi-resource cluster resource allocation issue.”
BackgroundSelf-Supervised Long-Short Term Memory Network for Solving Complex Job Shop Scheduling Problem
Shao, X., & Kim, C. S. (2021) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2021.08.016 · Cited by 12
“E . g., Mao et al. [25] applied DRL with one deep NN (DNN) structure to manage the resource of CPU and memory.”
MethodSmart DAG Tasks Scheduling between Trusted and Untrusted Entities Using the MCTS Method
Cheng, Y., Wu, Z. S., Liu, K., Wu, Q., & Wang, Y. (2019) · Sustainability · doi:10.3390/su11071826 · Cited by 12
“[13] proposed using deep reinforcement learning to solve a simplified task s scheduling problem.”
BackgroundMethodCoordinating Disaster Emergency Response with Heuristic Reinforcement LearningPreprint
Nguyen, L., Yang, Z., Zhu, J., Li, J., & Jin, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05010 · Cited by 12
“Their initial results show that model based on deep learning outperforms the heuristic model, and their model can adapt to different conditions, converges quickly, and learns strategies that are sensible in hindsight [36].”
Background
Metadata
- Authors
- Mao, H., Alizadeh, M., Menache, I., & Kandula, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 15th ACM Workshop on Hot Topics in Networks
- Publisher
- ACM
- Published
- 2016-11-09
- Pages
- 50-56
- OpenAlex
- W2546571074
- Cited by
- 1,353
- References
- 41
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 98 · 2023: 140 · 2022: 151 · 2021: 191 · 2020: 180 · 2019: 155 · 2018: 62 · 2017: 24
- FWCI
- 68.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Hot Topics in Networks (HotNets) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- resource management, deep reinforcement learning, task scheduling, resource allocation, workload modeling, online decision making, experiential learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
