The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Resource Management with Deep Reinforcement Learning
Abstract
Resource management problems in systems and networking often manifest as difficult online decision making tasks where appropriate solutions depend on understanding the workload and environment. Inspired by recent advances in deep reinforcement learning for AI problems, we consider building systems that learn to manage resources directly from experience. We present DeepRM, an example solution that translates the problem of packing tasks with multiple resource demands into a learning problem. Our initial results show that DeepRM performs comparably to state-of-the-art heuristics, adapts to different conditions, converges quickly, and learns strategies that are sensible in hindsight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resource management problems in practical settings.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [294]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. (2010) · Communications of the ACM · doi:10.1145/1721654.1721672 · Cited by 9,464
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Research works printed but not linked: 20
- Agarwal S., 2012, NSDI, 281
- Ananthanarayanan G., 2010, In OSDI, number, 1, 24
- Bertsekas D. P., 1995, Decision and Control
- Chilimbi T., 2014, OSDI, 571
- Dong M., 2015, NSDI, 395
- Ghodsi A., 2011, NSDI'11, 323
- Heller B., 2010, NSDI'10
- Junchen J., 2015, SIGCOMM '15
- Kaelbling L. P., 1996, Journal of artificial intelligence research
- Mahadevan S., 1998, FLAIRS Conference
- Mnih V., 2016, CoRR
- Mnih V., 2013, CoRR
- Mnih V., 2015, Nature
- J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. · arXiv
- 10.5555/551283 · DOI
- Sutton R. S., 1999, NIPS
- Winstein K., 2013, NSDI, 459
- Yi S., 2016, SIGCOMM
- Yin X., 2016, SIGCOMM '16
- Zhang W., 1995, IJCAI. Citeseer
Cited by
Sorted by citations
Items 601–700 of 1,000
Towards Scalable Verification of RL-Driven SystemsPreprint
Amir, G., Schapira, M., & Katz, G. (2021) · arXiv.org · doi:10.48550/arxiv.2105.11931 · Cited by 4
“We evaluate whiRL 2.0 on three case studies of DRL systems: the Aurora [27] congestion controller, the Pensieve [43] adaptive video streamer, and the DeepRM [42] compute resource scheduler.”
BackgroundMethodInfluentialHarvesting Idle Resources in Serverless Computing via Reinforcement Learning.Preprint
Yu, H., Wang, H., Li, J., & Park, S. (2021) · arXiv (Cornell University) · Cited by 8
Reinforcement Learning for Service Function Chain Allocation in Fog Computing
Santos, J., Wauters, T., Volckaert, B., & Turck, F. D. (2021) · doi:10.1002/9781119675525.ch7 · Cited by 8
Optimized dynamic service placement for enhanced scheduling in fog-edge computing environments
Lin, Y., Shi, Y., & Mohammadnezhad, N. (2024) · Sustainable Computing Informatics and Systems · doi:10.1016/j.suscom.2024.101037 · Cited by 7
Enhancing QoE of Adaptive Video Streaming by Generating Fine-Grained Throughput
An, C., Zhang, H., Kang, J., Liu, Z., Zhou, A., Liu, L., & Ma, H. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3508262 · Cited by 8
“The most recent solutions have enhanced the rule-driven algorithms by machine learning or deep learning techniques [5], [8], [17], [24], [25], [26], [27], [28], [29], [30], i.e. , learning-driven ABR algorithms.”
AI Surrogate Model for Distributed Computing Workloads
Park, D. K., Ren, Y., Kilic, O. O., Korchuganova, T., Vatsavai, S. S., Boudreau, J. F., Chowdhury, T., Feng, S., Khan, R., Kim, J., et al. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/scw63240.2024.00018 · Cited by 7
“In job scheduling scenarios involving large, complex, and heterogeneous computing environments, heuristic algorithms typically experience significant performance drops [32], while neural-network-based algorithms [33]–[35] tend to maintain more efficient scheduling.”
Topology-aware GPU job scheduling with deep reinforcement learning and heuristics
Ayadi, H., An, A., Shao, Y., Pourmedheji, H., Deng, J., Huang, J. X., Feiman, M., & Zhou, H. (2025) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2025.105138 · Cited by 8
Dhinesh Kumar, R., & Rammohan, A. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.106100 · Cited by 7
Ding, G., Yang, S., Lin, H., Chen, Z., & Yang, J. C. (2026) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2026.3667549 · Cited by 7
“DeepRM [12]: Deep reinforcement learning scheduler trained on the training set.”
Machine Learning and Data Networks: Perspectives, Feasibility, and Opportunities
Lozada-Yánez, R., Molina-Granja, F., Yánez, P. E. L., & Yungán, J. I. G. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-45691-7_26 · Cited by 7
Birman, Y., Ido, Z., Katz, G., & Shabtai, A. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534433 · Cited by 7
“complex games [24] and scheduling [8].”
BackgroundMethodInfluentialLeveraging Domain Knowledge for Robust Deep Reinforcement Learning in Networking
Zheng, Y., Chen, H., Duan, Q., Lin, L., Shao, Y., Wang, W., Wang, X., & Xu, Y. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488863 · Cited by 7
“Representative applications include job scheduling [3], [4], congestion control [5], adaptive video streaming [6]–[9], and routing [10], [11].”
BackgroundTowards designing a generic and comprehensive deep reinforcement learning framework
Nguyen, N. D., Nguyen, T. T., Pham, N. T., Nguyen, V., Nguyen, D. T., Nguyen, T. T., Lim, C. P., Johnstone, M., Bhatti, A., Creighton, D., et al. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03550-z · Cited by 7
“Therefore, RL is used in a variety of domains, such as IT resources management [3], cyber-security [4], robotics [5–8], control systems [9–12], recommendation systems [13], stock trading strategies [14], bidding and advertising campaigns [15], and video games [16–18].”
BackgroundDu, H., Chen, Y., Feng, X., Xiang, Q., & Liu, H. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.109517 · Cited by 7
Wang, X., Shen, H., & Tian, H. (2023) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2023.3281710 · Cited by 7
“For example, DeepRM [22] formulates the problem of packing tasks with different resource requirements as a RL problem.”
BackgroundMethodVerifying the Generalization of Deep Learning to Out-of-Distribution Domains
Amir, G., Maayan, O., Zelazny, T., Katz, G., & Schapira, M. (2024) · Journal of Automated Reasoning · doi:10.1007/s10817-024-09704-7 · Cited by 8
Learning-Based Two-Tiered Online Optimization of Region-Wide Datacenter Resource Allocation
Chen, C.-L., Zhou, H., Chen, J., Pedramfar, M., Lan, T., Zhu, Z., Zhou, C., Ruiz, P. M., Kumar, N., Dong, H., et al. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3484213 · Cited by 8
“A traditional approach to improving resource allocation efficiency is to group servers into physical clusters, which reduces the number of candidate servers for container placement [40].”
BackgroundMethodPractically High Performant Neural Adaptive Video Streaming
Patel, S., Zhang, J., Narodytska, N., & Jyothi, S. A. (2024) · Proceedings of the ACM on Networking · doi:10.1145/3696401 · Cited by 8
A Survey for Blockchain-Enabled Resource Management in Edge Networks
Boamah, S. A., Louis, J. D., McNair, J. Y., & Gilbert, J. E. (2025) · IEEE Access · doi:10.1109/access.2025.3562126 · Cited by 8
“A common ML method in network resource management is Reinforcement Learning [30], [31].”
ALLIANCE Project: Architecting a Knowledge-Defined 5G-Enabled Network Infrastructure
Careglio, D., Spadaro, S., Cabellos‐Aparicio, A., Lázaro, J. A., Perelló, J., Barlet, P., Gené, J. M., & Paillissé, J. (2018) · International Conference on Transparent Optical Networks · doi:10.1109/icton.2018.8473626 · Cited by 7
“, [5]).”
Mitra, S., Mondal, S. S., Sheoran, N., Dhake, N., Nehra, R., & Simha, R. (2019) · ACM SIGOPS Asia-Pacific Workshop on Systems (APSys) · doi:10.1145/3343737.3343741 · Cited by 7
“ulative discounted reward: E[ ˝ ∞ t=0 γ tr t]where γ ∈(0,1)determines how much the future rewards contribute to the total reward. More details of theoretical background of RL can be found in [41] and [26]. Inspired by the recent trends in Deep Reinforcement Learning (DeepRL), in this paper, we use deepneural-networks…”
BackgroundMethodInfluentialBaseband Unit Aggregation Based on Deep Reinforcement Learning in Cloud Radio Access Networks
Gu, J., Zhu, M., Chen, B., Shen, T., & Ren, X. (2019) · International Conference on Optical Communications and Networks (ICOCN) · doi:10.1109/icocn.2019.8934931 · Cited by 7
“Results have been obtained by using a modified simulator presented in [3].”
BackgroundMethod"I’m Sorry Dave, I’m Afraid I Can’t Do That" Deep Q-Learning from Forbidden Actions
Seurin, M., Preux, P., & Pietquin, O. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207496 · Cited by 17
“, 2016), Resource Management (Mao et al., 2016), Chemical reaction (Zhou et al.”
BackgroundDeep Deterministic Policy Gradient for Portfolio Management
Khemlichi, F., Chougrad, H., Khamlichi, Y. I., Boushaki, A. E., & Ali, S. E. B. (2020) · IEEE Congress on Information Science and Technology (CiSt) · doi:10.1109/cist49399.2021.9357266 · Cited by 7
“Reinforcement Learning has been successfully applied in a large number of fields such as resource management [7], helicopter flight [8], and chemistry [9] among others.”
BackgroundDeep Reinforcement Learning for Intelligent Cloud Resource Management
Zhou, Z., Luo, K., & Chen, X. (2021) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infocomwkshps51825.2021.9484566 · Cited by 7
DRL-Deploy:Adaptive Service Function Chains Deployment with Deep Reinforcement Learning
Wei, X., Sheng, Y., Li, L., & Zhou, C. (2021) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-socialcom-sustaincom52081.2021.00027 · Cited by 7
Fuzzy Noisy Network for Stable Exploration
Gao, Q., Zhang, Y., & Liu, Y. (2021) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct52962.2021.9657841 · Cited by 7
Downlink Scheduling in LTE with Deep Reinforcement Learning, LSTMs and Pointers
Robinson, A., & Kunz, T. (2021) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom52596.2021.9652950 · Cited by 8
“The authors [3] defined the problem in a continuous rather than discrete manner, representing the state variables as distributions and then applied a policy gradient method based on the REINFORCE [6] algorithm seen in [7] to solve it.”
MethodImproving QoS through network isolation in PaaS
Stefano, A. D., Stefano, A. D., & Morana, G. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.01.010 · Cited by 7
Actor-Critic with Transformer for Cloud Computing Resource Three Stage Job Scheduling
Xu, Y., & Zhao, J. (2022) · International Conference on Cloud Computing and Big Data Analytics (ICCCBDA) · doi:10.1109/icccbda55098.2022.9778883 · Cited by 8
HELICON: Orchestrating low-latent & load-balanced Virtual Network Functions
Bunyakitanon, M., Vasilakos, X., Nejabati, R., & Simeonidou, D. E. (2022) · International Conference on Communications · doi:10.1109/icc45855.2022.9839199 · Cited by 7
“The greatest challenges regard designing the model itself, monitoring and feeding data to it, and the particular complexity [10] of MOO with RL. GRLmodule Mao et al. [11] show the benefits of applying Deep RL agents to large-scale systems.”
BackgroundPeng, H., Wu, C., Zhan, Y., & Xia, Y. (2022) · ACM Conference on Research in Adaptive and Convergent Systems (RACS) · doi:10.1145/3538641.3561487 · Cited by 7
“Some related works do not consider the dependencies among the tasks, such as DeepRM [13] and DeepRM+[7].”
BackgroundInter-Cell Network Slicing With Transfer Learning Empowered Multi-Agent Deep Reinforcement Learning
Hu, T., Liao, Q., Liu, Q., & Carle, G. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3273310 · Cited by 7
“Recent advances in model-free approaches, especially deep reinforcement learning (DRL) [9], [10], have shown promising potential in automatically learning to manage radio access networks without the need for prior models.”
BackgroundWeRLman: To Tackle Whale (Transactions), Go Deep (RL)
Bar-Zur, R., Abu-Hanna, A., Eyal, I., & Tamar, A. (2022) · IEEE Symposium on Security and Privacy · doi:10.1109/sp46215.2023.10179444 · Cited by 31
“Deep RL has been applied to various decision-making domains, including resource management [43] and games [22], [24].”
Artificial Intelligence Advancement for 6G Communication: A Visionary Approach
Miya, J., Raj, S., Ansari, M. A., Kumar, S. M., & Kumar, R. (2023) · Springer eBooks · doi:10.1007/978-3-031-30101-8_15 · Cited by 8
Padur, S. K. R. (2019) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit1936414 · Cited by 7
Data-driven resource flexing for network functions visualization
Cao, L., Fahmy, S., Sharma, P. K., & Zhe, S. (2018) · ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS) · doi:10.1145/3230718.3230725 · Cited by 7
“[25] solve the multi-resource allocation problem in a reinforcement learning framework.”
BackgroundSpoken Language Acquisition Based on Reinforcement Learning and Word Unit Segmentation
Gao, S., Hou, W., Tanaka, T., & Shinozaki, T. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053326 · Cited by 8
“It has proven to be effective in many challenging tasks such as computer resource management [19], robotics [20] and even chemistry [21].”
BackgroundHugo: A Cluster Scheduler that Efficiently Learns to Select Complementary Data-Parallel Jobs
Thamsen, L., Verbitskiy, I., Nedelkoski, S., Tran, V. T., Meyer, V., Xavier, M. G., Kao, O., & Rose, C. A. F. D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-48340-1_40 · Cited by 7
“– Solution Practicality: Some systems do not incorporate objectives besides throughput [13, 22, 14], while others assume control over more than just job order [5, 6, 14] or require instrumentation not generally supported [12].”
BackgroundInfluentialTransfer of Hierarchical Reinforcement Learning Structures for Robotic Manipulation Tasks
Scheiderer, C., Mosbach, M., Posada-Moreno, A. F., & Meisen, T. (2020) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci51800.2020.00091 · Cited by 7
“Especially in combination with artificial neural networks, many control tasks such as games [11], resource management [12], and robotics [13] have successfully been learned with reinforcement learning.”
BackgroundTowards Low-latent & Load-balanced VNF Placement with Hierarchical Reinforcement Learning
Vasilakos, X., Bunyakitanon, M., Nejabati, R., & Simeonidou, D. E. (2021) · IEEE International Mediterranean Conference on Communications and Networking (MeditCom) · doi:10.1109/meditcom49071.2021.9647532 · Cited by 7
“Regarding large-scale systems, in particular, RL agents such as in the current work are proved [12] to perform better than ad-hoc heuristics for a multi-resource cluster scheduling problem, as well as over other model-based approaches [13] derived from queuing theory.”
BackgroundIntelligent scaling for 6G IoE services for resource provisioning
Alharbi, A., Alyami, H., Poongodi, M., Rauf, H. T., & Kadry, S. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.755 · Cited by 7
RetractedAn Efficient Combinatorial Optimization Model Using Learning-to-Rank Distillation
Woo, H., Lee, H., & Cho, S. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20845 · Cited by 7
Mitigating Bottlenecks in Wide Area Data Analytics via Machine Learning
Wang, H., & Li, B. (2018) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2018.2816951 · Cited by 7
“For example, DeepRM [31] builds a deep reinforcement learning model for strategies of cluster resource management.”
MethodDeepScheduling: Grid Computing Job Scheduler Based on Deep Reinforcement Learning
Casagrande, L. C., Koslovski, G., Miers, C. C., & Pillon, M. A. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-44041-1_89 · Cited by 7
Learning Index Selection with Structured Action SpacesPreprint
Welborn, J., Schaarschmidt, M., & Yoneki, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.07440 · Cited by 9
“…twenty years in areas like routing (Boyan & Littman, 1994) and server resource allocation (Tesauro et al., 2006) and recently, re-1 search in deep RL has featured a few applications in areas like cluster scheduling (Mao et al., 2016) and TensorFlow device placement (Mirhoseini et al., 2017; 2018).”
I'm sorry Dave, I'm afraid I can't do that, Deep Q-learning from forbidden actionPreprint
Seurin, M., Preux, P., & Pietquin, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02078 · Cited by 7
“, 2016), Resource Management (Mao et al., 2016), Chemical reaction (Zhou et al.”
BackgroundMava: a research framework for distributed multi-agent reinforcement learning.Preprint
Pretorius, A., Tessera, K.-A., Smit, A. P., Formanek, C., Grimbly, S. J., Eloff, K., Danisa, S., Francis, L., Shock, J. P., Kamper, H., et al. (2021) · arXiv (Cornell University) · Cited by 8
Exploring the Role of Machine Learning in Scientific Workflows: Opportunities and ChallengesPreprint
Nouri, A., Davis, P. E., Subedi, P., & Parashar, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13999 · Cited by 7
“provide an optimized solution to the problem of scheduling multi-objective workflows, several models have been proposed using ML algorithms [67].”
Heidarpour, A. R., Heidarpour, M. R., Ardakani, M., Tellambura, C., & Uysal, M. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3400692 · Cited by 6
“Therefore, we treat time as frozen and not progressing while the controller decides on task dispatching [17].”
BackgroundLi, P., Cui, P., & Wang, H. (2025) · Sensors · doi:10.3390/s25061707 · Cited by 6
“These methods rely on specifically designed heuristic strategies, may lead to local optima [50], and are unable to meet the real-time computation requirements of large-scale dynamic scenarios.”
A Deep Reinforcement Learning and Graph Convolution Approach to On-Street Parking Search Navigation
Zhao, X., & Yan, Y. (2025) · Sensors · doi:10.3390/s25082389 · Cited by 6
“Mao et al. [32] effectively utilized deep Q-networks (DQN) for resource allocation issues, whereas Li et al. [33] introduced a policy gradient-based approach for vehicle routing optimization.”
DRL-Based Time-Varying Workload Scheduling With Priority and Resource Awareness
Liu, Q., Fan, Q., Zhang, X., Li, X., Wang, K., & Xiong, Q. (2025) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2025.3559610 · Cited by 6
“Mao et al. [20] pioneered the application of reinforcement learning for workload scheduling method named DeepRM.”
Chinnaraju, A. (2024) · World Journal of Advanced Engineering Technology and Sciences · doi:10.30574/wjaets.2024.13.2.0636 · Cited by 6
Gao, X., Dong, H., Zhang, L.-J., Wang, Y., Yang, X., & Li, Z. (2025) · Applied Sciences · doi:10.3390/app15168928 · Cited by 7
Song, X., Feng, J., Liu, L., Pei, Q., Yu, F. R., & Zhang, N. (2025) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2025.3619100 · Cited by 6
“For instance, the authors in [33] demonstrated the advantages of DRL in handling highly dynamic task arrival and time-varying system states.”
A fault-tolerant and load-balancing scheduler for independent tasks on cloud-based virtual machines
Hagras, T., & El-Sayed, G. A. (2025) · Cluster Computing · doi:10.1007/s10586-025-05857-1 · Cited by 6
Self-Healing AI Systems Using Multi-Agent Learning
Patel, T., Elengovan, A. K., Ranganathan, V., Parikh, M., & Kole, D. (2026) · International Seminar on Intelligent Business and Edge-Computing Research (ISIBER) · doi:10.1109/isiber68248.2026.11470173 · Cited by 6
Photovoltaic Power Forecasting in Dunhuang based on Recurrent Neural Network
Lv, Q., Ma, M., Jiang, X., Jiang, M., Yong, B., Shen, J., & Zhou, Q. (2021) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys53884.2021.00203 · Cited by 6
MARS: Malleable Actor-Critic Reinforcement Learning Scheduler
Baheri, B., Tronge, J., Fang, B., Li, A., Chaudhary, V., & Guan, Q. (2022) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/ipccc55026.2022.9894315 · Cited by 8
“Recently, several studies also started to leverage deep reinforcement learning in resource allocation and job scheduling in a distributed environment, such as DeepRM [29], and Decima 2022 IEEE International Performance, Computing, and Communications Conference (IPCCC)”
BackgroundZhang, J., Wang, T., & Cheng, L. (2023) · Applied Sciences · doi:10.3390/app131910689 · Cited by 7
Anticipatory Resource Allocation for ML Training
Chugh, T., Kandula, S., Krishnamurthy, A., Mahajan, R., & Menache, I. (2023) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3620678.3624669 · Cited by 6
“SIA VC uses two mechanisms - virtual clocks [14, 77] to determine the scheduling order of jobs and logical resource-time plans [27, 28, 43, 62] to avoid slowdowns.”
MethodSelf-citationAlgorithmic Complexity Attacks on Dynamic Learned Indexes
Yang, R., Kornaropoulos, E. M., & Cheng, Y. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3636218.3636232 · Cited by 6
DeepCTS: A Deep Reinforcement Learning Approach for AI Container Task Scheduling
Xiao, Z., Liu, K., Hu, M., & Wu, D. (2024) · Asia Conference on Algorithms, Computing and Machine Learning (CACML) · doi:10.1145/3654823.3654885 · Cited by 6
Artificial Intelligence in IoT Security: Review of Advancements, Challenges, and Future Directions
Srinivasan, N. (2024) · International Journal of Innovative Technology and Exploring Engineering · doi:10.35940/ijitee.g9911.13070624 · Cited by 9
Hops: Fine-grained heterogeneous sensing, efficient and fair Deep Learning cluster scheduling system
Wang, Q., Wang, F., & Zheng, X. (2024) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3698038.3698515 · Cited by 8
Janjani, H., Agarwal, T., Gopinath, M. P., Sharma, V., & Raja, S. P. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3508089 · Cited by 6
“Mao et al. [17] introduce DeepRM, which represents new thinking about how to take advantage of DRL for resource management, concretely for the task allocation problem with variations of diverse resource requests.”
Intelligent energy pairing scheduler (InEPS) for heterogeneous HPC clusters
López, M., Stafford, E., & Bosque, J. L. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-024-06907-y · Cited by 6
“DeepRM [9] was one of the first initiatives to apply reinforcement learning to the scheduling problem as an alternative to heuristic approaches.”
Thamsen, L., Beilharz, J., Tran, V. T., Nedelkoski, S., & Kao, O. (2020) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.5823 · Cited by 6
A Topology-Aware Performance Prediction Model for Distributed Deep Learning on GPU Clusters
Lin, Z., Chen, X., Zhao, H., Luan, Y., Yang, Z. C., & Dai, Y. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378252 · Cited by 6
“Therefore, we implement a simulator to simulate jobs submitted, profiled, scheduled and running in a GPU cluster following [13].”
MethodD2A: Operating a Service Function Chain Platform With Data-Driven Scheduling Policies
Kramer, P., Diederich, P., Kramer, C., Pries, R., Kellerer, W., & Blenk, A. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3177694 · Cited by 6
“NCO: RL and NCO can learn the admission of jobs [64], and the assignment of jobs to compute resources [38], [39], [40], [65].”
BackgroundDeep Reinforcement Learning for Autonomous Robotic Tensegrity (ART)
Hosmer, T., & Tigas, P. (2019) · ACADIA quarterly · doi:10.52842/conf.acadia.2019.016 · Cited by 6
Archana, R., & Pradeep Mohan Kumar, K. (2023) · Sustainable Computing Informatics and Systems · doi:10.1016/j.suscom.2023.100887 · Cited by 6
DECCo-A Dynamic Task Scheduling Framework for Heterogeneous Drone Edge Cluster
Zhang, Z., Wu, D., Zhang, F., & Wang, R. (2023) · Drones · doi:10.3390/drones7080513 · Cited by 6
Gong, Y., Li, B., Liang, B., & Zhan, Z. (2019) · International Workshop on Quality of Service · doi:10.1145/3326285.3329065 · Cited by 6
“policy gradient method to compute the time for jobs to be scheduled in a cluster [6].”
MethodIsmael: Using Machine Learning to Predict Acceptance of Virtual Clusters in Data Centers
Zerwas, J., Kalmbach, P., Schmid, S., & Blenk, A. (2019) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2019.2927291 · Cited by 6
“In the context of resource management of cloud networks, DeepRM [32] is a system that learns to schedule data center tasks from experience.”
BackgroundEnergy-aware Scheduling of Jobs in Heterogeneous Cluster Systems Using Deep Reinforcement Learning
Esmaili, A., & Pedram, M. (2020) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed48828.2020.9137025 · Cited by 6
“However, [4] does not consider heterogeneity of computing machines in terms of their energy profile in the cluster and thus does not examine energy awareness in its”
BackgroundMethodInfluentialSmart DAG Task Scheduling with Efficient Pruning-Based MCTS Method
Liu, K., Wu, Z. S., Wu, Q., & Cheng, Y. (2019) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-sustaincom-socialcom48970.2019.00058 · Cited by 8
“Reinforcement learning [9] can learn from past scheduling strategies and adjust current actions to optimize scheduling results.”
BackgroundAuto-scaling for a Streaming Architecture with Fuzzy Deep Reinforcement Learning
Nguyen-Doan, D., Zaharie, D., & Petcu, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-48340-1_37 · Cited by 6
Venkateswaran, P., Hsu, C.-H., Mehrotra, S., & Venkatasubramanian, N. (2020) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp50058.2020.00023 · Cited by 6
“4: Reinforcement Learning with DNN driven policy [13].”
Jointly-Learned State-Action Embedding for Efficient Reinforcement Learning
Pritz, P. J., Ma, L., & Leung, K. K. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482357 · Cited by 10
“For instance, resource management in computing clusters [8, 22], portfolio management [16], and recommender systems [19, 21] all suffer from extremely large state/action spaces, thus challenging to be tackled by RL.”
BackgroundSwapna, A. I., Rosa, R. V., Rothenberg, C. E., Pasquini, R., & Baliosián, J. (2020) · IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN) · doi:10.1109/nfv-sdn50289.2020.9289852 · Cited by 6
“The described model evaluation is performed using a reference environment for DNN-based RL experiments as proposed in [12].”
BackgroundMethodVeRLPy: Python Library for Verification of Digital Designs with Reinforcement Learning
Shibu, A. J., Sadhana, S., Shilpa, N., & Kumar, P. (2021) · International Conference on AI-ML Systems (AIMLSystems) · doi:10.1145/3486001.3486236 · Cited by 6
“For instance, RL has seen use in resource management [19, 20, 28], network routing [5, 7] and most recently in chip placement [21].”
GreenTE.ai: Power-Aware Traffic Engineering via Deep Reinforcement Learning
Pan, T., Peng, X., Shi, Q., Bian, Z., Lin, X., Song, E., Li, F., Xu, Y., & Huang, T. (2021) · International Workshop on Quality of Service · doi:10.1109/iwqos52092.2021.9521281 · Cited by 6
HR Management Big Data Mining Based on Computational Intelligence and Deep Learning
Gen-liang, Z., & Xue, Z. (2021) · International Journal of Antennas and Propagation · doi:10.1155/2021/1657236 · Cited by 6
“In the automatic evaluation, standard ROUGE metrics were used, including ROUGE-1, ROUGE-2, and ROUGE-L, which measure the statistics of unary word overlap, binary word overlap, and longest common subsequence (LCS) [31] in the comparison of real and automatically generated results, respectively.”
MethodInfluentialRetractedRuya: Memory-Aware Iterative Optimization of Cluster Configurations for Big Data Processing
Will, J., Thamsen, L., Bader, J., Scheinert, D., & Kao, O. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020295 · Cited by 6
Neural network agent playing spin Hamiltonian games on a quantum computer
Sotnikov, O. M., & Mazurenko, V. V. (2020) · Journal of Physics A Mathematical and Theoretical · doi:10.1088/1751-8121/ab73ad · Cited by 6
NEURAL NETWORK AGENT PLAYING SPIN HAMILTONIAN GAMES ON A QUANTUM COMPUTER
Sotnikov, O. M., & Mazurenko, V. V. (2020) · Electronic Archive of Ural Federal University (ELAR UrFU) · Cited by 6
Preemptive Mobile Computation Offloading With Hard Deadlines and Concurrent Local Execution
Teymoori, P., Hekmati, A., Todd, T. D., Zhao, D., & Karakostas, G. (2021) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2021.3061106 · Cited by 6
“, [23], [24] can be used to effectively learn parameters of the system (e.”
People-Centric Evolutionary System for Dynamic Production Scheduling
Nguyen, S., Zhang, M., Alahakoon, D., & Tan, K. C. (2020) · doi:10.26686/wgtn.13158308.v1 · Cited by 6
Wave: Offloading Resource Management to SmartNIC Cores
Humphries, J. T., Natu, N., Kaffes, K., Novaković, S., Turner, P., Levy, H. M., Culler, D., & Kozyrakis, C. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3676642.3736113 · Cited by 5
Deep reinforcement learning for job scheduling on load-aware heterogeneous cluster
Yao, Z., Ding, L., Zhang, H., Li, H., Chen, L., & Li, Z. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01325-0 · Cited by 6
“Mao et al. proposed a RL based scheduling strategy named DeepRM, which adopt deep reinforcement learning to schedule task on a resource pool [21], the essence of DeepRM is to determine the order of task execution with an intelligent agent.”
Exploiting Wide-Area Resource Elasticity With Fine-Grained Orchestration for Serverless Analytics
Yue, X., Yang, S., Zhu, L., Trajanovski, S., Li, F., & Fu, X. (2024) · IEEE Transactions on Networking · doi:10.1109/tnet.2024.3486788 · Cited by 7
Launchpad: Learning to Schedule Using Offline and Online RL Methods
Venkataswamy, V., Grigsby, J., Grimshaw, A. S., & Qi, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-74430-3_4 · Cited by 5
Li, Z., Gong, J., Xiong, X., & Wang, D. (2025) · IEEE Access · doi:10.1109/access.2024.3524636 · Cited by 5
Learning Traffic Encoding Matrices for Delay-Aware Traffic Engineering in SD-WANs
Ghaderi, M., Liu, W., Xiao, S., & Li, F.-L. (2022) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms54207.2022.9789845 · Cited by 6
“Recent works on the application of machine learning in network control can be found in [48]– [50].”
BackgroundDeep Reinforcement Agent for Failure-aware Job scheduling in High-Performance Computing
Yang, K., Cao, R., Zhou, Y., Zhang, J., Shao, E., & Tan, G. (2021) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads53394.2021.00061 · Cited by 5
“By modeling abundant information of the state of jobs and HPC system and automatically learning to maximize cumulative reward through interaction with the HPC environment, RL-based job scheduler obtains more satisfied scheduling performance than schedulers based on ad-hoc heuristics [3]–[5].”
BackgroundMethodInfluentialResource Management in Cloud Based on Deep Reinforcement Learning
Liu, C., Li, W., Wan, J., Li, L., Ma, Z., & Wang, Y. (2022) · International Conference on Computer Communication and the Internet (ICCCI) · doi:10.1109/iccci55554.2022.9850259 · Cited by 5
CAMDNN: Content-Aware Mapping of a Network of Deep Neural Networks on Edge MPSoCs
Heidari, S., Ghasemi, M., Kim, Y. G., Wu, C.-J., & Vrudhula, S. B. K. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3207137 · Cited by 5
“However, lack of simulation environment makes RL-based solutions not practical for runtime scheduling on resource-constrained devices due to slow convergence [26], [27].”
BackgroundShiomoto, K., Otoshi, T., & Murata, M. (2023) · Annals of Telecommunications · doi:10.1007/s12243-022-00940-9 · Cited by 5
Metadata
- Authors
- Mao, H., Alizadeh, M., Menache, I., & Kandula, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 15th ACM Workshop on Hot Topics in Networks
- Publisher
- ACM
- Published
- 2016-11-09
- Pages
- 50-56
- OpenAlex
- W2546571074
- Cited by
- 1,353
- References
- 41
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 98 · 2023: 140 · 2022: 151 · 2021: 191 · 2020: 180 · 2019: 155 · 2018: 62 · 2017: 24
- FWCI
- 68.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Hot Topics in Networks (HotNets) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- resource management, deep reinforcement learning, task scheduling, resource allocation, workload modeling, online decision making, experiential learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
