The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Resource Management with Deep Reinforcement Learning
Abstract
Resource management problems in systems and networking often manifest as difficult online decision making tasks where appropriate solutions depend on understanding the workload and environment. Inspired by recent advances in deep reinforcement learning for AI problems, we consider building systems that learn to manage resources directly from experience. We present DeepRM, an example solution that translates the problem of packing tasks with multiple resource demands into a learning problem. Our initial results show that DeepRM performs comparably to state-of-the-art heuristics, adapts to different conditions, converges quickly, and learns strategies that are sensible in hindsight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, RL combined with deep-learning has been shown as a promising alternative [294] that can be leveraged to tackle various resource management problems in practical settings.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [294]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., et al. (2010) · Communications of the ACM · doi:10.1145/1721654.1721672 · Cited by 9,464
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Research works printed but not linked: 20
- Agarwal S., 2012, NSDI, 281
- Ananthanarayanan G., 2010, In OSDI, number, 1, 24
- Bertsekas D. P., 1995, Decision and Control
- Chilimbi T., 2014, OSDI, 571
- Dong M., 2015, NSDI, 395
- Ghodsi A., 2011, NSDI'11, 323
- Heller B., 2010, NSDI'10
- Junchen J., 2015, SIGCOMM '15
- Kaelbling L. P., 1996, Journal of artificial intelligence research
- Mahadevan S., 1998, FLAIRS Conference
- Mnih V., 2016, CoRR
- Mnih V., 2013, CoRR
- Mnih V., 2015, Nature
- J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. J. Schulman S. Levine P. Moritz M. I. Jordan and P. Abbeel. Trust region policy optimization. CoRR abs/1502.05477 2015. · arXiv
- 10.5555/551283 · DOI
- Sutton R. S., 1999, NIPS
- Winstein K., 2013, NSDI, 459
- Yi S., 2016, SIGCOMM
- Yin X., 2016, SIGCOMM '16
- Zhang W., 1995, IJCAI. Citeseer
Cited by
Sorted by citations
Items 501–600 of 1,000
Köroğlu, Y., & Sen, A. (2019) · arXiv.org · doi:10.48550/arxiv.1911.05403 · Cited by 15
“…machine learning methodology that has driven impressive advances in artificial intelligence in recent years, exceeding human performance in domains ranging from resource management [33], traffic light control [5], playing sophisticated games such as chess [40] and atari [37], to chemistry [49].”
BackgroundLearning Long-Term Reward Redistribution via Randomized Return DecompositionPreprint
Ren, Z., Guo, R., Zhou, Y., & Peng, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13485 · Cited by 12
“Scaling reinforcement learning (RL) algorithms to practical applications has become the focus of numerous recent studies, including resource management (Mao et al., 2016), industrial control (Hein et al.”
BackgroundZhang, C., Wu, F., Wang, Z., Wang, X., Ma, H., & Liu, Y. (2025) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2025.3620615 · Cited by 11
Management of resource sharing in emergency response using data-driven analytics
Zhang, J., Tutun, S., Anvaryazdi, S. F., Amini, M., Sundaramoorthi, D., & Sundaramoorthi, H. (2023) · Annals of Operations Research · doi:10.1007/s10479-023-05702-x · Cited by 11
“…such as cellular networks(ElHalawanyetal.,2020),radionetworks(Kwasinskietal.,2020),communications (Yeetal.,2019;Alelaiwi,2020),computingnetworks(Yangetal.,2018),businessprocess management(Huangetal.,2011),computerresourcemanagement(Maoetal.,2016),and cloud-datacenters(Chenetal.,2019;Duetal.,2019).”
Venus: Enhancing QoE of Crowdsourced Live Video Streaming by Exploiting Multiflow Viewer Assistance
Zhang, H., An, C., Zhou, A., Zhu, Y., Sun, W., Lu, Y., Chen, J., Liu, L., Ma, H., & Fei, A. (2024) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3636534.3649354 · Cited by 11
“…the RL algorithms presented in QAR [40] and DROM[54] primarily focused on optimizing SDN routing, i.e. , selecting the most efficient transmission paths among various candidates within a single flow context; DeepRM [42] is concerned with the allocation and scheduling of cluster computing resources.”
MethodCloud continuum testbeds and next-generation ICTs: Trends, challenges, and perspectives
Casino, F., López-Iturri, P., & Patsakis, C. (2024) · Computer Science Review · doi:10.1016/j.cosrev.2024.100696 · Cited by 18
Learning-based Adaptive Data Placement for Low Latency in Data Center Networks
Liu, K., Wang, J., Liao, Z., Yu, B., & Pan, J. (2018) · Conference on Local Computer Networks · doi:10.1109/lcn.2018.8638050 · Cited by 11
“[17] investigated the resource management problems with policy gradients.”
BackgroundA Deep Reinforcement Learning Approach to Multi-Component Job Scheduling in Edge Computing
Cao, Z. G., Zhang, H., Cao, Y., & Liu, B. (2019) · International Conference on Mobile Ad-Hoc and Sensor Networks (MSN) · doi:10.1109/msn48538.2019.00018 · Cited by 11
“Mao et al. [16], [17] leverage DRL techniques to solve a multi-resource cluster scheduling problem, and they find that their RL agents sometimes can perform better than heuristics.”
BackgroundMethodInfluentialConsolidation of Services in Mobile Edge Clouds using a Learning-based Framework
Ghafouri, S., Saleh-Bigdeli, A. A., & Doyle, J. C. (2020) · IEEE World Congress on Services (SERVICES) · doi:10.1109/services48979.2020.00037 · Cited by 11
“However, they lack generalization and are not robust to noisy online environments [32].”
BackgroundQuyen, N. H., Duy, P. T., Vy, N. C., Hien, D. T. T., & Pham, V.-H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21280-2_20 · Cited by 11
Wang, Y., Kong, L., Zhu, M., Gu, J., Liu, X., Tong, W., Hua, B., Lei, M., Cai, Y., & Zhang, J. (2023) · Journal of Lightwave Technology · doi:10.1109/jlt.2023.3346455 · Cited by 11
“This trick of reducing the action space is also adopted in [8], facilitating a more manageable and efficient learning process for the DRL agent.”
BackgroundMethodInfluentialDeepAalo: Auto-adjusting Demotion Thresholds for Information-agnostic Coflow Scheduling
Wang, S., Wang, S., Huo, R., Huang, T., Liu, J., & Liu, Y. (2020) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infocomwkshps50562.2020.9162722 · Cited by 11
“For example, DeepRM [24] translates the problem of packing tasks with multiple resource demands into a RL problem.”
BackgroundZubow, A., Rösler, S., Gawłowicz, P., & Dressler, F. (2021) · IEEE Workshop on Mobile Computing Systems and Applications (WMCSA) · doi:10.1145/3446382.3448358 · Cited by 11
“Here, techniques like Reinforcement Learning (RL) are becoming very popular [7, 17].”
MethodDeep Reinforcement Learning in M2M Communication for Resource Scheduling
Che, Y., Lin, F., & Liu, J. (2021) · World Conference on Computing and Communication Technologies (WCCCT) · doi:10.1109/wccct52091.2021.00025 · Cited by 11
“In [3], the proposed Deep RM method expresses the state of the packaged tasks with multiple resource demands as an image firstly, then uses the deep reinforcement learning method to process the image, make online decisions, and allocate resources to different machines.”
MethodNon-Markovian Reinforcement Learning using Fractional Dynamics
Gupta, G., Yin, C., Deshmukh, J. V., & Bogdan, P. (2021) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc45484.2021.9683076 · Cited by 11
“The RL paradigm now contains a number of different kinds of algorithms, and has been successfully used across a diverse set of applications including autonomous vehicles, resource management in computer clusters [2], traffic light control [3], web system configuration [4], and personalized recommendations [5].”
Ortíz, F., Tarchi, D., Rodríguez‐Osorio, R. M., Vanelli‐Coralli, A., Salas‐Natera, M. A., & Ayala, S. L. (2021) · International Journal of Satellite Communications and Networking · doi:10.1002/sat.1422 · Cited by 11
“This section presents the flexible payload architecture considered and mathematically defines the problem to be solved for the DRM.”
BackgroundInfluentialRevisiting the Classics: Online RL in the Programmable Dataplane
Simpson, K. A., & Pezaros, D. P. (2022) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms54207.2022.9789930 · Cited by 11
“Adaptive techniques such as reinforcement learning (RL) have led the charge in data-driven networking, enhancing automatic traffic optimisation [5, 27], congestion control [40], adaptive routing [12, 48], resource management [26], and packet classification [22].”
BackgroundVerifying Generalization in Deep Learning
Amir, G., Maayan, O., Zelazny, T., Katz, G., & Schapira, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37703-7_21 · Cited by 11
Deb, S., Ge, Z., Isukapalli, S., Puthenpura, S. C., Venkataraman, S., Yan, H., & Yates, J. M. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098157 · Cited by 11
“In recent years, the broad scope of designing data driven learning systems for network management and control has received aention: [13] develops a system for automatically learning control decisions for resource management in data centers, [3] proposes a system for proactively predicting disk failures, [17] designs…”
BackgroundA New Approach for Resource Scheduling with Deep Reinforcement LearningPreprint
Ye, Y., Ren, X., Wang, J., Xu, L., Guo, W., Huang, W., & Tian, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08122 · Cited by 11
“We use the workload setting as in [5].”
MethodInfluentialDemocratizing Optimization with Generative AIPreprint
Simchi‐Levi, D., Dai, T., Menache, I., & Wu, M. X. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5511218 · Cited by 11
Self-citationWang, Y., Chen, J., Wu, Z., Chen, P., Li, X., & Hao, J. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.10.015 · Cited by 10
Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms
Reidys, B., Zardoshti, P., Goiri, Í., Irvene, C., Berger, D. S., Ma, H., Arya, K., Cortez, E., Stark, T., Bak, E., et al. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3669940.3707226 · Cited by 10
“While more sophisticated learning-based approaches could offer marginal improvements, their decisions are often harder to interpret, making them impractical [28, 59, 60].”
Du, H., & Zhang, S. (2020) · IEEE Access · doi:10.1109/access.2020.2982320 · Cited by 10
“An example of a baseline is a ‘‘time-based’’ baseline [20], [21], which sets bt to the cumulative reward from time t onwards, averaged over all training episodes.”
BackgroundDynamic Subcarrier Assignment in OFDMA-PONs Based on Deep Reinforcement Learning
Zhu, M., Gu, J., Chen, B., & Gu, P. (2022) · IEEE photonics journal · doi:10.1109/jphot.2022.3148259 · Cited by 10
Understanding via Exploration: Discovery of Interpretable Features With Deep Reinforcement Learning
Wei, J., Qiu, Z., Wang, F., Lin, W., Gui, N., & Gui, W. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3184956 · Cited by 10
“…interpretability for DRL, some researchers turn to integrating prior knowledge of the target object with a given problem-specific DNN. DeepRM [38], a resource management algo-rithm based on DRL, provides a more efficient environment abstraction by only using images of the first M jobs and the…”
MethodTask Allocation with Load Management in Multi-Agent Teams
Wu, H., Ghadami, A., Bayrak, A. E., Smereka, J. M., & Epureanu, B. I. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811374 · Cited by 10
Predicting job finish time based on parameter features and running logs in supercomputing system
Wang, Q., Zhang, H., Li, J., Shen, Y., & Liu, X. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04582-5 · Cited by 10
“In order to verify the influence of RunningNet on the job scheduling algorithm, we modify the scheduler of DeepRM[8] and introduce the prediction error of the job into the scheduler.”
BackgroundMethodInfluentialResource Management in Cloud Computing Using Deep Reinforcement Learning: A Survey
Feng, Y., & Liu, F. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-7652-0_56 · Cited by 10
Mohamed, M. M., & Tsuji, T. (2023) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.23768 · Cited by 11
Effective Scheduling Function Design in SDN Through Deep Reinforcement Learning
Huang, V., Chen, G., & Fu, Q. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761938 · Cited by 9
“Recently, reinforcement learning (RL) has been successfully applied to various resource management problems [12], [21] and is considered to be a powerful paradigm for designing SFs with several key advantages.”
BackgroundDeep Neural Network Based Cell Sleeping Control and Beamforming Optimization in Cloud-RAN
Du, G., Wang, L., Liao, Q., & Hu, H. (2019) · IEEE Vehicular Technology Conference · doi:10.1109/vtcfall.2019.8891410 · Cited by 10
“It is noteworthy that DNN has also been applied in wireless communication [7]–[9].”
BackgroundRL-Based Radio Resource Slicing Strategy for Software-Defined Satellite Networks
Deng, Z., Du, Q. S., Li, N., & Zhang, Y. (2019) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct46805.2019.8947198 · Cited by 10
“Thirdly, if system wants to optimize some indicators which are difficult to model [6], current schemes can’t give a suitable resource allocation plan.”
BackgroundFormula-E race strategy development using distributed policy gradient reinforcement learning
Liu, X., Fotouhi, A., & Auger, D. J. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106781 · Cited by 10
Fundy: A Scalable and Extensible Resource Manager for Cloud Resources
Ke, X., Guo, C., Ji, S., Bergsma, S., Hu, Z., & Guo, L. (2021) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud53861.2021.00070 · Cited by 10
“Previous researchers have represented hosts and requests with 2D matrices, but the proposal is to train scheduling parameters using deep reinforcement learning rather than massively parallelizing scheduling as Fundy does [18].”
BackgroundGet Your Memory Right: The Crispy Resource Allocation Assistant for Large-Scale Data Processing
Will, J., Thamsen, L., Bader, J., Scheinert, D., & Kao, O. (2022) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e55432.2022.00014 · Cited by 10
Performance and Cost-Aware Task Scheduling via Deep Reinforcement Learning in Cloud Environment
Zhao, Z., Shi, X., & Shang, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20984-0_43 · Cited by 10
A Survey of Edge Computing Resource Allocation and Task Scheduling Optimization
Xu, X., Ding, H., Wang, J., & Hua, L. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-4390-2_11 · Cited by 10
DDPG-Based Radio Resource Management for User Interactive Mobile Edge Networks
Chen, P., Chen, Y., Huang, W., Huang, C., & Tirkkonen, O. (2020) · Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit) · doi:10.1109/6gsummit49458.2020.9083926 · Cited by 10
“Since it is challenging to quantify advantages arising from proactive transmissions, we adopt negative rewards [13] to punish cases falling behind.”
BackgroundXia, C., Wang, R., Jin, X. H., Xu, C., Li, D., & Zeng, P. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3367350 · Cited by 9
Reinforcement learning-based autonomous attacker to uncover computer network vulnerabilities
Ahmed, A. M., Nguyen, T. T., Abdelrazek, M., & Aryal, S. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09668-0 · Cited by 11
Dynamic Flow Scheduling for DNN Training Workloads in Data Centers
Zhao, X., Wu, C., & Zhu, X. (2024) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2024.3450670 · Cited by 9
Resource Allocation with Graph Neural Networks-Multi Agent Reinforcement Learning for 6G HetNets
Agila, R., Estrada, R. L., & Rohoden, K. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.08.006 · Cited by 9
Wang, M. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.12.092 · Cited by 9
Model reusability in Reinforcement Learning
Nikookar, S., Nia, S. N., Roy, S. B., Amer-Yahia, S., & Omidvar-Tehrani, B. (2025) · The VLDB Journal · doi:10.1007/s00778-025-00920-0 · Cited by 9
Intranet User-Level Security Traffic Management with Deep Reinforcement Learning
Jin, Q., & Wang, L. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852447 · Cited by 11
“[14] regard job scheduling problems as packing tasks with multiple resource demands, translate the job demands to the collection of images as an input of training algorithm and adopt the standard policy gradient algorithm which assumes a discrete state and action space.”
Optimizing Resource Allocation for Data-Parallel Jobs Via GCN-Based Prediction
Hu, Z., Li, D., Zhang, D., Zhang, Y., & Peng, B. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3055019 · Cited by 9
“For the scheduling problem of packing tasks with multiple resource demands, a deep reinforcement learning method is presented in [19], and results show that it achieves higher resource utilization and less completion time.”
MethodAn SMDP approach for Reinforcement Learning in HPC cluster schedulers
Cunha, R. L. F., & Chaimowicz, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.09.025 · Cited by 9
Review on Reinforcement Learning in CartPole Game
Mothanna, Y. M., & Hewahi, N. M. (2022) · International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT) · doi:10.1109/3ict56508.2022.9990767 · Cited by 9
“Alizadeh Kober et al. [11] applied state-of-the-art deep reinforcement learning.”
BackgroundMulti-agent Reinforcement Learning-based Adaptive Heterogeneous DAG Scheduling
Zhadan, A., Allahverdyan, A., Kondratov, I. V., Mikheev, V., Petrosian, O., Romanovskii, A. B., & Kharin, V. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3610300 · Cited by 9
“Other works [16, 35] argue that, speciically”
He, R., Ren, B., Xie, J., Guo, D., Zhou, Y., Zhao, L., & Li, Y. (2023) · Computer Networks · doi:10.1016/j.comnet.2023.109985 · Cited by 9
Imitation learning enabled fast and adaptive task scheduling in cloud
Kang, K., Ding, D., Xie, H., Zhao, L., Li, Y., & Xie, Y. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.01.002 · Cited by 10
Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning
Jeong, S., Yoo, G., Yoo, M., Yeom, I., & Woo, H. (2019) · Sensors · doi:10.3390/s19204410 · Cited by 9
“, cloud task scheduling [11,12], cluster resource management [13,14], network traffic scheduling [15], and database query processing [16].”
BackgroundMethodDeep Reinforcement Learning Based Train Driving Optimization
Huang, J., Zhang, E., Zhang, J., Huang, S., & Zhong, Z. (2019) · Chinese Automation Congress · doi:10.1109/cac48633.2019.8996988 · Cited by 9
“On the other hand, in the field of computer systems, [10] applied DRL models to implement resource management in systems and networks, and utilized policy gradients to deal with multi-resource cluster scheduling problems, and process online dynamic jobs online to optimize average work speed reduction, completion time…”
BackgroundReinforcing Cloud Environments via Index Policy for Bursty Workloads
Balasubramanian, V., Aloqaily, M., Tunde-Onadele, O., Yang, Z., & Martin Reisslein, B. (2020) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms47738.2020.9110417 · Cited by 9
“Mao et al. in [13] have proposed a deepRM model that performs comparably to our study, however, Mao et al. do not consider a cloud data-center environment that is focus of our framework.”
ResultGeetha, R., & Parthasarathy, V. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02334-y · Cited by 9
“Reinforcement learning was proposed for undergoing smart resource allocation for applications like heterogeneous cloud computing (Mao et al. 2016).”
RetractedCooperative channel assignment for VANETs based on multiagent reinforcement learning
Wang, Y., Zheng, K., Tian, D., Duan, X., & Zhou, J. (2020) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900308 · Cited by 9
“…great prospects in sequential decision-making problems (Mnih et al., 2013, 2015; Li L et al., 2016; Silver et al., 2017), and a variety of RL-based solutions have been applied in dynamic resource management and other domains (Mao et al., 2016; He et al., 2017; Liu N et al., 2017; Xu et al., 2017).”
BackgroundLiu, H., Ma, Y., Chen, P., Xia, Y., Ma, Y., Zheng, W., & Li, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59618-7_13 · Cited by 9
Geometric deep reinforcement learning for dynamic DAG scheduling
Grinsztajn, N., Beaumont, O., Jeannot, E., & Preux, P. (2020) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci47803.2020.9308278 · Cited by 9
“Most of them [18], [21] consider tasks arriving sequentially and randomly, without a DAG structure.”
BackgroundScheduling the NASA Deep Space Network with Deep Reinforcement Learning
Goh, E. Y., Venkataram, H. S., Hoffmann, M., Johnston, M. D., & Wilson, B. D. (2021) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero50100.2021.9438519 · Cited by 10
“Furthermore, an RL agent trained on REINFORCE — another policy gradient algorithm similar to PPO — was shown to perform similarly and sometimes better than existing heuristics-based approaches for scheduling multi-resource clusters [25].”
MethodHecate: AI-driven WAN Traffic Engineering for Science
Kiran, M., Campbell, S. T., & Buraglio, N. (2022) · doi:10.1109/indis56561.2022.00011 · Cited by 9
“Similar to [25], we normalize this to prevent skewing results for longer flows.”
ResultLeveraging Reinforcement Learning for Task Resource Allocation in Scientific Workflows
Bader, J., Zunker, N., Becker, S., & Kao, O. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020688 · Cited by 9
“Mao et al. propose DeepRM [19], a resource manager using deep reinforcement learning.”
BackgroundRequest Dispatching Over Distributed SDN Control Plane: A Multiagent Approach
Huang, V., Chen, G., Zuo, X., Zomaya, A. Y., Sohrabi, N., Tari, Z., & Fu, Q. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3266448 · Cited by 9
“Recently, machine learning has been successfully applied to various resource management problems [16]–[20].”
InfluentialAn Initial Investigation of Protocol Customization
Hong, K., Chen, Q. A., & Mao, Z. M. (2017) · doi:10.1145/3141235.3141236 · Cited by 9
“, deep RL), has shown unique advantages in adapting to real-world dynamics for decision making in network systems [26, 47, 76].”
BackgroundCao, Y., Zhao, Y., Li, J., Lin, R., Zhang, J., & Chen, J. (2019) · Optical Fiber Communication Conference · doi:10.1364/ofc.2019.m2a.7 · Cited by 9
“Such a state is an input to a neural network (NN), which is used to generate the policy (including {state, action} pairs) [6].”
BackgroundJob placement using reinforcement learning in GPU virtualization environment
Oh, J., & Kim, Y. (2020) · Cluster Computing · doi:10.1007/s10586-019-03044-7 · Cited by 9
“Mao [39] proposed a resource management technology based on the Deep Q-Networks (DQN) reinforcement learning.”
MethodA Simple Reward-free Approach to Constrained Reinforcement LearningPreprint
Miryoosefi, S., & Jin, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05216 · Cited by 9
“, 2015), scheduling of resources on a computer cluster (Mao et al., 2016), imitation learning (Syed & Schapire, 2007; Ziebart et al.”
BackgroundInfluentialA Simulator for Intelligent Workload Managers in Heterogeneous Clusters
Herrera, A., Ibáñez, M., Stafford, E., & Bosque, J. L. (2021) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid51090.2021.00029 · Cited by 8
“Most recent solutions leverage deep artificial neural networks for learning the optimal scheduling policy, such as DeepRM [15] and Wrangler [30].”
BackgroundMethodInfluentialAn Advantage Actor-Critic Deep Reinforcement Learning Method for Power Management in HPC Systems
Khasyah, F. R., Santiyuda, G., Kaunang, G., Makhrus, F., Amrizal, M. A., & Takizawa, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-29927-8_8 · Cited by 8
HiVAT: Improving QoE for Hybrid Video Streaming Service With Adaptive Transcoding
Zhu, Y., Huang, Y., Qiao, X., Tang, J., & Su, X. J. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3399398 · Cited by 8
Kiaee, F., & Arianyan, E. (2024) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2024.104049 · Cited by 8
Xiao, Y., Zhai, Z., Yu, S., Xu, Z., Li, L., Zhang, F., Cao, L., & Chen, X. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3509440 · Cited by 8
Two-Stage Learning Approach for Semantic-Aware Task Scheduling in Container-Based Clouds
Zhu, L., Huang, K., Hu, Y., & Wang, Y. (2024) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2024.3520101 · Cited by 8
“Inspired by DeepRM [17], which utilizes images to represent the resource state of a single-host system, we propose a novel approach that employs container-based cloud state heatmaps, including a resource metric heatmap (RM-CCSH) and a task metric heatmap (TM-CCSH), to characterize the scheduling…”
InfluentialCaravan: Practical Online Learning of In-Network ML Models with Labeling Agents
Zhang, Q., Imran, A., Bardhi, E., Swamy, T., Zhang, N., Shahbaz, M., & Olukotun, K. (2024) · Workshop on Practical Adoption Challenges of ML for Systems (PACMI) · doi:10.1145/3704742.3704964 · Cited by 9
“In academia, several proposals make a case for using ML to improve systems security through anomaly and intrusion detection [12, 19, 42] and to optimize systems performance through inference, diagnosis, and forecasting of systems’ behavior [18, 24, 26, 27].”
Control System for the Navigation of the Agricultural Robots: A Review
Polania, E. C. M., Fernandes, H. R., Albiero, D., & Garcia, A. (2026) · Journal of Field Robotics · doi:10.1002/rob.70199 · Cited by 8
Scheduling Out-of-Coverage Vehicular Communications Using Reinforcement Learning
Şahin, T., Khalili, R., Boban, M., & Adam Wolisz, A. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3186910 · Cited by 8
“We are motivated to use an RL-based approach due to its successful applications for resource allocation problems in general [12], and for vehicular communications in particular [13].”
MethodBigMEC: Scalable Service Migration for Mobile Edge Computing
Brandherm, F., Gedeon, J., Abboud, O., & Mühlhäuser, M. (2022) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec54971.2022.00018 · Cited by 8
“Therefore, similar to related fields, such as job scheduling for the cloud [16], [17], reinforcement learning has recently been embraced for its ability to solve MDPs in a model-free way, adapting to client mobility and other network events while treating them as a black box.”
BackgroundYang, G., Si, P., Jin, K., Sun, T., & Wu, W. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3248651 · Cited by 8
“4 in [37] and [39], Δθ can be calculated as”
Mohamed, N. B., Hassan, M. Z., & Kaddoum, G. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3263756 · Cited by 8
Fast DRL-based scheduler configuration tuning for reducing tail latency in edge-cloud jobs
Wen, S., Han, R., Liu, C. H., & Chen, L. Y. (2023) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-023-00465-z · Cited by 8
“Recently, deep reinforcement learning (DRL), which is an important extension of the traditional reinforcement learning (RL) method, has been applied to various sophisticated online optimization problems with large solution spaces [40, 44, 63].”
BackgroundMethodLearning Scheduling Policies for Co-Located Workloads in Cloud Datacenters
Li, J., Xiao, D., Yao, J., Long, Y., & Wu, W. (2023) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2023.3319383 · Cited by 8
Papadis, N., & Tassiulas, L. (2023) · Lecture notes in operations research · doi:10.1007/978-3-031-48731-6_1 · Cited by 8
Fair channel allocation in IEEE 802.11p for high throughput and low-latency
Hota, L., Sahoo, B., & Kumar, A. (2025) · Physical Communication · doi:10.1016/j.phycom.2025.102683 · Cited by 8
User-profile-based analytics for detecting cloud security breaches
Tiwari, T., Türk, A., Oprea, A., Olcoz, K., & Coskun, A. K. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258494 · Cited by 8
“Even though machine learning and statistical techniques have been applied before in the context of cloud computing to detect performance anomalies [16], [17], [18], optimize resource allocation [19], and reduce energy usage [20], these approaches have not, to the best of our knowledge, been implemented to defend again…”
BackgroundSelf-programming networks: Architecture and algorithms
Geng, Y., Liu, S., Wang, F., Yin, Z., Prabhakar, B., & Rosenblum, M. (2017) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2017.8262813 · Cited by 9
“The recent work of Mao et al on using Reinforcement Learning (RL) for resource management [15] and video streaming [16] motivate us to consider artificial intelligence techniques for learning and controlling SPNs.”
Dynamic and Decentralized Global Analytics via Machine Learning
Wang, H., Niu, D., & Li, B. (2018) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3267809.3267812 · Cited by 8
“A few efforts have been made to perform resource management with machine learning techniques [14, 27], workload classification [33], cluster configuration [2] and database management system tuning [32].”
MethodTurbo: Dynamic and Decentralized Global Analytics via Machine Learning
Wang, H., Niu, D., & Li, B. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2964667 · Cited by 8
“A few efforts have been made to perform resource management with machine learning techniques [36], [37], workload classification [26], cluster configuration [38] and database management system tuning [39].”
MethodTwo Decades of AI4NETS - AI/ML for Data Networks: Challenges & Research Directions
Casas, P. (2020) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms47738.2020.9110266 · Cited by 12
“with initial results on deep learning [35], [41], [44], transfer learning [50], XAI for network security [21], deep reinforcement learning for network management [31], [33], [34], etc.”
BackgroundOkanlawon, A., Yang, H., Bose, A. J., Hsu, W. W. Y., Andresen, D., & Tanash, M. (2020) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci51800.2020.00230 · Cited by 8
“Our interest in this work is born of recent growing interest in using machine learning to optimize HPC systems [3, 4].”
BackgroundAdversarial Deep Learning for Online Resource Allocation
Du, B., Huang, Z., & Wu, C. (2021) · ACM Transactions on Modeling and Performance Evaluation of Computing Systems · doi:10.1145/3494526 · Cited by 8
Reinforcement learning based monotonic policy for online resource allocation
Mishra, P., & Moustafa, A. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2021.09.023 · Cited by 8
Toward Building a Digital Twin of Job Scheduling and Power Management on an HPC System
Ohmura, T., Shimomura, Y., Egawa, R., & Takizawa, H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-22698-4_3 · Cited by 8
Resource scheduling techniques in cloud from a view of coordination: a holistic survey
Wang, Y., Yu, J., & Yu, Z. (2023) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2100298 · Cited by 8
“Mao et al. (2016) claimed that reinforcement learning (RL) is well-suited for modeling resource management systems and decision-making policies.”
BackgroundZhu, M., Wang, Y., Gu, J., Cai, X., Liu, X., Tong, W., Cai, Y., & Zhang, J. (2023) · Journal of Optical Communications and Networking · doi:10.1364/jocn.482813 · Cited by 8
A DRL-based online VM scheduler for cost optimization in cloud brokers
Li, X., Pan, L., & Liu, S. (2023) · World Wide Web · doi:10.1007/s11280-023-01145-3 · Cited by 8
“[11] is the most representative work on DRL-based resource management solutions.”
BackgroundMethodDeep Reinforcement Learning based Energy Scheduling for Edge Computing
Yang, Q., & Li, P. (2020) · IEEE International Conference on Smart Cloud (SmartCloud) · doi:10.1109/smartcloud49737.2020.00041 · Cited by 8
“In this subsection, to implement our simulation experiment, we first set the job to arrive online given to a Bernoulli process [13].”
BackgroundMethodInfluentialTowards a common environment for learning scheduling algorithms
Cunha, R. L. F., & Chaimowicz, L. (2020) · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots50786.2020.9285940 · Cited by 8
“In DeepRM [14] and in Algorithm 1, baselines are the total returns received from time step t averaged over all trajectories.”
BackgroundMethodResultInfluentialOne protocol to rule them All: Wireless network-on-chip using deep reinforcement learning
Jog, S., Liu, Z., Franques, A., Fernando, V., Abadal, S., Torrellas, J., & Hassanieh, H. (2021) · UPCommons institutional repository (Universitat Politècnica de Catalunya) · Cited by 8
Optimizing the resource usage of actor-based systems
Nguyen, H. T. T., Do, T. V., & Rotter, C. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103143 · Cited by 8
A Deep Reinforcement Learning Approach for Traffic Signal Control OptimizationPreprint
Li, Z., Xu, C., & Zhang, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.06115 · Cited by 15
“Deep RL has been prevailing in last several years with different applications, such as resources management in computer clusters (Mao et al. , 2016), games (Mnih et al. , 2013), natural language processing (NLP) (Young et al. , 2018), robotics (Kober, Bagnell and Peters, 2013), web system…”
Background
Metadata
- Authors
- Mao, H., Alizadeh, M., Menache, I., & Kandula, S.
- Affiliations
- Massachusetts Institute of Technology (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 15th ACM Workshop on Hot Topics in Networks
- Publisher
- ACM
- Published
- 2016-11-09
- Pages
- 50-56
- OpenAlex
- W2546571074
- Cited by
- 1,353
- References
- 41
- Citations by year
- 2026: 171 · 2025: 181 · 2024: 98 · 2023: 140 · 2022: 151 · 2021: 191 · 2020: 180 · 2019: 155 · 2018: 62 · 2017: 24
- FWCI
- 68.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Hot Topics in Networks (HotNets) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- resource management, deep reinforcement learning, task scheduling, resource allocation, workload modeling, online decision making, experiential learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
