A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average 2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Compared to DQL, the experiment results indicate that Double-DQL can save more energy and achieve higher training efficiency.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [257]
Citations
Cites
Tensor Decompositions and Applications
Kolda, T. G., & Bader, B. W. (2009) · SIAM Review · doi:10.1137/07070111x · Cited by 10,301 · cited 1×
“Furthermore, we will investigate the use of tensor decompositions and tensor-train networks [30], [31], [32], [33] to improve the training efficiency of the presented double deep Q-learning model in the future work.”
MethodEdge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733 · cited 3×
“Second, cloud computing cannot ensure the privacy protection sometimes in Internet of Things and cyber-physical-social systems [14].”
BackgroundResultSelf-citation(2011) · Cited by 6,993
MiBench: A free, commercially representative embedded benchmark suite
Guthaus, M. R., Ringenberg, J. S., Ernst, D. J., Austin, T. M., Mudge, T., & Brown, R. B. (2001) · doi:10.1109/wwc.2001.15 · Cited by 3,103
Oseledets, I. V. (2011) · SIAM Journal on Scientific Computing · doi:10.1137/090752286 · Cited by 2,636
MiBench: A free, commercially representative embedded benchmark suite
Guthaus, M. R., Ringenberg, J. S., Ernst, D., Austin, T. M., Mudge, T., & Brown, R. B. (2005) · Proceedings of the Fourth Annual IEEE International Workshop on Workload Characterization. WWC-4 (Cat. No.01EX538) · doi:10.1109/wwc.2001.990739 · Cited by 1,588 · cited 1×
“We use one MiBench as the experimental taskset which includes a set of commercially representative embedded programs [29].”
MethodA survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Self-citationReal-time dynamic voltage scaling for low-power embedded operating systems
Pillai, P., & Shin, K. G. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502044 · Cited by 1,138 · cited 2×
“Another two typical DVFS algorithms are CC and LA [19].”
BackgroundMethodPower-aware scheduling for periodic real-time tasks
Aydin, H., Melhem, R., Mosse, D., & Mejia-Alvarez, P. (2004) · IEEE Transactions on Computers · doi:10.1109/tc.2004.1275298 · Cited by 484
Zhou, Z., Dong, M., Ota, K., Wang, G., & Yang, L. T. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2015.2497712 · Cited by 272 · cited 1×
“Internet of Things integrate many physical and sensing devices including RFID (Radio Frequency Identification), camera and so on to the cyber space while cyber-physical-social systems combine the human behaviors and social relations to the Internet of Things [4], [5].”
BackgroundSelf-citation
Research works printed but not linked: 1
- huang, 2017, Hybrid DVFS scheduling for real-time systems based on reinforcement learning, Proc IEEE Int Conf Green Comput Commun Conf, 522
Cited by
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“computing resources of the edge device, in [257], DoubleDQL [89] is to determine the best Dynamic Voltage and Frequency Scaling (DVFS) algorithm.”
MethodA smart agriculture IoT system based on deep reinforcement learning
Bu, F., & Wang, X. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.04.041 · Cited by 342
Tuli, S., Ilager, S., Ramamohanarao, K., & Buyya, R. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3017079 · Cited by 257
“Recent works have explored different value-based RL techniques to optimize several aspects of Resource Management Systems (RMS) in distributed environments [16], [17], [18, 19].”
BackgroundMethodEnergy-efficient edge based real-time healthcare support system
(2020) · Advances in Computers · doi:10.1016/bs.adcom.2019.09.007 · Cited by 211
AL-Jumaili, A. H. A., Muniyandi, R. C., Hasan, M. K., Paw, J. K. S., & Singh, M. S. J. (2023) · Sensors · doi:10.3390/s23062952 · Cited by 234
A job scheduling algorithm for delay and performance optimization in fog computing
Jamil, B., Shojafar, M., Ahmed, I., Ullah, A., Munir, K., & Ijaz, H. (2019) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.5581 · Cited by 172
Liu, Z., Yao, C., Yu, H., & Wu, T. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.02.068 · Cited by 149
The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning
Yu, J., Su, Y., & Liao, Y. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.00063 · Cited by 152
“(6) THE double DQN (DDQN) algorithm estimates the maximum action in the target network through the network and uses this estimated action to select Q(s) in the target network (Zhang et al., 2018; Han et al., 2019).”
BackgroundTuli, S., Poojara, S. R., Srirama, S. N., Casale, G., & Jennings, N. R. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3087349 · Cited by 112
“[5], [33], [34], [35] X X Deep RL X X X”
A novel approach for IoT tasks offloading in edge-cloud environments
Almutairi, J., & Aldossary, M. (2021) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-021-00243-9 · Cited by 143
“The EdgeCloudSim has been used since it provides the vital functionality of Edge-Cloud environment such as support offloading and users mobility [41], [45], [59], [60].”
MethodRen, T., Niu, J., Dai, B., Liu, X., Hu, Z., Xu, M., & Guizani, M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3071531 · Cited by 123
Data fusion based coverage optimization in heterogeneous sensor networks: A survey
Deng, X., Jiang, Y., Yang, L. T., Lin, M., Yi, L., & Wang, M. (2018) · Information Fusion · doi:10.1016/j.inffus.2018.11.020 · Cited by 106
Self-citationDu, G., Zou, Y., Zhang, X., Guo, L., & Guo, N. (2021) · Energy · doi:10.1016/j.energy.2021.122523 · Cited by 105
Wang, H., Zhang, H., Liu, X., Long, K., & Nallanathan, A. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3230407 · Cited by 112
“The DDQN algorithm contains two key techniques, which are experience replay and fixed targets [31].”
MethodApplication placement in Fog computing with AI approach: Taxonomy and a state of the art survey
Nayeri, Z. M., Ghafarian, T., & Javadi, B. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103078 · Cited by 115
Jauro, F., Chiroma, H., Gital, A. Y., Almutairi, M. S., Abdulhamid, S. M., & Abawajy, J. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106582 · Cited by 115
Recent Advances in Collaborative Scheduling of Computing Tasks in an Edge Computing Paradigm
Chen, S., Li, Q., Zhou, M., & Abusorrah, A. M. (2021) · Sensors · doi:10.3390/s21030779 · Cited by 111
“(4) Find any-time task scheduling algorithms.”
A Survey on Energy Management for Mobile and IoT Devices
Pasricha, S., Ayoub, R. Z., Kishinevsky, M., Mandal, S. K., & Ogras, U. Y. (2020) · IEEE Design and Test · doi:10.1109/mdat.2020.2976669 · Cited by 87
“To address the limitations of the Q-table-based approach, deep Q-learning-based power management policies approximate Q-tables using a deep neural network and experience replay buffers [37], [38].”
MethodSecurity-Aware Dynamic Scheduling for Real-Time Optimization in Cloud-Based Industrial Applications
Meng, S., Huang, W., Yin, X., Khosravi, M. R., Li, Q., Wan, S., & Qi, L. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.2995348 · Cited by 82
“[22] presented an energy-aware scheduling approach based on”
BackgroundZeng, L., Liu, Q., Shen, S., & Liu, X. (2023) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010058 · Cited by 78
DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference
Zhang, Z., Zhao, Y., Li, H., Lin, C., & Liu, J. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3357218 · Cited by 99
“Prior work [10], [12], [14], [15], [32], [33], [34], [35] has proposed a series of deep reinforcement learning-based DVFS techniques to reduce energy consumption.”
BackgroundHu, Z., Niu, J., Ren, T., Dai, B., Li, Q., Xu, M., & Das, S. K. (2021) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3116280 · Cited by 75
Panda, S. K., Lin, M., & Zhou, T. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3153399 · Cited by 80
“Zhang et al. [34] proposed a DRL-based DVFS technique selection scheme for energy-efficient edge scheduling on edge devices.”
Self-citationShuai, B., Zhou, Q., Li, J., He, Y., Li, Z., Williams, H., Xu, H., & Shuai, S. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.114900 · Cited by 72
“mechanism can reduce the overestimation between the actual value and the approximation [45].”
BackgroundA Survey of Low-Energy Parallel Scheduling Algorithms
Xie, G., Xiao, X., Hao, P., Li, R., & Li, K. (2021) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2021.3057983 · Cited by 71
“However, most machine learning algorithms are designed for energy-effi-cient non-parallel scheduling algorithms [32], [81], [82], [83], [84], [85], [86], and only a few recent studies are designed for energy-efficient parallel scheduling algorithms [79], [80].”
AL-Jumaili, A. H. A., Muniyandi, R. C., Hasan, M. K., Singh, M. S. J., Paw, J. K. S., & Amir, M. (2023) · Energy Reports · doi:10.1016/j.egyr.2023.09.029 · Cited by 74
QoE-Driven Content-Centric Caching With Deep Reinforcement Learning in Edge-Enabled IoT
He, X., Wang, K., & Xu, W. (2019) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2019.2937608 · Cited by 65
“r Then, the improved DNN, based on fixed target network [20], experience replay [21], and adaptive learning rate [22] methods, is proposed to balance the Q-value accuracy and DRL-accelerated stability.”
MethodA Survey of Domain-Specific Architectures for Reinforcement Learning
Rothmann, M., & Porrmann, M. (2022) · IEEE Access · doi:10.1109/access.2022.3146518 · Cited by 69
“For example, RL has been used for scheduling in data centers [42] and for adaptive power management [12], [27], [44], [91].”
MethodMachine learning-based computation offloading in multi-access edge computing: A survey
Choudhury, A., Ghose, M., Islam, A., & Yogita, Y. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103090 · Cited by 72
A Deep Q-Learning Approach for Dynamic Management of Heterogeneous Processors
Gupta, U., Mandal, S. K., Mao, M., Chakrabarti, C., & Ogras, U. Y. (2019) · IEEE Computer Architecture Letters · doi:10.1109/lca.2019.2892151 · Cited by 62
Huang, H., Ye, Q., & Zhou, Y.-T. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3115054 · Cited by 65
“reality, and image processing, have gained enormous popularity [1].”
BackgroundFaashouse: Sustainable Serverless Edge Computing Through Energy-Aware Resource Scheduling
Aslanpour, M. S., Toosi, A. N., Cheema, M. A., & Chhetri, M. B. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3354296 · Cited by 70
“In[25],schedulingimprove-ments are practiced by energy-aware tuning of CPU frequency by machine learning models, which appear computation-hungry for the resource-constrained extreme edge, whereas our proposal solves the problem with a reasonable computation complexity.”
BackgroundA comprehensive review on internet of things task offloading in multi-access edge computing
Dayong, W., Bakar, K. A., Isyaku, B., Eisa, T. A. E., & Abdelmaboud, A. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e29916 · Cited by 65
A Momentum-Accelerated Hessian-Vector-Based Latent Factor Analysis Model
Li, W., Luo, X., Yuan, H., & Zhou, M. (2022) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2022.3177316 · Cited by 51
“[20], and optimization methods and their convergence analyses play a critical role in building a learning system for service-oriented applications [49], [50], [51].”
BackgroundYao, C., Wu, S., Liu, Z., & Li, P. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.11.011 · Cited by 50
Self-citationA survey on system level energy optimisation for MPSoCs in IoT and consumer electronics
Ali, H., Tariq, U. U., Hardy, J., Zhai, X., Liu, L., Zheng, Y., Bensaali, F., Amira, A., Fatema, K., & Antonopoulos, N. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100416 · Cited by 52
Parameterized Deep Reinforcement Learning With Hybrid Action Space for Edge Task Offloading
Wang, T., Deng, Y., Zhao, Y., Wang, Y., & Cai, H. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3327121 · Cited by 48
“In recent years, deep reinforcement learning (DRL) has emerged as an effective method to deal with the intractable task offloading problem in the dynamically changing edge environment [16], [17], [18], and a number of DRL-based approaches have been proposed.”
MethodZhou, X., Yang, J., Li, Y.-J., Li, S., & Su, Z. (2024) · Computer Communications · doi:10.1016/j.comcom.2024.107925 · Cited by 57
A Unified Smart Chinese Medicine Framework for Healthcare and Medical Services
Zhang, Q., Bai, C., Yang, L. T., Chen, Z., Li, P., & Yu, H. (2019) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2914447 · Cited by 48
“Deep reinforcement learning is especially good at solving the problem of continuous decisionmaking and it has achieved remarkable results in recent years [26].”
BackgroundSelf-citationOrtíz, F., Tarchi, D., Rodríguez‐Osorio, R. M., Vanelli‐Coralli, A., Salas‐Natera, M. A., & Ayala, S. L. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3087586 · Cited by 54
An Energy-aware Online Learning Framework for Resource Management in Heterogeneous Platforms
Mandal, S. K., Bhat, G., Doppa, J. R., Pande, P. P., & Ogras, U. Y. (2020) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3386359 · Cited by 54
“Deep Q-Learning has been used to manage control knobs of an SoC in prior work [15, 42, 48].”
MethodAlonso, R. S., Sittón-Candanedo, I., Vara, R. C., Prieto, J., & Corchado, J. M. (2020) · Sustainability · doi:10.3390/su12145706 · Cited by 40
“There are several Deep Reinforcement Learning algorithms [69], including the variants of Deep Q-Learning (DQN), such as double DQN [66], dueling DQN, Deep Recurrent Q-Networks (DQRN) and multi-step DQN, among others, and those based on policy gradient, such as REINFORCE, Advantage Actor-Critic (A2C), Natural Policy Gr…”
BackgroundMethod6G-Empowered Offloading for Realtime Applications in Multi-Access Edge Computing
Huang, H., Ye, Q., & Zhou, Y.-T. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2022.3188921 · Cited by 39
“Without loss of generosity, a realtime scheduling method, Earlier Deadline First (EDF), is leveraged to ensure the deadlines of ready tasks in every waiting queue [36].”
MethodDynamic decision support for resource offloading in heterogeneous Internet of Things environments
Jaddoa, A., Sakellari, G., Panaousis, E., Loukas, G., & Sarigiannidis, P. G. (2019) · Simulation Modelling Practice and Theory · doi:10.1016/j.simpat.2019.102019 · Cited by 42
“It has been used extensively in the related literature (e.g., [33],[34], and [35]).”
MethodGOSH: Task Scheduling Using Deep Surrogate Models in Fog Computing Environments
Tuli, S., Casale, G., & Jennings, N. R. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3136672 · Cited by 46
“In general, uncertainty may arise from several factors, includ-1 [8], [14], [15] Deep RL [9], [10] Policy Gradient [11], [16], [17] MaxWeight [12], [18] Heuristics [19], [20], [21] HGP [3] GOBI/GOBI* This work GOSH/GOSH* ing model approximations, measurement imprecision and fluctuations of…”
BackgroundMachine Learning in Advanced IC Design: A Methodological Survey
Chen, T., Zhang, G. L., Yu, B., Li, B., & Schlichtmann, U. (2022) · IEEE Design and Test · doi:10.1109/mdat.2022.3216799 · Cited by 42
“In [54], a generated network approximates the value for each DVFS algorithm, and a target network is used to train the parameters for producing the target values.”
MethodDeep Reinforcement Learning for the management of Software-Defined Networks in Smart Farming
Alonso, R. S., Sittón-Candanedo, I., Vara, R. C., Prieto, J., & Corchado, J. M. (2020) · IEEE International Conference on Omni-layer Intelligent Systems (COINS) · doi:10.1109/coins49042.2020.9191634 · Cited by 40
“There are a number of important Deep Reinforcement Learning algorithms, as pointed out by [37], those who mention the most outstanding ones in their works: Deep Qlearning (DQN), such as double DQN, DQN duels, Deep Recurrent Q-Networks (DQRN) and multi-step DQN, among others, and those based on the policy gradient, suc…”
BackgroundDey, S., Singh, A. K., Wang, X., & McDonald-Maier, K. D. (2020) · Design, Automation, and Test in Europe · doi:10.23919/date48585.2020.9116294 · Cited by 34
“In mobile MPSoC, the big CPU cores consume the most energy [21] and are also the focus of hot spots on the chip, and hence in this case, we have focused on the thermal behaviour of the big cores for the comparative”
BackgroundArshed, J. U., Ahmed, M., Muhammad, T., Afzal, M., Arif, M., & Bazezew, M. B. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/6355192 · Cited by 36
“[46] proposed a double deep Q-learning model (DDQ) in edge computing to reduce energy consumption.”
BackgroundIncremental Deep Computation Model for Wireless Big Data Feature Learning
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2019) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2903092 · Cited by 32
Self-citationAsghari, A., & Sohrabi, M. K. (2021) · Cluster Computing · doi:10.1007/s10586-021-03368-3 · Cited by 35
Deep reinforcement learning stock market trading, utilizing a CNN with candlestick images
Brim, A., & Flann, N. S. (2022) · PLoS ONE · doi:10.1371/journal.pone.0263181 · Cited by 32
Anand, J., & Karthikeyan, B. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.105890 · Cited by 27
Personalized Sports Health Recommendation System Assisted by Q-Learning Algorithm
Yang, Y., & Zhao, Y. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2295693 · Cited by 27
“In order to predict the continuous system state and reduce the energy consumption of edge computing devices, researchers such as Zhang Qingchen (Zhang et al., 2019) proposed a double deep Q-Learning model for energy-efficient edge scheduling (DDQ-EES), which saved an average of 2% to 2.4% of energy than QQL-EES (Quick…”
InfluentialEA-DFPSO: An intelligent energy-efficient scheduling algorithm for mobile edge networks
Lu, Y., Liu, L., Gu, J., Panneerselvam, J., & Yuan, B. (2021) · Digital Communications and Networks · doi:10.1016/j.dcan.2021.09.011 · Cited by 27
Energy efficient mixed task handling on real-time embedded systems using FreeRTOS
Ramegowda, D., & Lin, M. (2022) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2022.102708 · Cited by 25
Self-citationSmart Chinese medicine for hypertension treatment with a deep learning model
Zhang, Q., Bai, C., Chen, Z., Li, P., Wang, S., & Gao, H. (2018) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2018.12.012 · Cited by 28
Self-citationDeep reinforcement learning in edge networks: Challenges and future directions
Hazra, A., Tummala, V. M. R., Mazumdar, N., Sah, D. K., & Adhikari, M. (2024) · Physical Communication · doi:10.1016/j.phycom.2024.102460 · Cited by 34
Finite-Time Analysis for Double Q-learningPreprint
Xiong, H., Zhao, L., Liang, Y., & Zhang, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14257 · Cited by 24
Multipath Planning Acceleration Method With Double Deep R-Learning Based on a Genetic Algorithm
Palacios-Morocho, E., Inca, S., & Monserrat, J. F. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3277981 · Cited by 23
“Thereductionofconstant updatesresultsinamorestablepolicy[40].”
5GhNet: an intelligent QoE aware RAT selection framework for 5G-enabled healthcare network
Priya, B., & Malhotra, J. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03606-x · Cited by 23
“+ O ( 𝜁 ∗ T ∗ i ) (Zhang et al. 2018).”
BackgroundKavousighahfarokhi, A., Hannan, M. A., Abu, S. M., Ker, P. J., Wong, R. T., & Jang, G. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.115904 · Cited by 22
Yan, H., Zhang, X., Chen, H., Zhou, Y., Bao, W., & Yang, L. T. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.01.014 · Cited by 24
Self-citationQuality-aware energy efficient scheduling model for fog computing comprised IoT network
Narayana, P., Bhukya, S., Jatoth, C., & Parvataneni, P. (2021) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2021.107603 · Cited by 22
Rouzbahani, H. M., Karimipour, H., & Lei, L. (2022) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2022.102712 · Cited by 27
Traceable and undeniable ciphertext-policy attribute-based encryption for cloud storage service
Yu, G., Wang, Y., Cao, Z., Lin, J., & Wang, X. (2019) · International Journal of Distributed Sensor Networks · doi:10.1177/1550147719841276 · Cited by 22
Reinforcement Learning Based Offloading for Realtime Applications in Mobile Edge Computing
Huang, H., Ye, Q., & Du, H. (2020) · International Conference on Communications · doi:10.1109/icc40277.2020.9148748 · Cited by 24
Sun, K., Zhang, J., & Zhang, Y. (2019) · Wireless Communications and Mobile Computing · doi:10.1155/2019/6513418 · Cited by 21
Evolving graph construction for successive recommendation in event-based social networks
Liu, S., Wang, B., Xu, M., & Yang, L. T. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.02.036 · Cited by 20
Self-citationEdge computing: A systematic mapping study
Sakhdari, J., Zolfaghari, B., Izadpanah, S., Zargar, S. M., Rahati-Quchani, M., Shadi, M., Abrishami, S., & Rasoolzadegan, A. (2023) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.7741 · Cited by 20
“Reinforcement learning [222, 223], neural network [224, 225] and clustering [110, 226] are of the most widely used learning techniques in the edge computing literature.”
MethodA Novel on Transmission Line Tower Big Data Analysis Model Using Altered K-means and ADQL
Jung, S.-H., & Huh, J. (2019) · Sustainability · doi:10.3390/su11133499 · Cited by 17
Dey, S., Singh, A. K., & McDonald-Maier, K. D. (2019) · IET Computers & Digital Techniques · doi:10.1049/iet-cdt.2019.0037 · Cited by 16
“Whereas, most real world (real-time) applications [25–27] on mobile devices such as social media, email composer/reader, music player, etc. which also rely on different types of operations and are not predictable, consist of ape-riodic and sporadic tasks as well.”
BackgroundMethodInfluentialKumaran, K., & Sasikala, E. (2023) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-022-01440-2 · Cited by 20
“The mobile devices are enhanced by the ability of computing, the power consumption is minimized by the offloading of computing on the server and better performance is offered by the comparison of local processing [26].”
BackgroundImproved krill group-based region growing algorithm for image segmentation
Teng, L., Li, H., Yin, S., & Sun, Y. S. (2019) · International Journal of Image and Data Fusion · doi:10.1080/19479832.2019.1604574 · Cited by 15
“The image segmentation problem can be described as a multi-objective optimisation problem, such as minimising overall bias, maximising pixel connectivity, minimising the bit error rate of the classi fi er (L A and Chan 2002, Yin and Zhang 2018).”
BackgroundYu, Z., Liu, W., Liu, X., & Wang, G. (2021) · International Workshop on Quality of Service · doi:10.1109/iwqos52092.2021.9521327 · Cited by 15
A Review on Deep Reinforcement Learning for the management of SDN and NFV in Edge-IoT
Alonso, R. S., Prieto, J., la Prieta, F. D., Rodríguez, S., & Corchado, J. M. (2021) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps52748.2021.9682179 · Cited by 16
“Furthermore, Double DQN has been applied, for instance, as DRL method aimed at defending cyber-physical systems from unknown cyber-attacks [72] or optimize the use of energy in edge scheduling [36].”
MethodBinocular Vision-Based Motion Planning of an AUV: A Deep Reinforcement Learning Approach
Yan, J. J., You, K.-J., Cao, W., Yang, X., & Guan, X. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3321884 · Cited by 15
“Therefore, clip functions are incorporated into the reward function, which have been considered in some existing studies, e.g., [39], [40], [41], [42].”
BackgroundMobile edge computing offloading scheme based on improved multi-objective immune cloning algorithm
Zhu, S., Cai, J., & Sun, E.-L. (2023) · Wireless Networks · doi:10.1007/s11276-022-03157-9 · Cited by 15
SimTune: bridging the simulator reality gap for resource management in edge-cloud computing
Tuli, S., Casale, G., & Jennings, N. R. (2022) · Scientific Reports · doi:10.1038/s41598-022-23924-0 · Cited by 18
“For instance, some methods model the scheduling problem as a Markov Decision Process (MDP) and use a deep-reinforce-ment learning strategy, namely deep Q-Learning (DQL) to schedule workloads in a heterogeneous computing environment 28–30 .”
BackgroundMethodSEA-LEAP: Self-Adaptive and Locality-Aware Edge Analytics Placement
Lujic, I., Maio, V. D., Venugopal, S., & Brandić, I. (2021) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3104458 · Cited by 18
Adaptive Routing Strategy Based on Improved Double Q-Learning for Satellite Internet of Things
Zhou, J., Gong, X., Sun, L., Xie, Y. C., & Yan, X. (2021) · Security and Communication Networks · doi:10.1155/2021/5530023 · Cited by 14
“[40] applied double Q-learning to the energy-saving scheduling of edge computing.”
MethodWang, Z., Wan, Q., Qin, Y., Fan, S., & Xiao, Z. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02198-2 · Cited by 13
“In the configuration, the neural network of pedestrian adopts full connection structure, and the activation function adopts Relu function (Zhang et al. 2019).”
BackgroundLi, D., Lan, M., & Hu, Y. C. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00666-0 · Cited by 16
“However, the increasing popularity of IoT cloud computing has brought multiple problems, including unreasonable resource allocation, non-standard effective allocation, and threats to security and privacy, which are the core problems in this field [29].”
BackgroundResearch on Improved Pedestrian Detection Algorithm Based on Convolutional Neural Network
Wang, J., Li, H., Yin, S., & Sun, Y. (2019) · IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) · doi:10.1109/ithings/greencom/cpscom/smartdata.2019.00063 · Cited by 12
“The deep structure makes deep learning highly expressive and learning, especially good at extracting complex global features and context information, which is difficult for shallow models [8-12].”
BackgroundTrading Data Size and CNN Confidence Score for Energy Efficient CPS Node Communications
Raïs, I., Anshus, O. J., Bjørndalen, J. M., Balouek‐Thomert, D., & Parashar, M. (2020) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid49817.2020.00-46 · Cited by 12
“Finally, papers like [28] proposes to reduce the energy consumption of devices by modeling their energy consumption with learning approaches, answering parameters like the voltage or frequency for a given workload.”
BackgroundAn Efficient Multimodal Emotion Identification Using FOX Optimized Double Deep Q-Learning
Selvi, R., & Vijayakumaran, C. (2023) · Wireless Personal Communications · doi:10.1007/s11277-023-10685-w · Cited by 12
“The learning algorithm is formulated in this part for training the Double Deep Q-learning (DDQ) technique [21].”
MethodTang, M., Cai, S., & Lau, V. K. N. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3175965 · Cited by 12
“There are various standard techniques, such as value iteration [32] or Q-learning [53], that can be used to solve the Bellman equation (14).”
MethodTeng, L., & Li, H. (2019) · International Journal of Distributed Sensor Networks · doi:10.1177/1550147719847133 · Cited by 11
Morphing Strategy Design for UAV based on Prioritized Sweeping Reinforcement Learning
Li, R., Wang, Q., Liu, Y., & Dong, C. (2020) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon43393.2020.9254664 · Cited by 11
“Literature [10] proposed a double-deep Q Network (DQN) algorithms for large-scale function approximation problems.”
BackgroundRenovation of EdgeCloudSim: An Efficient Discrete-Event Approach
Freymann, R., Shi, J., Chen, J.-J., & Chen, K.-H. (2021) · International Conference on Fog and Mobile Edge Computing (FMEC) · doi:10.1109/fmec54266.2021.9732572 · Cited by 12
Shinzaki, M., Koda, Y., Yamamoto, K., Nishio, T., Morikura, M., Shirato, Y., Uchida, D., & Kita, N. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3116231 · Cited by 12
Dynamic Resource Allocation for Scalable Video Streaming in OFDMA Wireless Networks
Jiao, L., Yin, H., & Wu, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2973929 · Cited by 10
AI Back-End as a Service for Learning Switching of Mobile Apps Between the Fog and the Cloud
Athanasopoulos, D., & Liu, D. (2021) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3117927 · Cited by 12
Edge resource slicing approaches for latency optimization in AI-edge orchestration
Chandrika, P. K., Mekala, M. S., & Srivastava, G. (2022) · Cluster Computing · doi:10.1007/s10586-022-03817-7 · Cited by 11
“EC makes devices smarter due to adequate resource consumption and workload execution, but it is significantly complicated [63].”
BackgroundIntelligent Multi-connectivity Based Energy-Efficient Framework for Smart City
Priya, B., & Malhotra, J. (2023) · Journal of Network and Systems Management · doi:10.1007/s10922-023-09740-5 · Cited by 9
“N o [63].”
Stone masonry design automation via reinforcement learning
Kang, S., Dy, J., & Kane, M. B. (2023) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060423000100 · Cited by 12
“The DDQN approach has seen success in domains such as edge scheduling (Zhang et al., 2019) and autonomous vehicle speed control (Zhang et al., 2018).”
MethodZhao, Y., Deng, Y., Wang, T., & Cai, H. (2023) · International Conference on Communications · doi:10.1109/icc45041.2023.10279699 · Cited by 9
“Due to the high computation complexity and high dynamics of edge environment, in recent years, deep reinforcement learning (DRL) has been introduced to efficiently tackle the challenging edge task offloading problem [6] [7] [8].”
BackgroundAn Improved DVFS Algorithm for Energy-Efficient Real-time Task Scheduling
Yu, K., Yang, Y., Xiao, H., & Chen, J. (2020) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-smartcity-dss50907.2020.00033 · Cited by 9
“1 which illustrates a representative DVFS algorithm [11].”
BackgroundSafety-Critical Offloading with Constrained Reinforcement Learning for Multi-access Edge Computing
Huang, H., Ye, Q., & Zhou, Y. (2025) · ACM Transactions on Sensor Networks · doi:10.1145/3715695 · Cited by 8
Deep Reinforcement Learning for Dependent Task Offloading in Multi-Access Edge Computing
Ye, H., Li, J., & Lu, Q. (2024) · IEEE Access · doi:10.1109/access.2024.3494775 · Cited by 8
“The participating comparative algorithms are as follows : DDQN: A DDQN network for task allocation [42].”
Metadata
- Authors
- Zhang, Q., Lin, M., Yang, L. T., Chen, Z., Khan, S. U., & Li, P.
- Affiliations
- University of Electronic Science and Technology of China (CN)
- St. Francis Xavier University (CA)
- Dalian University of Technology (CN)
- North Dakota State University (US)
- Venue
- IEEE Transactions on Services Computing
- Published
- 2019-09-01
- Volume
- 12
- Issue
- 5
- Pages
- 739-749
- ISSN
- 1939-1374 (ISSN-L)
- 2372-0204
- OpenAlex
- W2888896501
- Cited by
- 158
- References
- 33
- Citations by year
- 2026: 6 · 2025: 12 · 2024: 16 · 2023: 20 · 2022: 21 · 2021: 29 · 2020: 29 · 2019: 22
- FWCI
- 12.85
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Services Computing (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- double deep Q-learning, edge computing, dynamic voltage and frequency scaling, target network, experience replay, rectified linear units, gradient vanishing, Q-function approximation, energy consumption reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.