The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling
Abstract
Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average 2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Compared to DQL, the experiment results indicate that Double-DQL can save more energy and achieve higher training efficiency.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [257]
Citations
Cites
Tensor Decompositions and Applications
Kolda, T. G., & Bader, B. W. (2009) · SIAM Review · doi:10.1137/07070111x · Cited by 10,301 · cited 1×
“Furthermore, we will investigate the use of tensor decompositions and tensor-train networks [30], [31], [32], [33] to improve the training efficiency of the presented double deep Q-learning model in the future work.”
MethodEdge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733 · cited 3×
“Second, cloud computing cannot ensure the privacy protection sometimes in Internet of Things and cyber-physical-social systems [14].”
BackgroundResultSelf-citation(2011) · Cited by 6,993
MiBench: A free, commercially representative embedded benchmark suite
Guthaus, M. R., Ringenberg, J. S., Ernst, D. J., Austin, T. M., Mudge, T., & Brown, R. B. (2001) · doi:10.1109/wwc.2001.15 · Cited by 3,103
Oseledets, I. V. (2011) · SIAM Journal on Scientific Computing · doi:10.1137/090752286 · Cited by 2,636
MiBench: A free, commercially representative embedded benchmark suite
Guthaus, M. R., Ringenberg, J. S., Ernst, D., Austin, T. M., Mudge, T., & Brown, R. B. (2005) · Proceedings of the Fourth Annual IEEE International Workshop on Workload Characterization. WWC-4 (Cat. No.01EX538) · doi:10.1109/wwc.2001.990739 · Cited by 1,588 · cited 1×
“We use one MiBench as the experimental taskset which includes a set of commercially representative embedded programs [29].”
MethodA survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Self-citationReal-time dynamic voltage scaling for low-power embedded operating systems
Pillai, P., & Shin, K. G. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502044 · Cited by 1,138 · cited 2×
“Another two typical DVFS algorithms are CC and LA [19].”
BackgroundMethodPower-aware scheduling for periodic real-time tasks
Aydin, H., Melhem, R., Mosse, D., & Mejia-Alvarez, P. (2004) · IEEE Transactions on Computers · doi:10.1109/tc.2004.1275298 · Cited by 484
Zhou, Z., Dong, M., Ota, K., Wang, G., & Yang, L. T. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2015.2497712 · Cited by 272 · cited 1×
“Internet of Things integrate many physical and sensing devices including RFID (Radio Frequency Identification), camera and so on to the cyber space while cyber-physical-social systems combine the human behaviors and social relations to the Internet of Things [4], [5].”
BackgroundSelf-citation
Research works printed but not linked: 1
- huang, 2017, Hybrid DVFS scheduling for real-time systems based on reinforcement learning, Proc IEEE Int Conf Green Comput Commun Conf, 522
Cited by
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“computing resources of the edge device, in [257], DoubleDQL [89] is to determine the best Dynamic Voltage and Frequency Scaling (DVFS) algorithm.”
MethodA smart agriculture IoT system based on deep reinforcement learning
Bu, F., & Wang, X. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.04.041 · Cited by 342
Tuli, S., Ilager, S., Ramamohanarao, K., & Buyya, R. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3017079 · Cited by 257
“Recent works have explored different value-based RL techniques to optimize several aspects of Resource Management Systems (RMS) in distributed environments [16], [17], [18, 19].”
BackgroundMethodEnergy-efficient edge based real-time healthcare support system
(2020) · Advances in Computers · doi:10.1016/bs.adcom.2019.09.007 · Cited by 211
AL-Jumaili, A. H. A., Muniyandi, R. C., Hasan, M. K., Paw, J. K. S., & Singh, M. S. J. (2023) · Sensors · doi:10.3390/s23062952 · Cited by 234
Metadata
- Authors
- Zhang, Q., Lin, M., Yang, L. T., Chen, Z., Khan, S. U., & Li, P.
- Affiliations
- University of Electronic Science and Technology of China (CN)
- St. Francis Xavier University (CA)
- Dalian University of Technology (CN)
- North Dakota State University (US)
- Venue
- IEEE Transactions on Services Computing
- Published
- 2019-09-01
- Volume
- 12
- Issue
- 5
- Pages
- 739-749
- ISSN
- 1939-1374 (ISSN-L)
- 2372-0204
- OpenAlex
- W2888896501
- Cited by
- 158
- References
- 33
- Citations by year
- 2026: 6 · 2025: 12 · 2024: 16 · 2023: 20 · 2022: 21 · 2021: 29 · 2020: 29 · 2019: 22
- FWCI
- 12.85
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Services Computing (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- double deep Q-learning, edge computing, dynamic voltage and frequency scaling, target network, experience replay, rectified linear units, gradient vanishing, Q-function approximation, energy consumption reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling. https://app.cochat.ai/papers/10.1109/tsc.2018.2867482
- The paper
- Zhang, Q., Lin, M., Yang, L. T., Chen, Z., Khan, S. U., & Li, P. (2019). A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling. IEEE Transactions on Services Computing. https://doi.org/10.1109/tsc.2018.2867482
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.