CoChat

A Double Deep Q-Learning Model for Energy-Efficient Edge Scheduling

Zhang, Qingchen, Lin, Man, Yang, Laurence T., Chen, Zhikui, Khan, Samee U., Li, Peng · 2019 · IEEE Transactions on Services Computing

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Abstract

Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average 2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.

Overview

Integrity signals

Registry

Crossref · 10.1109/tsc.2018.2867482

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 1

  1. huang, 2017, Hybrid DVFS scheduling for real-time systems based on reinforcement learning, Proc IEEE Int Conf Green Comput Commun Conf, 522

Cited by

See all 161

Metadata

Authors
Zhang, Q., Lin, M., Yang, L. T., Chen, Z., Khan, S. U., & Li, P.
Affiliations
  • University of Electronic Science and Technology of China (CN)
  • St. Francis Xavier University (CA)
  • Dalian University of Technology (CN)
  • North Dakota State University (US)
Venue
IEEE Transactions on Services Computing
Published
2019-09-01
Volume
12
Issue
5
Pages
739-749
ISSN
  • 1939-1374 (ISSN-L)
  • 2372-0204
OpenAlex
W2888896501
Cited by
158
References
33
Citations by year
2026: 6 · 2025: 12 · 2024: 16 · 2023: 20 · 2022: 21 · 2021: 29 · 2020: 29 · 2019: 22
FWCI
12.85
Type
journal article
Language
en
Open access
closed
Locations
IEEE Transactions on Services Computing (publishedVersion)
Funders
National Science Foundation
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
  • Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
  • Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
Keywords
double deep Q-learning, edge computing, dynamic voltage and frequency scaling, target network, experience replay, rectified linear units, gradient vanishing, Q-function approximation, energy consumption reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.