An Online Learning Algorithm for Demand Response in Smart Grid
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Demand response program with real-time pricing can encourage electricity users toward scheduling their energy usage to off-peak hours. A user needs to schedule the energy usage of his appliances in an online manner since he may not know the energy prices and the demand of his appliances ahead of time. In this paper, we study the users' long-term load scheduling problem and model the changes of the price information and load demand as a Markov decision process, which enables us to capture the interactions among users as a partially observable stochastic game. To make the problem tractable, we approximate the users' optimal scheduling policy by the Markov perfect equilibrium (MPE) of a fully observable stochastic game with incomplete information. We develop an online load scheduling learning (LSL) algorithm based on the actor-critic method to determine the users' MPE policy. When compared with the benchmark of not performing demand response, simulation results show that the LSL algorithm can reduce the expected cost of users and the peak-to-average ratio in the aggregate load by 28% and 13%, respectively. When compared with the short-term scheduling policies, the users with the long-term policies can reduce their expected cost by 17%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiagent systems: algorithmic, game-theoretic, and logical foundations
(2009) · Choice Reviews Online · doi:10.5860/choice.46-5662 · Cited by 1,756
An adaptive optimal controller for discrete-time Markov environments
Witten, I. H. (1977) · Information and Control · doi:10.1016/s0019-9958(77)90354-0 · Cited by 180 · cited 3×
“The concept of the actor-critic was originally introduced by Witten [27] and then elaborated by Barto et al.”
Method
Cited by
Reinforcement learning for demand response: A review of algorithms and modeling techniques
Vázquez-Canteli, J. R., & Nagy, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.11.002 · Cited by 883
Enabling Massive IoT Toward 6G: A Comprehensive Survey
Guo, F., Yu, F. R., Zhang, H., Li, X., Ji, H., & Leung, V. C. M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3063686 · Cited by 701
Antonopoulos, I., Robu, V., Couraud, B., Kirli, D., Norbu, S., Kiprakis, A., Flynn, D., Elizondo-Gonzalez, S., & Wattam, S. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109899 · Cited by 656
Reinforcement learning for building controls: The opportunities and challenges
Wang, Z., & Hong, T. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115036 · Cited by 577
Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning
Wan, Z., Li, H., He, H., & Prokhorov, D. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2879572 · Cited by 548
Du, Y., & Li, F. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2930299 · Cited by 393
State-of-the-art on research and applications of machine learning in the building life cycle
Hong, T., Wang, Z., Luo, X., & Zhang, W. (2020) · Energy and Buildings · doi:10.1016/j.enbuild.2020.109831 · Cited by 333
Artificial Intelligence Enabled Demand Response: Prospects and Challenges in Smart Grid Environment
Khan, M. A., Saleh, A. M., Waseem, M., & Sajjad, I. A. (2023) · IEEE Access · doi:10.1109/access.2022.3231444 · Cited by 179
Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids
Chung, H.-M., Maharjan, S., Zhang, Y., & Eliassen, F. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3007167 · Cited by 120
“To further address the discretization issue associated with Q-learning [18]–[23], DQN [24]–[26], and policy gradient [27], [28], the employed model-free approach is similar to model-based methods that provide a continuous value as the solution without discretizing the action space.”
BackgroundMethod
Metadata
- Authors
- Bahrami, S., Wong, V. W. S., & Huang, J.
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2018-09
- Volume
- 9
- Issue
- 5
- Pages
- 4712-4725
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W2586852908
- Cited by
- 144
- References
- 2
- FWCI
- 9.29
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.