Reinforcement Learning Techniques for Optimal Power Control in Grid-Connected Microgrids: A Comprehensive Review
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Utility grids are undergoing several upgrades. Distributed generators that are supplied by intermittent renewable energy sources (RES) are being connected to the grids. As RES get cheaper, more customers are opting for peer-to-peer energy interchanges through the smart metering infrastructure. Consequently, power management in grid-tied RES-based microgrids has become a challenging task. This paper reviews the applications of reinforcement learning (RL) algorithms in managing power in grid-tide microgrids. Unlike other optimization methods such as numerical and soft computing techniques, RL does not require an accurate model of the optimization environment in order to arrive at an optimal solution. In this paper, various challenges associated with the control of power in grid-tied microgrids are described. The application of RL techniques in addressing those challenges is reviewed critically. This review identifies the need to improve and scale multi-agent RL methods to enable seamless distributed power dispatch among interconnected microgrids. Finally, the paper gives directions for future research, e.g., the hybridization of intrinsic and extrinsic reward schemes, the use of transfer learning to improve the learning outcomes of RL in complex power systems environments and the deployment of priority-based experience replay in post-disaster microgrid power flow control.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 2×
“To address this, Google DeepMind Technologies developed the asynchronous advantage actor-critic (A3C) algorithm [87].”
MethodDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · cited 3×
“There are two learning methods that produce intrinsic motivation in the agent, namely, curiosity-driven learning and hindsight experience replay [92].”
BackgroundInfluentialA Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Olivares, D. E., Mehrizi-Sani, A., Etemadi, A. H., Canizares, C. A., Iravani, R., Kazerani, M., Hajimiragha, A. H., Gomis-Bellmunt, O., Saeedifard, M., Palma-Behnke, R., et al. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2295514 · Cited by 3,137 · cited 2×
“…under a stochastic environment, (b) the development of demand response strategies that enable customers to cooperate with the grid in energy trading contracts, and (c) the design of modern voltage and frequency regulation methods to accommodate the increased in power conversion stages [11], [14].”
BackgroundYilmaz, M., & Krein, P. T. (2013) · IEEE Transactions on Power Electronics · doi:10.1109/tpel.2012.2212917 · Cited by 3,118
Review of Deep Learning Algorithms and Architectures
Shrestha, A., & Mahmood, A. (2019) · IEEE Access · doi:10.1109/access.2019.2912200 · Cited by 1,883 · cited 2×
“Many training algorithms have been developed to get the neural network to arrive at the optimal θ , namely, momentum, back propagation, Levenberg–Marquardt algorithm, etc., [75].”
Background
Cited by
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656
Multi-agent hierarchical reinforcement learning for energy management
Jendoubi, I., & Bouffard, F. (2023) · Applied Energy · doi:10.1016/j.apenergy.2022.120500 · Cited by 123
Comparative analysis of Q-learning, SARSA, and deep Q-network for microgrid energy management
Ramesh, S., N, S. B., Sathyavarapu, S. J., Sharma, V., A. A., N. K., & Khanna, M. (2025) · Scientific Reports · doi:10.1038/s41598-024-83625-8 · Cited by 35
A multi-agent deep reinforcement learning based energy management for behind-the-meter resources
Wilk, P., Wang, N., & Li, J. (2022) · The Electricity Journal · doi:10.1016/j.tej.2022.107129 · Cited by 9
Metadata
- Authors
- Arwa, E. O., & Folly, K. A.
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 208992-209007
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3105693230
- Cited by
- 146
- References
- 14
- FWCI
- 6.83
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.