Two-Stage Volt/Var Control in Active Distribution Networks With Multi-Agent Deep Reinforcement Learning Method
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
The high penetration of intermittent renewable energy resources in active distribution networks (ADN) results in a great challenge for the conventional Volt-Var control (VVC). This article proposes a two-stage deep reinforcement learning (DRL)-based real-time VVC method to mitigate fast voltage violation while minimizing the network power loss. In the first stage, on-load tap changer (OLTC) and capacitor banks (CBs) are dispatched hourly based on the optimal power flow method. The optimization problem is formulated as a mixed-integer second-order cone programming (MISOCP) which can be effectively solved. In the second stage, the reactive power of photovoltaics (PVs) is regulated dynamically to mitigate fast voltage fluctuation based on the well-learned control strategy and local measurements. The real-time VVC problem is formulated and solved using a multi-agent deep deterministic policy gradient (MADDPG) method, which features offline centralized training and online decentralized application. Rather than using the critic network to evaluate the output of the actor-network, the gradient of the action-value function to action is derived analytically based on the voltage sensitivity method. The proposed approach is tested on the IEEE 33-bus distribution system and comparative simulation results show the enhanced control effect in mitigating voltage violations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Deilami, S., Masoum, A. S., Moses, P. S., & Masoum, M. A. S. (2011) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2011.2159816 · Cited by 1,145
Options for Control of Reactive Power by Distributed Photovoltaic Generators
Turitsyn, K., Sulc, P., Backhaus, S., & Chertkov, M. (2011) · Proceedings of the IEEE · doi:10.1109/jproc.2011.2116750 · Cited by 774
Distributed Optimal Power Flow for Smart Microgrids
Dall'Anese, E., Zhu, H., & Giannakis, G. B. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2248175 · Cited by 679
Photovoltaic penetration issues and impacts in distribution network – A review
Karimi, M., Mokhlis, H., Naidu, K., Uddin, S., & Bakar, A. H. A. (2016) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2015.08.042 · Cited by 545
Antoniadou-Plytaria, K. E., Kouveliotis-Lysikatos, I. N., Georgilakis, P. S., & Hatziargyriou, N. D. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2679238 · Cited by 545 · cited 2×
“The conventional centralized VVC depends highly on communication and computation requirements and may lack the capability to address fast voltage violation problems [7], [8].”
BackgroundMethodReview of Challenges and Research Opportunities for Voltage Control in Smart Grids
Sun, H., Guo, Q., Qi, J., Ajjarapu, V., Bravo, R., Chow, J., Li, Z., Moghe, R., Nasr-Azadani, E., Tamrakar, U., et al. (2019) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2019.2897948 · Cited by 496
Li, P., Ji, H., Wang, C., Zhao, J., Song, G., Ding, F., & Wu, J. (2017) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2017.2686009 · Cited by 447
Constrained EV Charging Scheduling Based on Safe Deep Reinforcement Learning
Li, H., Wan, Z., & He, H. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2955437 · Cited by 428
A Fully Distributed Reactive Power Optimization and Control Method for Active Distribution Networks
Zheng, W., Wu, W., Zhang, B., Sun, H., & Liu, Y. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2396493 · Cited by 371
A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning
Wang, S., Duan, J., Shi, D., Xu, C., Li, H., Diao, R., & Wang, Z. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.2990179 · Cited by 315
Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning
Yang, Q., Wang, G., Sadeghi, A., Giannakis, G. B., & Sun, J. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2951769 · Cited by 291
A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters
Cao, D., Hu, W., Zhao, J., Huang, Q., Chen, Z., & Blaabjerg, F. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.3000652 · Cited by 275
Paliwal, P., Patidar, N. P., & Nema, R. (2014) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2014.07.200 · Cited by 271
Glavic, M., Fonteneau, R., & Ernst, D. (2017) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2017.08.1217 · Cited by 241
Coordinated Voltage Control in Distribution Networks Including Several Distributed Energy Resources
Kulmala, A., Repo, S., & Jarventausta, P. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2297971 · Cited by 239
Wang, L., Bai, F., Yan, R., & Saha, T. K. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2789897 · Cited by 229
Real-Time Optimal Power Flow: A Lagrangian Based Deep Reinforcement Learning Approach
Yan, Z., & Xu, Y. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.2987292 · Cited by 200
Li, P., Zhang, C., Wu, Z., Xu, Y., Hu, M., & Dong, Z. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2950120 · Cited by 195
Optimal operation of smart distribution networks: A review of models, methods and future research
Evangelopoulos, V. A., Georgilakis, P. S., & Hatziargyriou, N. D. (2016) · Electric Power Systems Research · doi:10.1016/j.epsr.2016.06.035 · Cited by 181
Optimal Tap Setting of Voltage Regulation Transformers Using Batch Reinforcement Learning
Xu, H., Domínguez-García, A. D., & Sauer, P. W. (2019) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2019.2948132 · Cited by 180
Combined Local and Centralized Voltage Control in Active Distribution Networks
Bidgoli, H. S., & Van Cutsem, T. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2017.2716407 · Cited by 176
Multiagent-Based Reinforcement Learning for Optimal Reactive Power Dispatch
Xu, Y., Zhang, W., Liu, W., & Ferrese, F. T. (2012) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218596 · Cited by 149
Optimal Local Reactive Power Control by PV Inverters
Weckx, S., & Driesen, J. (2016) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2016.2572162 · Cited by 137
A Multi-Mode Control Strategy for VAr Support by Solar PV Inverters in Distribution Networks
Alam, M. J. E., Muttaqi, K. M., & Sutanto, D. (2015) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2014.2344661 · Cited by 132
An Adaptive Distributionally Robust Model for Three-Phase Distribution Network Reconfiguration
Zheng, W., Huang, W., Hill, D. J., & Hou, Y. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3030299 · Cited by 121
Inverter-Less Hybrid Voltage/Var Control for Distribution Circuits With Photovoltaic Generators
Wang, Z., Chen, H., Wang, J., & Begovic, M. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2324569 · Cited by 120
Two-Level Distributed Volt/Var Control Using Aggregated PV Inverters in Distribution Networks
Wang, Y., Zhao, T., Ju, C., Xu, Y., & Wang, P. (2020) · IEEE Transactions on Power Delivery · doi:10.1109/tpwrd.2019.2955506 · Cited by 112
Ji, H., Wang, C., Li, P., Zhao, J., Song, G., Ding, F., & Wu, J. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.07.065 · Cited by 106
Reactive Power Ancillary Service of Synchronous DGs in Coordination With Voltage Control Devices
Kim, Y. J., Kirtley, J. L., & Norford, L. K. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2472967 · Cited by 97
Huang, W., Zheng, W., & Hill, D. J. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3012025 · Cited by 93
Advanced Network Management Systems: A Risk-Based AC OPF Approach
Alnaser, S. W., & Ochoa, L. F. (2014) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2014.2320901 · Cited by 71
National Renewable Energy Lab. (NREL), G., Ding, F., Solar Energy Technologies Office (EE-4S) USDOE Office of Energy Efficiency and Renewable Energy (EERE), Nagarajan, A., Chakraborty, S., Baggu, M., Nguyen, A., Walinga, S., McCarty, M., & Bell, F. (2016) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/1337541 · Cited by 68
Sun, X., Zhang, W., & Chen, J. (2019) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2019.105734 · Cited by 25
Self-citationAdvanced network management systems: A risk-based AC OPF approach
Alnaser, S. W., & Ochoa, L. F. (2015) · Research Explorer (The University of Manchester) · Cited by 13
Cited by
Applications of Physics-Informed Neural Networks in Power Systems - A Review
Huang, B., & Wang, J. (2023) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3162473 · Cited by 619
Chen, X., Qu, G., Tang, Y., Low, S., & Li, N. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3154718 · Cited by 415
An Overview of Soft Open Points in Electricity Distribution Networks
Jiang, X., Zhou, Y., Ming, W., Yang, P., & Wu, J. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3148599 · Cited by 278
Optimal local volt/var control for photovoltaic inverters in active distribution networks
Sun, X., Qiu, J., & Zhao, J. (2021) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2021.3080039 · Cited by 111
Self-citationCao, D., Zhao, J., Hu, J., Pei, Y., Huang, Q., Chen, Z., & Hu, W. (2024) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3267069 · Cited by 103
“Many DRL-based approaches have been proposed to regulate the voltage [14], [15], [16], [17] and deal with fast PV fluctuations [18].”
Background
Metadata
- Authors
- Sun, X., & Qiu, J.
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2021-07
- Volume
- 12
- Issue
- 4
- Pages
- 2903-2912
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W3122780823
- Cited by
- 289
- References
- 35
- FWCI
- 12.03
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.