Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review
Open accessAbstract
With the growing integration of distributed energy resources (DERs), flexible loads, and other emerging technologies, there are increasing complexities and uncertainties for modern power and energy systems. This brings great challenges to the operation and control. Besides, with the deployment of advanced sensor and smart meters, a large number of data are generated, which brings opportunities for novel data-driven methods to deal with complicated operation and control issues. Among them, reinforcement learning (RL) is one of the most widely promoted methods for control and optimization problems. This paper provides a comprehensive literature review of RL in terms of basic ideas, various types of algorithms, and their applications in power and energy systems. The challenges and further works are also discussed.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The surge in workload and operational responsibilities strains existing EMSs, making targeted processing during power system operation, particularly during fault states, challenging.
“This surge in workload and operational responsibilities strains the existing energy management systems (EMSs), typically numerical analysis software, making targeted processing during power system operation, particularly during fault states, challenging [4].”
From A Comprehensive Review on the Role of Artificial Intelligence in Power System Stability, Control, and Protection: Insights and Future Directions (2024) · cites this paper as [4]
The Introduction (page unavailable) says growing complexity and uncertainty create challenges for operating power and energy systems, and the IES Management section says coordinating energy supply makes management challenging. Neither passage says increased workload strains existing EMS software or makes targeted processing difficult during fault states. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824 · see citing works
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · see citing works
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · see citing works
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578 · see citing works
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377 · see citing works
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · see citing works
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300 · see citing works
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · see citing works
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217 · see citing works
Cooperative Multi-agent Control Using Deep Reinforcement Learning
Gupta, J. K., Egorov, M., & Kochenderfer, M. J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-71682-4_5 · Cited by 926 · see citing works
Reinforcement learning for demand response: A review of algorithms and modeling techniques
Vázquez-Canteli, J. R., & Nagy, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.11.002 · Cited by 883 · see citing works
A survey and critique of multiagent deep reinforcement learning
Hernández-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 747 · see citing works
Human-level performance in 3D multiplayer games with population-based reinforcement learning
Jaderberg, M., Czarnecki, W. M., Dunning, I., Marris, L., Lever, G., Castañeda, A. G., Beattie, C., Rabinowitz, N. C., Morcos, A. S., Ruderman, A., et al. (2019) · Science · doi:10.1126/science.aau6249 · Cited by 662 · see citing works
On-Line Building Energy Optimization Using Deep Reinforcement Learning
Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2834219 · Cited by 649 · see citing works
Actor-Attention-Critic for Multi-Agent Reinforcement LearningPreprint
Iqbal, S., & Sha, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02912 · Cited by 629 · see citing works
Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning
Wan, Z., Li, H., He, H., & Prokhorov, D. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2879572 · Cited by 548 · cited 2× · see citing works
“Reference [80] proposes a DQN-based approach for the optimization of EV charging while [81] extends this work by integrating the DQN algorithm with the long-short term memory (LSTM) NN.”
MethodDeep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations
Duan, J., Shi, D., Diao, R., et al. (2019) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2019.2941134 · Cited by 436 · see citing works
Wu, J., He, H., Peng, J., Li, Y., & Li, Z. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.104 · Cited by 434 · see citing works
Constrained EV Charging Scheduling Based on Safe Deep Reinforcement Learning
Li, H., Wan, Z., & He, H. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2955437 · Cited by 434 · see citing works
Adaptive Power System Emergency Control Using Deep Reinforcement Learning
Huang, Q., Huang, R., Hao, W., Tan, J., Fan, R., & Huang, Z. H. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2933191 · Cited by 424 · see citing works
A Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home Energy Management
Xu, X., Jia, Y., Xu, Y., Xu, Z., Chai, S., & Lai, C. S. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2971427 · Cited by 420 · cited 2× · see citing works
“Reference [105] [106] [107] [108] [109] [110] [111] [112] [113] Field Demand-side management Distribution network Demand-side management Operational control Operational control Operational control Distribution network Demand response Distribution network Algorithm DQN DQN Q-learning DQN MADDPG MADDPG MADDPG + attentio…”
MethodYan, Z., & Xu, Y. H. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2881359 · Cited by 398 · see citing works
Du, Y., & Li, F. F. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2930299 · Cited by 396 · see citing works
Lu, R., Hong, S. H., & Yu, M. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2909266 · Cited by 390 · see citing works
Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings
Yu, L., Sun, Y., Xu, Z., Shen, C. Q., Yue, D., Jiang, T., & Guan, X. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3011739 · Cited by 355 · see citing works
Multi-Timescale Coordinated Voltage/Var Control of High Renewable-Penetrated Distribution Systems
Xu, Y. H., Dong, Z. Y., Zhang, R., et al. (2017) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2017.2669343 · Cited by 345 · see citing works
Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning
Ruelens, F., Claessens, B., Vandael, S., Schutter, B. D., Babuška, R., & Belmans, R. J. M. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2517211 · Cited by 339 · see citing works
Wei, W., Yu, N., Gao, Y., & Shi, J. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2962625 · Cited by 338 · see citing works
Wu, Y., Tan, H., Peng, J., et al. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.04.021 · Cited by 336 · see citing works
A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning
Wang, S., Duan, J., Shi, D., Xu, C., Li, H., Diao, R., & Wang, Z. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.2990179 · Cited by 318 · see citing works
Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems
Zhang, Y., Wang, X., Wang, J., & Zhang, Y. Y. C. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3010130 · Cited by 306 · see citing works
Deep Reinforcement Learning for Strategic Bidding in Electricity Markets
Ye, Y., Qiu, D., Sun, M., Papadaskalopoulos, D. A., & Štrbac, G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2936142 · Cited by 298 · see citing works
Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning
Yang, Q., Wang, G., Sadeghi, A., Giannakis, G. B., & Sun, J. Q. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2951769 · Cited by 295 · see citing works
Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial ObservabilityPreprint
Omidshafiei, S., Pazis, J., Amato, C., How, J. P., & Vian, J. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.06182 · Cited by 289 · see citing works
Bui, Y.-H., Hussain, A., & Kim, H. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2924025 · Cited by 283 · see citing works
Self-Optimizing Memory Controllers: A Reinforcement Learning Approach
İpek, E., Mutlu, O., Martínez, J. F., et al. (2008) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2008.21 · Cited by 280 · see citing works
Liu, T., Zou, Y., Liu, D., & Sun, F. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2015.2475419 · Cited by 279 · see citing works
Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning
Hu, Y., Li, W., Xu, K., Zahid, T., Qin, F., & Li, C. (2018) · Applied Sciences · doi:10.3390/app8020187 · Cited by 276 · see citing works
Reinforcement learning for microgrid energy management
Kuznetsova, E., Li, Y., Ruiz, C., Zio, E., Ault, G., & Bell, K. R. W. (2013) · Energy · doi:10.1016/j.energy.2013.05.060 · Cited by 275 · see citing works
A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters
Cao, D., Hu, W., Zhao, J., et al. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.3000652 · Cited by 275 · see citing works
Self-citationYe, Y., Qiu, D., Wu, X., Štrbac, G., & Ward, J. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2976771 · Cited by 246 · see citing works
Optimal electricity supply bidding by Markov decision process
Song, H., Liu, C.-C., Lawarree, J. P., & Dahlgren, R. W. (2000) · IEEE Transactions on Power Systems · doi:10.1109/59.867150 · Cited by 240 · see citing works
Deep reinforcement learning enabled self-learning control for energy efficient driving
Qi, X., Luo, Y., Wu, G., Boriboonsomsin, K., & Barth, M. (2019) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2018.12.018 · Cited by 235 · see citing works
Network Partition-Based Zonal Voltage Control for Distribution Networks With Distributed PV Systems
Zhao, B., Xu, Z., Xu, C., Wang, C., & Lin, F. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2648779 · Cited by 231 · see citing works
Real-Time Residential Demand Response
Li, H., Wan, Z., & He, H. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2978061 · Cited by 206 · see citing works
Li, P., Zhang, C., Wu, Z., et al. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2950120 · Cited by 200 · see citing works
Integrating high levels of variable renewable energy into electric power systems
Kroposki, B. (2017) · Journal of Modern Power Systems and Clean Energy · doi:10.1007/s40565-017-0339-3 · Cited by 199 · see citing works
Pagola, F. L., Arriaga, J. I. P., & Verghese, G. C. (1989) · IEEE Power Engineering Review · doi:10.1109/mper.1989.4310488 · Cited by 195 · see citing works
Duan, J., Yi, Z., Shi, D., Lin, C., Lu, X. B., & Wang, Z. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2896618 · Cited by 188 · see citing works
Batch-Constrained Reinforcement Learning for Dynamic Distribution Network Reconfiguration
Gao, Y., Wei, W., Shi, J., et al. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3005270 · Cited by 187 · see citing works
Reinforcement Learning-Based Plug-in Electric Vehicle Charging with Forecasted Price
Chis, A., Lundén, J., & Koivunen, V. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2603536 · Cited by 180 · see citing works
Chung, C. Y., Wang, L., Howell, F., et al. (2004) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2003.820700 · Cited by 169 · see citing works
A Reinforcement Learning Model to Assess Market Power Under Auction-Based Energy Pricing
Nanduri, V., & Das, T. K. (2007) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2006.888977 · Cited by 168 · see citing works
Optimal Tap Setting of Voltage Regulation Transformers Using Batch Reinforcement Learning
Xu, H., Domínguez-García, A. D., & Sauer, P. W. (2019) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2019.2948132 · Cited by 158 · see citing works
Model-Free Emergency Frequency Control Based on Reinforcement Learning
Chen, C., Cui, M., Li, F. F., et al. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3001095 · Cited by 158 · see citing works
Zhang, B., Hu, W., Cao, D., et al. (2019) · Energy Conversion and Management · doi:10.1016/j.enconman.2019.112199 · Cited by 155 · see citing works
Self-citationLiu, Y., Yuen, C., Hassan, N. U., et al. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2371780 · Cited by 152 · see citing works
A Learning-Based Power Management Method for Networked Microgrids Under Incomplete Information
Zhang, Q., Dehghanpour, K., Wang, Z., & Huang, Q. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2933502 · Cited by 136 · see citing works
Zhang, B., Hu, W., Li, J., Cao, D., Huang, R., Huang, Q. P., Chen, Z., & Blaabjerg, F. (2020) · Energy Conversion and Management · doi:10.1016/j.enconman.2020.113063 · Cited by 134 · see citing works
Self-citationDeep Reinforcement Learning Solutions for Energy Microgrids Management
François-Lavet, V., Taralla, D., Ernst, D., & Raphael Fonteneau, D. (2016) · ORBi (University of Liège) · Cited by 128 · see citing works
Lenient Multi-Agent Deep Reinforcement Learning
Palmer, G., Tuyls, K., Bloembergen, D., & Savani, R. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/qdcv6054 · Cited by 128 · see citing works
Deep Reinforcement Learning for Joint Bidding and Pricing of Load Serving Entity
Xu, H., Sun, H., Nikovski, D. N., Kitamura, S., Mori, K., & Hashimoto, H. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2903756 · Cited by 119 · see citing works
Autonomous Voltage Control for Grid Operation Using Deep Reinforcement Learning
Diao, R., Wang, Z., Shi, D., Chang, Q., Duan, J., & Zhang, X. (2019) · 2019 IEEE Power & Energy Society General Meeting (PESGM) · doi:10.1109/pesgm40551.2019.8973924 · Cited by 115 · see citing works
Al-Saffar, M., & Musı́lek, P. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2972208 · Cited by 113 · see citing works
Zhang, G., Hu, W., Cao, D., Huang, Q. P., Yi, J., Chen, Z., & Blaabjerg, F. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2997790 · Cited by 113 · see citing works
Self-citationKazmi, H., Mehmood, F., Lodeweyckx, S., & Driesen, J. (2018) · Energy · doi:10.1016/j.energy.2017.12.019 · Cited by 109 · see citing works
Demand side management for a residential customer in multi-energy systems
Sheikhi, A., Rayati, M., & Ranjbar, A. M. (2016) · Sustainable Cities and Society · doi:10.1016/j.scs.2016.01.010 · Cited by 106 · see citing works
Reinforcement Learning: A SurveyPreprint
Kaelbling, L. P., Littman, M. L., & Moore, A. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/9605103 · Cited by 99 · see citing works
Li, G., & Shi, J. (2012) · Applied Energy · doi:10.1016/j.apenergy.2012.04.022 · Cited by 93 · see citing works
Hadidi, R., & Jeyasurya, B. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2235864 · Cited by 79 · see citing works
Cao, D., Hu, W., Xu, X., Dragičević, T., Huang, Q. P., Liu, Z., Chen, Z., & Blaabjerg, F. (2019) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2019.105648 · Cited by 79 · see citing works
Self-citationMorshed, M. J., & Fekih, A. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2901935 · Cited by 77 · see citing works
Hashmy, Y., Yu, Z., Shi, D., et al. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3008364 · Cited by 75 · see citing works
Deep reinforcement learning-based vehicle energy efficiency autonomous learning system
Qi, X., Luo, Y., Wu, G., Boriboonsomsin, K., & Barth, M. (2017) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2017.7995880 · Cited by 71 · see citing works
AI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCs
Lan, T., Duan, J., Zhang, B., Shi, D., Wang, Z., Diao, R., & Zhang, X. (2020) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm41954.2020.9281518 · Cited by 66 · see citing works
Self-citationLiu, S., & Henze, G. P. (2006) · Journal of Solar Energy Engineering · doi:10.1115/1.2710491 · Cited by 59 · see citing works
Rahimiyan, M., & Mashhadi, H. R. (2007) · Electric Power Systems Research · doi:10.1016/j.epsr.2007.01.009 · Cited by 59 · see citing works
A data-driven approach for designing STATCOM additional damping controller for wind farms
Zhang, G., Hu, W., Cao, D., Yi, J., Huang, Q. P., Liu, Z., Chen, Z., & Blaabjerg, F. (2019) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2019.105620 · Cited by 58 · see citing works
Self-citationAn electricity supplier bidding strategy through Q-Learning
Xiong, G., Hashiyama, T., & Okuma, S. (2003) · IEEE Power Engineering Society Summer Meeting · doi:10.1109/pess.2002.1043645 · Cited by 52 · see citing works
Stochastic Control for Smart Grid Users With Flexible Demand
Liang, Y., He, L., Cao, X., et al. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2263201 · Cited by 51 · see citing works
Optimal robust adaptive fuzzy synergetic power system stabilizer design
Bouchama, Z., & Harmas, M. N. (2011) · Electric Power Systems Research · doi:10.1016/j.epsr.2011.11.003 · Cited by 51 · see citing works
Sistani, M. B. N., Akbarzadeh-Tootoonchi, M.-R., Bayaz, M. H. J.-D., et al. (2005) · Energy Conversion and Management · doi:10.1016/j.enconman.2005.08.012 · Cited by 49 · see citing works
Xu, X., Xu, Y. H., Wang, M., Li, J., Xu, Z., Chai, S., & He, Y. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3016336 · Cited by 47 · see citing works
A Framework-Based Approach to Utility Big Data Analytics
Zhu, J., Zhuang, E., Fu, J., et al. (2015) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2015.2462775 · Cited by 46 · see citing works
On-line Building Energy Optimization using Deep Reinforcement LearningPreprint
Mocanu, E., Mocanu, D. C., Nguyen, P. H., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05878 · Cited by 45 · see citing works
A Deep Policy Inference Q-Network for Multi-Agent SystemsPreprint
Hong, Z.-W., Su, S.-Y., Shann, T.-Y., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.07893 · Cited by 43 · see citing works
Vázquez-Canteli, J. R., Kämpf, J. H., & Nagy, Z. (2017) · Energy Procedia · doi:10.1016/j.egypro.2017.07.429 · Cited by 42 · see citing works
Policy: Bring sustainable energy to the developing world
Detchon, R., & Leeuwen, R. V. (2014) · Nature · doi:10.1038/508309a · Cited by 41 · see citing works
Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement LearningPreprint
Yang, Q., Wang, G., Sadeghi, A., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09374 · Cited by 37 · see citing works
Ke, D., Shen, F., Chung, C. Y., et al. (2017) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2017.2761913 · Cited by 36 · see citing works
Zhao, P., Yao, W., Wang, S., et al. (2013) · International Transactions on Electrical Energy Systems · doi:10.1002/etep.1788 · Cited by 34 · see citing works
An adaptive neuro-control approach for multi-machine power systems
Ni, Z., Tang, Y., Sui, X., et al. (2015) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2015.08.012 · Cited by 25 · see citing works
A Data-Driven Approach for Real-Time Residential EV Charging Management
Wan, Z., Li, H., He, H., & Prokhorov, D. V. (2018) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm.2018.8585945 · Cited by 20 · see citing works
Adaptive Power System Emergency Control using Deep Reinforcement LearningPreprint
Huang, Q., Huang, R., Hao, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03712 · Cited by 17 · see citing works
Autonomous Voltage Control for Grid Operation Using Deep Reinforcement LearningPreprint
Diao, R., Wang, Z., Shi, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.10597 · Cited by 16 · see citing works
Multi-objective energy management for we-energy in Energy Internet using reinforcement learning
Sun, Q., Wang, D., Ma, D., et al. (2017) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2017.8285243 · Cited by 15 · see citing works
AI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCsPreprint
Lan, T., Duan, J., Zhang, B., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.04263 · Cited by 14 · see citing works
Self-citationStudy on an Average Reward Reinforcement Learning Algorithm
Gao, Y. (2007) · Chinese Journal of Computers · Cited by 13 · see citing works
Deep Reinforcement Learning for Economic Energy Scheduling in Data Center Microgrids
Yang, X., Wang, Y., He, H., Sun, C., & Zhang, Y. (2019) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm40551.2019.8974083 · Cited by 12 · see citing works
Self-learning agent for battery energy management in a residential microgrid
Mbuwir, B. V., Spiessens, F., & Deconinck, G. (2018) · IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT Europe) · doi:10.1109/isgteurope.2018.8571568 · Cited by 11 · see citing works
(1992) · John Benjamins Publishing Company eBooks · doi:10.1075/z.ktt1.c7 · Cited by 10 · see citing works
Batch-Constrained Reinforcement Learning for Dynamic Distribution Network ReconfigurationPreprint
Gao, Y., Wang, W., Shi, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12749 · Cited by 9 · see citing works
Design and Application of Big Data Platform Architecture for Typical Scenarios of Power System
Cao, D., Li, J., Cai, D., et al. (2018) · IEEE Power & Energy Society General Meeting (PESGM) · doi:10.1109/pesgm.2018.8586266 · Cited by 6 · see citing works
Self-citationCao, D., Zhao, J., Hu, W., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13992 · Cited by 6 · see citing works
Self-citationHashmy, Y., Yu, Z., Shi, D., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.07829 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 14
- [12]12. S. Das and I. Pan, "On the mixed H ∞ /H 2 loop-shaping tradeoffs in frac- tional-order control of the AVR system," IEEE Transactions on Indus- trial Informatics, vol. 10, no. 4, pp. 1982-1991, Nov. 2013.
- [22]22. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cam- bridge: MIT Press, 2016.
- [23]23. R. S. Sutton and A. G. Barto, Reinforcement Learning: an Introduc- tion. Cambridge: MIT Press, 1998.
- [30]30. D. Silver, G. Lever, N. Heess et al., "Deterministic policy gradient al- gorithms," in Proceedings of International Conference on Machine Learning, Beijing, China, Jun. 2014, pp. 387-395.
- [33]33. J. Schulman, S. Levine, P. Abbeel et al., "Trust region policy optimiza- tion," in Proceedings of International Conference on Machine Learn- ing, Lille, France, Jul. 2015, pp. 1889-1897.
- [34]34. J. Schulman, F. Wolski, P. Dhariwal et al. (2017, Jul.). Proximal poli- cy optimization algorithms. [Online]. Available: https://arxiv. org/abs/ 1707.06347
- [36]36. J. N. Foerster, N. Nardelli, G. Farquhar et al., "Stabilizing experience replay for deep multi-agent reinforcement learning," in Proceedings of International Conference on Machine Learning, Sydney, Australia, Aug. 2017, pp. 1-10.
- [47]47. Z. Yan and Y. Xu, "Real-time optimal power flow: a Lagrangian based deep reinforcement learning approach," IEEE Transactions on Power Systems, vol. 35, no. 4, pp. 3270-3273, Apr. 2020.
- [62]62. G. P. Henze and S. Liu, "Experimental analysis of simulated reinforce- ment learning control for active and passive building thermal storage inventory-part 1: theoretical foundation." Energy and Buildings, vol. 38, no. 2, pp. 142-147, Feb. 2006.
- [63]63. G. P. Henze and S. Liu, "Experimental analysis of simulated reinforce- ment learning control for active and passive building thermal storage inventory -part 2: results and analysis," Energy and Buildings, vol. 38, no. 2, pp. 148-161, Feb. 2006.
- [88]88. A. C. Tellidou and A. G. Bakirtzis, "Agent-based analysis of capacity with holding and tacit collusion in electricity markets," IEEE Transac- tions on Power Systems, vol. 22, no. 4, pp. 1735-1742, Dec. 2007.
- [94]94. T. Yu, B. Zhou, K. W. Chan et al., "Stochastic optimal relaxed auto- matic generation control in non-Markov environment based on multi- step Q(λ) learning," IEEE Transactions on Power Systems, vol. 26, no. 3, pp. 1272 -1282, Aug. 2011.
- [110]110. Z. Yan and Y. Xu, "A multi-agent deep reinforcement learning method for cooperative load frequency control of multi-area power systems," IEEE Transactions on Power Systems, vol. 35, no. 6, pp. 4599-4608, Nov. 2020.
- [113]113. D. Cao, J. Zhao, W. Hu et al. (2020, May). Distributed voltage regula- tion of active distribution system based on enhanced multi-agent deep reinforcement learning. [Online]. Available: https://arxiv. org/abs/ 2006.00546
Cited by
Sorted by citations
Alhamrouni, I., Abdul Kahar, N. H., Salem, M., Swadi, M., Zahroui, Y., Kadhim, D. J., Mohamed, F. A., & Alhuyi Nazari, M. (2024) · Applied Sciences · doi:10.3390/app14146214 · Cited by 167 · see citing works
Multi-agent hierarchical reinforcement learning for energy management
Jendoubi, I., & Bouffard, F. (2023) · Applied Energy · doi:10.1016/j.apenergy.2022.120500 · Cited by 123 · see citing works
Comparative analysis of Q-learning, SARSA, and deep Q-network for microgrid energy management
Ramesh, S., N, S. B., Sathyavarapu, S. J., Sharma, V., A. A., N. K., & Khanna, M. (2025) · Scientific Reports · doi:10.1038/s41598-024-83625-8 · Cited by 35 · see citing works
- Authors
- Cao, D., Hu, W., Zhao, J., Zhang, G., Zhang, B., Liu, Z., Chen, Z., & Blaabjerg, F.
- Affiliations
- University of Electronic Science and Technology of China (CN)
- Mississippi State University (US)
- Aalborg University (DK)
- Venue
- Journal of Modern Power Systems and Clean Energy
- Published
- 2020
- Volume
- 8
- Issue
- 6
- Pages
- 1029-1042
- ISSN
- 2196-5420 (ISSN-L)
- 2196-5625
- OpenAlex
- W3108288031
- Cited by
- 460
- References
- 110
- Citations by year
- 2026: 91 · 2025: 92 · 2024: 92 · 2023: 82 · 2022: 74 · 2021: 28 · 2020: 1
- FWCI
- 15.89
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Journal of Modern Power Systems and Clean Energy (publishedVersion)
- VBN Forskningsportal (Aalborg Universitet) (acceptedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Power System Optimization
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- reinforcement learning, power system operation, energy system control, distributed energy resources, flexible loads, data-driven control, smart meters, power system optimization, power system uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
