The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Deep reinforcement learning for energy management in a microgrid with flexible demand
Open accessAbstract
In this paper, we study the performance of various deep reinforcement learning algorithms to enhance the energy management system of a microgrid. We propose a novel microgrid model that consists of a wind turbine generator, an energy storage system, a set of thermostatically controlled loads, a set of price-responsive loads, and a connection to the main grid. The proposed energy management system is designed to coordinate among the different flexible sources by defining the priority resources, direct demand control signals, and electricity prices. Seven deep reinforcement learning algorithms were implemented and are empirically compared in this paper. The numerical results show that the deep reinforcement learning algorithms differ widely in their ability to converge to optimal policies. By adding an experience replay and a semi-deterministic training phase to the well-known asynchronous advantage actor-critic algorithm, we achieved the highest model performance as well as convergence to near-optimal policies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
(1999) · Cited by 5,147
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Advances in neural information processing systems 11
Peter Sollich, A. (1999) · Cited by 3,168
International encyclopedia of the social & behavioral sciences
(2006) · Choice Reviews Online · doi:10.5860/choice.43sup-0452 · Cited by 1,947
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · Machine Learning · doi:10.1007/bf00992699 · Cited by 1,564
A Model Predictive Control Approach to Microgrid Operation Optimization
Parisio, A., Rikos, E., & Glielmo, L. (2014) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2013.2295737 · Cited by 929
Reinforcement learning for demand response: A review of algorithms and modeling techniques
Vázquez-Canteli, J. R., & Nagy, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.11.002 · Cited by 883
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817
Aggregate flexibility of thermostatically controlled loads
Hao, H., Sanandaji, B. M., Poolla, K., & Vincent, T. L. (2015) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2014.2328865 · Cited by 733
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709
Reinforcement Learning: State-of-the-Art
(2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3 · Cited by 691
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., & Zaremba, W. (2016) · arXiv · doi:10.48550/arxiv.1606.01540 · Cited by 582
Incentive-based demand response for smart grid with reinforcement learning and deep neural network
Lu, R., & Hong, S. H. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.12.061 · Cited by 394
Optimal energy management strategies for energy Internet via deep reinforcement learning approach
Hua, H., Qin, Y., Hao, C., & Cao, J. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.01.145 · Cited by 329
Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning
Kim, B.-G., Zhang, Y., van der Schaar, M., & Lee, J.-W. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2495145 · Cited by 299
2016 IEEE Symposium Series on Computational Intelligence (SSCI)
(2016) · doi:10.1109/ssci37083.2016 · Cited by 296
Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid
Foruzan, E., Soh, L.-K., & Asgarpoor, S. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2823641 · Cited by 294
Bui, Y.-H., Hussain, A., & Kim, H.-M. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2924025 · Cited by 282
Robust Operation of Microgrids via Two-Stage Coordinated Energy Storage and Direct Load Control
Zhang, C., Xu, Y., Dong, Z. Y., & Ma, J. (2017) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2016.2627583 · Cited by 277
Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning
Ji, Y., Wang, J., Xu, J., Fang, X., & Zhang, H. (2019) · Energies · doi:10.3390/en12122291 · Cited by 275
Cooperative Control of Distributed Energy Storage Systems in a Microgrid
Xu, Y., Zhang, W., Hug, G., Kar, S., & Li, Z. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2354033 · Cited by 265
Robust Coordination of Distributed Generation and Price-Based Demand Response in Microgrids
Zhang, C., Xu, Y., Dong, Z. Y., & Wong, K. P. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2653198 · Cited by 242
Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids
Kofinas, P., Dounis, A. I., & Vouros, G. A. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.017 · Cited by 241
Community energy storage: A smart choice for the smart grid?
Barbour, E., Parra, D., Awwad, Z., & González, M. C. (2018) · Applied Energy · doi:10.1016/j.apenergy.2017.12.056 · Cited by 239
Jian, L., Xue, H., Xu, G., Zhu, X., Zhao, D., & Shao, Z. Y. (2013) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2012.2198037 · Cited by 233
Use of model predictive control for experimental microgrid optimization
Parisio, A., Rikos, E., Tzamalis, G., & Glielmo, L. (2014) · Applied Energy · doi:10.1016/j.apenergy.2013.10.027 · Cited by 223
Arbitraging Intraday Wholesale Energy Market Prices With Aggregations of Thermostatic Loads
Mathieu, J. L., Kamgarpour, M., Lygeros, J., Andersson, G., & Callaway, D. S. (2015) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2014.2335158 · Cited by 212
Artificial intelligence based smart energy community management: A reinforcement learning approach
School of Electrical Engineering, S. U., Zhou, S., Hu, Z., School of Electrical Engineering, S. U., Gu, W., School of Electrical Engineering, S. U., Jiang, M., Department of Computer Science and Engineering, U. O. N. D., Zhang, X.-P., & School of Electronic, E. A. S. E. (2019) · CSEE Journal of Power and Energy Systems · doi:10.17775/cseejpes.2018.00840 · Cited by 189
Battery Energy Management in a Microgrid Using Batch Reinforcement Learning
Mbuwir, B., Ruelens, F., Spiessens, F., & Deconinck, G. (2017) · Energies · doi:10.3390/en10111846 · Cited by 187
Coordination of Generation Scheduling with PEVs Charging in Industrial Microgrids
Derakhshandeh, S. Y., Masoum, A. S., Deilami, S., Masoum, M. A. S., & Hamedani Golshan, M. E. (2013) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2013.2257184 · Cited by 177
Claessens, B., Vrancx, P., & Ruelens, F. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2629450 · Cited by 160
Deep reinforcement learning with experience replay based on SARSA
Zhao, D., Wang, H., Shao, K., & Zhu, Y. (2016) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7849837 · Cited by 153
Deep Reinforcement Learning Solutions for Energy Microgrids Management
François-Lavet, V., Taralla, D., Ernst, D., & Raphael Fonteneau, D. (2016) · ORBi (University of Liège) · Cited by 128
The operation of diesel gensets in a CERTS microgrid
Krishnamurthy, S., Jahns, T. M., & Lasseter, R. H. (2008) · IEEE Power Engineering Society General Meeting · doi:10.1109/pes.2008.4596500 · Cited by 122
Local Energy Trading Behavior Modeling With Deep Reinforcement Learning
Chen, T., & Su, W. (2018) · IEEE Access · doi:10.1109/access.2018.2876652 · Cited by 121
Energy Management System for Stand-Alone Wind-Powered-Desalination Microgrid
Guo, L., Liu, W., Li, X., Liu, Y., Jiao, B., Wang, W., Wang, C., & Li, F. F. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2377374 · Cited by 116
Value of Price Responsive Load for Wind Integration in Unit Commitment
Jonghe, C. D., Hobbs, B. F., & Belmans, R. J. M. (2013) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2013.2283516 · Cited by 86
Multi-agent Deep Reinforcement Learning for Zero Energy Communities
Prasad, A., & Dusparić, I. (2019) · IEEE PES Innovative Smart Grid Technologies Europe · doi:10.1109/isgteurope.2019.8905628 · Cited by 85
Model-based and model-free “plug-and-play” building energy efficient control
Baldi, S., Michailidis, I., Ravanis, C., & Kosmatopoulos, E. B. (2015) · Applied Energy · doi:10.1016/j.apenergy.2015.05.081 · Cited by 78
Phan, B. C., & Lai, Y. (2019) · Applied Sciences · doi:10.3390/app9194001 · Cited by 69
An ANN-based model for learning individual customer behavior in response to electricity prices
Nakabi, T. A., & Toivanen, P. (2019) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2019.100212 · Cited by 67
Self-citationReinforcement learning for control of flexibility providers in a residential microgrid
Mbuwir, B. V., Geysen, D., Spiessens, F., & Deconinck, G. (2019) · IET Smart Grid · doi:10.1049/iet-stg.2019.0196 · Cited by 48
Deep Reinforcement Learning for Energy Microgrids Management Considering Flexible Energy Sources
Tomin, N., Жуков, А. В., & Domyshev, A. (2019) · EPJ Web of Conferences · doi:10.1051/epjconf/201921701016 · Cited by 33
An Energy Management System for a community energy storage system
Huq, K. M. M., Baran, M. E., Lukic, S. M., & Nare, O. (2012) · IEEE Energy Conversion Congress and Exposition (ECCE) · doi:10.1109/ecce.2012.6342532 · Cited by 26
Littman, M. L. (2015) · Elsevier eBooks · doi:10.1016/b978-0-08-097086-8.43055-2 · Cited by 21
Ebell, N., Heinrich, F., Schlund, J., & Pruckner, M. (2018) · IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm) · doi:10.1109/smartgridcomm.2018.8587480 · Cited by 18
Nakabi, T. A., & Toivanen, P. (2019) · F1000Research · doi:10.12688/f1000research.20421.1 · Cited by 2
Self-citation
Cited by
Optimizing renewable energy systems through artificial intelligence: Review and future prospects
Ukoba, K., Olatunji, K. O., Adeoye, E., Jen, T.-C., & Madyira, D. M. (2024) · Energy & Environment · doi:10.1177/0958305x241256293 · Cited by 547
Responsive FLEXibility: A smart local energy system
Couraud, B., Andoni, M., Robu, V., Norbu, S., Chen, S., & Flynn, D. (2023) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2023.113343 · Cited by 57
Lahariya, M., Sadeghianpourhamami, N., & Develder, C. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4090508 · Cited by 6
Metadata
- Authors
- Nakabi, T. A., & Toivanen, P.
- Venue
- Sustainable Energy, Grids and Networks
- Publisher
- Elsevier BV
- Published
- 2021-03
- Volume
- 25
- Pages
- 100413
- ISSN
- 2352-4677 (ISSN-L)
- OpenAlex
- W3092305369
- Cited by
- 306
- References
- 55
- FWCI
- 14.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.