The DOI matches its Crossref record; 183 of the 183 references it cites were found in a registry or an index.
Applications of reinforcement learning in energy systems
Open accessAbstract
Energy systems undergo major transitions to facilitate the large-scale penetration of renewable energy technologies and improve efficiencies, leading to the integration of many sectors into the energy system domain. As the complexities in this domain increase, it becomes challenging to control energy flows using existing techniques based on physical models. Moreover, although data-driven models, such as reinforcement learning (RL), have gained considerable attention in many fields, a direct shift into RL is not feasible in the energy domain irrespective of the ongoing complexities. To this end, a top-down approach is used to understand this behavior by reviewing the current state of the art. We classified RL papers in the literature into seven categories based on their area of application. Subsequently, publications under each category were further examined relative to problem diversity, RL technique employed, performance improvement (compared with other white and gray box models), verification, and reproducibility; many of the articles reported a 10–20% performance improvement with the use of RL. In most studies, however, deep learning techniques and state-of-the-art actor-critic methods (e.g., twin delayed deep deterministic policy gradient and soft actor-critic) were not applied. This has remarkably hindered performance improvements and problems related to complex energy flows have not been considered. Approximately half of the publications reported the use of Q-learning. Furthermore, despite the availability of historical data in the energy system domain, batch RL algorithms have not been exploited. Emerging multi-agent RL applications may be considered as a positive development that can enable the management of complex interactions among multiple parties. Most studies lack proper benchmarking compared to model-based approaches or gray-box models, and a majority cover energy dispatch problems and building energy management. Although RL can adequately solve problems that are considerably integrated in several sectors, only a limited number of publications have discussed its broad application. The present study clearly demonstrates that even without the full utilization of RL capacity, this technique has a considerable potential in resolving the continuously increasing complexity within the energy system domain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Transforming Our World: The 2030 Agenda for Sustainable Development
(2018) · A New Era in Global Health · doi:10.1891/9780826190123.ap02 · Cited by 19,351
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 15,539
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
(2012) · Cited by 9,062
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · cited 1×
“Soft Actor Critic (SAC) SAC [57] algorithm incorporates the entropy measure of the policy into the reward to encourage exploration.”
BackgroundAddressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650 · cited 1×
“To address this issue [56], proposed a variant of DDPG built on Double Q -learning, by making the following changes: 1.”
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5086 · Cited by 3,515
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Buttler, A., & Spliethoff, H. (2018) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.09.003 · Cited by 2,510 · cited 1×
“Such an interlink, which connects several sectors, is also known as sector coupling [168].”
BackgroundDistributed generation: a definition
Ackermann, T., Andersson, G., & Söder, L. (2001) · Electric Power Systems Research · doi:10.1016/s0378-7796(01)00101-8 · Cited by 2,502
Artificial intelligence and statistics
Yu, B., & Kumbier, K. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700813 · Cited by 2,329
A Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235 · cited 1×
“However, learning in a multi-agent environment is considerably more exigent when compared to a single-agent setting because the agent has to interact with both the environment and other agents [198].”
BackgroundDeep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
From model-based control to data-driven control: Survey, classification and perspective
Hou, Z.-S., & Wang, Z. (2013) · Information Sciences · doi:10.1016/j.ins.2012.07.014 · Cited by 1,405
Tree-Based Batch Mode Reinforcement Learning
Ernst, D., Geurts, P., & Wehenkel, L. A. (2005) · Journal of Machine Learning Research · doi:10.5555/1046920.1088690 · Cited by 1,352
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Nguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Zhang, K., Yang, Z., & Başar, T. (2021) · Studies in Systems, Decision and Control · doi:10.1007/978-3-030-60990-0_12 · Cited by 1,264
Cyber–Physical System Security for the Electric Power Grid
Sridhar, S., Hahn, A., & Govindarasu, M. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2011.2165269 · Cited by 1,208
(2011) · Encyclopedia of Machine Learning · doi:10.1007/978-0-387-30164-8_588 · Cited by 950
Applications of artificial neural-networks for energy systems
Kalogirou, S. A. (2000) · Applied Energy · doi:10.1016/s0306-2619(00)00005-2 · Cited by 933
Panteli, M., & Mancarella, P. (2015) · Electric Power Systems Research · doi:10.1016/j.epsr.2015.06.012 · Cited by 932
Reinforcement learning for demand response: A review of algorithms and modeling techniques
Vázquez-Canteli, J. R., & Nagy, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.11.002 · Cited by 883
Multiagent cooperation and competition with deep reinforcement learning
Tampuu, A., Matiisen, T., Kodelja, D., Kuzovkin, I., Korjus, K., Aru, J., Aru, J., & Vicente, R. (2017) · PLOS ONE · doi:10.1371/journal.pone.0172395 · Cited by 877
Quantifying the impacts of climate change and extreme climate events on energy systems
Perera, A. T. D., Nik, V. M., Chen, D., Scartezzini, J.-L., & Hong, T. (2020) · Nature Energy · doi:10.1038/s41560-020-0558-0 · Cited by 808
Self-citationDyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 2×
“The Dyna algorithm [42] alternates between learning the model based on the gathered data that executes the current policy on the environment and improving the policy with imaginary data from the learned model.”
BackgroundMethodModeling Cyber–Physical Systems
Derler, P., Lee, E. A., & Vincentelli, A. S. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2011.2160929 · Cited by 762
Riedmiller, M. (2005) · Lecture Notes in Computer Science · doi:10.1007/11564096_32 · Cited by 757
A survey and critique of multiagent deep reinforcement learning
Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 671
On-Line Building Energy Optimization Using Deep Reinforcement Learning
Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2834219 · Cited by 649
A review of urban energy system models: Approaches, challenges and opportunities
Keirstead, J., Jennings, M., & Sivakumar, A. (2012) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2012.02.047 · Cited by 641
Reinforcement learning for building controls: The opportunities and challenges
Wang, Z., & Hong, T. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115036 · Cited by 577
Iyengar, G. N. (2005) · Mathematics of Operations Research · doi:10.1287/moor.1040.0129 · Cited by 566
Xiong, R., Cao, J., & Yu, Q. (2018) · Applied Energy · doi:10.1016/j.apenergy.2017.11.072 · Cited by 564
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01561 · Cited by 553
Robust adversarial reinforcement learning
Pinto, L., Davidson, J., Sukthankar, R., & Gupta, A. (2017) · arXiv · doi:10.48550/arxiv.1703.02702 · Cited by 507 · cited 1×
“[174], for example [175], and [176].”
Deep Reinforcement Learning for Building HVAC Control
Wei, T., Wang, Y., & Zhu, Q. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062224 · Cited by 478
Paradigm shift in urban energy systems through distributed generation: Methods and models
Manfren, M., Caputo, P., & Costa, G. (2011) · Applied Energy · doi:10.1016/j.apenergy.2010.10.018 · Cited by 437
A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach
Lu, R., Hong, S. H., & Zhang, X. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.072 · Cited by 430
Value-function reinforcement learning in Markov games
Littman, M. L. (2001) · Cognitive Systems Research · doi:10.1016/s1389-0417(01)00015-8 · Cited by 397
Incentive-based demand response for smart grid with reinforcement learning and deep neural network
Lu, R., & Hong, S. H. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.12.061 · Cited by 394
Emergent Complexity via Multi-Agent CompetitionPreprint
Bansal, T., Pachocki, J., Sidor, S., Sutskever, I., & Mordatch, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.03748 · Cited by 388
Cheng, L., & Yu, T. (2019) · International Journal of Energy Research · doi:10.1002/er.4333 · Cited by 380
FOR LEAST-SQUARES POLICY ITERATION
Schwab, D. (2016) · Cited by 372
Lange, S., Gabel, T., & Riedmiller, M. (2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3_2 · Cited by 370
Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
Kumar, A., Fu, J., Tucker, G., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00949 · Cited by 344
Zhang, Z., Chong, A., Pan, Y., Zhang, C., & Lam, K. P. (2019) · Energy and Buildings · doi:10.1016/j.enbuild.2019.07.029 · Cited by 339
Wu, Y., Tan, H., Peng, J., Zhang, H., & He, H. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.04.021 · Cited by 332
Optimal energy management strategies for energy Internet via deep reinforcement learning approach
Hua, H., Qin, Y., Hao, C., & Cao, J. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.01.145 · Cited by 329
Dynamic Energy Management System for a Smart Microgrid
Venayagamoorthy, G. K., Sharma, R. K., Gautam, P. K., & Ahmadi, A. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2514358 · Cited by 328
Li, Y., Wang, C., Li, G., Wang, J., Zhao, D., & Chen, C. (2020) · Energy Conversion and Management · doi:10.1016/j.enconman.2020.112526 · Cited by 318
DeepMDP: Learning Continuous Latent Space Models for Representation LearningPreprint
Gelada, C., Kumar, S., Buckman, J., Nachum, O., & Bellemare, M. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02736 · Cited by 300
Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning
Kim, B.-G., Zhang, Y., van der Schaar, M., & Lee, J.-W. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2495145 · Cited by 299
Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid
Foruzan, E., Soh, L.-K., & Asgarpoor, S. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2823641 · Cited by 294
Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
Zhang, K., Yang, Z., Liu, H., Zhang, T., & Başar, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08757 · Cited by 293
Optimal control of HVAC and window systems for natural ventilation through reinforcement learning
Chen, Y., Norford, L. K., Samuelson, H. W., & Malkawi, A. (2018) · Energy and Buildings · doi:10.1016/j.enbuild.2018.03.051 · Cited by 289
Liu, T., Zou, Y., Liu, D., & Sun, F. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2015.2475419 · Cited by 278
Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning
Ji, Y., Wang, J., Xu, J., Fang, X., & Zhang, H. (2019) · Energies · doi:10.3390/en12122291 · Cited by 275
Off-Policy Deep Reinforcement Learning without Exploration
Fujimoto, S., Meger, D., & Precup, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02900 · Cited by 268
Learning contact-rich manipulation skills with guided policy search
Levine, S., Wagener, N., & Abbeel, P. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7138994 · Cited by 267
Optimal Demand Response Using Device-Based Reinforcement Learning
Wen, Z., O'Neill, D., & Maei, H. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2396993 · Cited by 266
Reinforcement learning-based real-time energy management for a hybrid tracked vehicle
Zou, Y., Liu, T., Liu, D., & Sun, F. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.03.082 · Cited by 249
Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement LearningPreprint
Papoudakis, G., Christianos, F., Rahman, A., & Albrecht, S. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.04737 · Cited by 248
Model-Ensemble Trust-Region Policy Optimization
Kurutach, T., Clavera, I., Duan, Y., Tamar, A., & Abbeel, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10592 · Cited by 247
Distributed Economic Dispatch in Microgrids Based on Cooperative Reinforcement Learning
Liu, W., Zhuang, P., Liang, H., Peng, J., & Huang, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2801880 · Cited by 247
Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement LearningPreprint
Anschel, O., Baram, N., & Shimkin, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01929 · Cited by 245
Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids
Kofinas, P., Dounis, A. I., & Vouros, G. A. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.017 · Cited by 241
Park, J. Y., Dougherty, T., Fritz, H., & Nagy, Z. (2019) · Building and Environment · doi:10.1016/j.buildenv.2018.10.028 · Cited by 238 · cited 3×
“In these works, while learning the model, the transition dynamics is represented in tabular form [10,76,90,144], RF [98], NN [98], or DNN ensemble [111].”
BackgroundSun, W. Y., Jiang, N., Krishnamurthy, A., Agarwal, A., & Langford, J. C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08540 · Cited by 238
Mauree, D., Naboni, E., Coccolo, S., Perera, A. T. D., Nik, V. M., & Scartezzini, J.-L. (2019) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2019.06.005 · Cited by 237
Self-citationMavromatidis, G., Orehounig, K., & Carmeliet, J. (2018) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2018.02.021 · Cited by 235
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies
Zhang, K., Koppel, A., Zhu, H., & Başar, T. (2020) · SIAM Journal on Control and Optimization · doi:10.1137/19m1288012 · Cited by 234
A reinforcement learning framework for optimal operation and maintenance of power grids
Rocchetta, R., Bellani, L., Compare, M., Zio, E., & Patelli, E. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.03.027 · Cited by 226
Benchmarking Model-Based Reinforcement Learning
Wang, T., Bao, X., Clavera, I., Hoang, J., Wen, Y., Langlois, E., Zhang, S., Zhang, G., Abbeel, P., & Ba, J. (2019) · arXiv · doi:10.48550/arxiv.1907.02057 · Cited by 226
Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical GuaranteesPreprint
Luo, Y., Xu, H., Li, Y., Tian, Y., Darrell, T., & Ma, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03858 · Cited by 222
Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System Problem
Ernst, D., Glavic, M., Capitanescu, F., & Wehenkel, L. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007630 · Cited by 214
Perera, A. T. D., Attalage, R. A., Perera, K. K. C. K., & Dassanayake, V. P. C. (2013) · Applied Energy · doi:10.1016/j.apenergy.2013.02.049 · Cited by 210
Self-citationValladares, W., Galindo, M., Gutiérrez, J., Wu, W.-C., Liao, K.-K., Liao, J.-C., Lu, K.-C., & Wang, C.-C. (2019) · Building and Environment · doi:10.1016/j.buildenv.2019.03.038 · Cited by 200
Fusing TensorFlow with building energy simulation for intelligent energy management in smart cities
Vázquez-Canteli, J. R., Ulyanin, S., Kämpf, J., & Nagy, Z. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2018.11.021 · Cited by 195
Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning
Parisotto, E., Ba, J. L., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06342 · Cited by 186
Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning
Peng, B., Li, X., Gao, J., Liu, J., & Wong, K.-F. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1203 · Cited by 184
Wei, C., Zhang, Z., Qiao, W., & Qu, L. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2015.2420792 · Cited by 178 · cited 1×
“For its branches, Q-learning in the tabular setting is used in all domains – BEMS [78,96], dispatch [83,85], energy markets [129,132], grid [88,138], vehicle energy systems [10,142], and devices [149,151].”
MethodKar, S., Moura, J. M. F., & Poor, H. V. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2241057 · Cited by 175
Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice
Ruelens, F., Claessens, B. J., Quaiyum, S., De Schutter, B., Babuska, R., & Belmans, R. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2640184 · Cited by 173
Distral: Robust Multitask Reinforcement Learning
Teh, Y. W., Bapst, V., Czarnecki, W. M., Quan, J., Kirkpatrick, J., Hadsell, R. T., Heess, N., & Pascanu, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.04175 · Cited by 172
A review of reinforcement learning methodologies for controlling occupant comfort in buildings
Han, M., May, R., Zhang, X., Wang, X., Pan, S., Yan, D., Jin, Y., & Xu, L. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2019.101748 · Cited by 162
Simulating Cyber-Physical Energy Systems: Challenges, Tools and Methods
Palensky, P., Widl, E., & Elsheikh, A. (2014) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmcc.2013.2265739 · Cited by 160
Zhou, Q., Li, J., Shuai, B., Williams, H., He, Y., Li, Z., Xu, H., & Yan, F. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.113755 · Cited by 158
Wang, Y., Velswamy, K., & Huang, B. (2017) · Processes · doi:10.3390/pr5030046 · Cited by 157
Stochastic Variance-Reduced Policy GradientPreprint
Papini, M., Binaghi, D., Canonaco, G., Pirotta, M., & Restelli, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05618 · Cited by 157
Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning
Liu, T., Wang, B., & Yang, C. (2018) · Energy · doi:10.1016/j.energy.2018.07.022 · Cited by 146
Lee, S., & Choi, D. (2019) · Sensors · doi:10.3390/s19183937 · Cited by 141
Wang, Y., Lin, X., & Pedram, M. (2015) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2015.2467190 · Cited by 138
A reinforcement learning approach for MPPT control method of photovoltaic sources
Kofinas, P., Doltsinis, S. C., Dounis, A. I., & Vouros, G. A. (2017) · Renewable Energy · doi:10.1016/j.renene.2017.03.008 · Cited by 136
Sigaud, O., & Garçia, F. (2013) · Wiley eBooks · doi:10.1002/9781118557426.ch2 · Cited by 133
Satisfaction based Q-learning for integrated lighting and blind control
Cheng, Z., Zhao, Q., Wang, F., Jiang, Y., Xia, L., & Ding, J. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.05.067 · Cited by 132
Perera, A. T. D., Nik, V. M., Mauree, D., & Scartezzini, J. (2017) · Applied Energy · doi:10.1016/j.apenergy.2016.12.127 · Cited by 123
Self-citationLocal Energy Trading Behavior Modeling With Deep Reinforcement Learning
Chen, T., & Su, W. (2018) · IEEE Access · doi:10.1109/access.2018.2876652 · Cited by 121
2014 IEEE Innovative Smart Grid Technologies - Asia (ISGT ASIA)
(2014) · doi:10.1109/isgtasia31837.2014 · Cited by 121
Yu, T., Wang, H. Z., Zhou, B., Chan, K. W., & Tang, J. (2014) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2014.2357079 · Cited by 120
Multi-agent reinforcement learning for modeling and control of thermostatically controlled loads
Kazmi, H., Suykens, J. A. K., Bálint, A., & Driesen, J. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.01.140 · Cited by 110
Kazmi, H., Mehmood, F., Lodeweyckx, S., & Driesen, J. (2018) · Energy · doi:10.1016/j.energy.2017.12.019 · Cited by 109
Action robust reinforcement learning and applications in continuous control
Tessler, C., Efroni, Y., & Mannor, S. (2019) · arXiv · doi:10.48550/arxiv.1901.09184 · Cited by 108
Qi, X., Wu, G., Boriboonsomsin, K., et al. (2016) · Transportation Research Record Journal of the Transportation Research Board · doi:10.3141/2572-01 · Cited by 108
Yuan, J., Yang, L., & Chen, Q. (2018) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2018.03.033 · Cited by 108
Combined Reinforcement Learning via Abstract Representations
François-Lavet, V., Bengio, Y., Precup, D., & Pineau, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013582 · Cited by 108
Artificial Neural Network Based Reinforcement Learning for Wind Turbine Yaw Control
Saenz‐Aguirre, A., Zulueta, E., Fernández‐Gámiz, U., Lozano, J., & López-Guede, J. M. (2019) · Energies · doi:10.3390/en12030436 · Cited by 108
Reinforcement Learning–Based Energy Management Strategy for a Hybrid Electric Tracked Vehicle
Liu, T., Zou, Y., Liu, D., & Sun, F. (2015) · Energies · doi:10.3390/en8077243 · Cited by 107
Evaluation of Reinforcement Learning Control for Thermal Energy Storage Systems
Henze, G. P., & Schoenmann, J. (2003) · Science and Technology for the Built Environment · doi:10.1080/10789669.2003.10391069 · Cited by 107
A reinforcement learning approach to autonomous decision-making in smart electricity markets
Peters, M., Ketter, W., Saar‐Tsechansky, M., & Collins, J. E. (2013) · Machine Learning · doi:10.1007/s10994-013-5340-0 · Cited by 106
Remani, T., Jasmin, E. A., & Ahamed, T. P. I. (2019) · IEEE Systems Journal · doi:10.1109/jsyst.2018.2855689 · Cited by 106
Robust Reinforcement Learning for Continuous Control with Model MisspecificationPreprint
Mankowitz, D. J., Levine, N., Jeong, R., Shi, Y., Kay, J., Abdolmaleki, A., Springenberg, J. T., Mann, T., Hester, T., & Riedmiller, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07516 · Cited by 105
Convergence Rate of Distributed Averaging Dynamics and Optimization in Networks
Nedić, A. (2015) · Foundations and Trends® in Systems and Control · doi:10.1561/2600000004 · Cited by 105
A Geometric Perspective on Optimal Representations for Reinforcement LearningPreprint
Bellemare, M. G., Dabney, W., Dadashi, R., Taïga, A. A., Castro, P. S., Roux, N. L., Schuurmans, D., Lattimore, T., & Lyle, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11530 · Cited by 103
2018 IEEE International Energy Conference (ENERGYCON)
(2018) · doi:10.1109/energycon42420.2018 · Cited by 100
Liu, T., Xu, C., Guo, Y., & Chen, H. (2019) · International Journal of Refrigeration · doi:10.1016/j.ijrefrig.2019.07.018 · Cited by 97
Heo, S., Nam, K., Loy-Benitez, J., Li, Q., Lee, S.-C., & Yoo, C. (2019) · Energy and Buildings · doi:10.1016/j.enbuild.2019.109440 · Cited by 90
Model-Augmented Actor-Critic: Backpropagating through PathsPreprint
Clavera, I., Fu, V., & Abbeel, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.08068 · Cited by 89
Introducing reinforcement learning to the energy system design process
Perera, A. T. D., Wickramasinghe, P. U., Nik, V. M., & Scartezzini, J. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.114580 · Cited by 86
Self-citationMobility-Aware Vehicle-to-Grid Control Algorithm in Microgrids
Ko, H., Pack, S., & Leung, V. C. M. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2816935 · Cited by 83
Advanced Building Control via Deep Reinforcement Learning
Jia, R., Jin, M., Sun, K., Hong, T., & Spanos, C. J. (2019) · Energy Procedia · doi:10.1016/j.egypro.2019.01.494 · Cited by 83
Ruelens, F., Iacovella, S., Claessens, B., & Belmans, R. J. M. (2015) · Energies · doi:10.3390/en8088300 · Cited by 82
Heterogeneous Energy Storage Optimization for Microgrids
Qiu, X., Nguyen, T. A., & Crow, M. L. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2461134 · Cited by 82
Zhang, X., Li, S., He, T., Yang, B. F., Yu, T., Li, H., Jiang, L., & Sun, L. (2019) · Energy · doi:10.1016/j.energy.2019.03.053 · Cited by 81
Online Optimization With Predictions and Switching Costs: Fast Algorithms and the Fundamental Limit
Li, Y., Qu, G., & Li, N. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3040249 · Cited by 81
Generalizable occupant-driven optimization model for domestic hot water production in NZEB
Kazmi, H., D’Oca, S., Delmastro, C., Lodeweyckx, S., & Corgnati, S. P. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.04.108 · Cited by 80
Reddy, N. P., Pasdeloup, D., Zadeh, M., & Skjetne, R. (2019) · IEEE Transportation Electrification Conference and Expo · doi:10.1109/itec.2019.8790451 · Cited by 80
Model-free control of thermostatically controlled loads connected to a district heating network
Claessens, B., Vanhoudt, D., Desmedt, J., et al. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.08.052 · Cited by 77
Learning Contact-Rich Manipulation Skills with Guided Policy SearchPreprint
Levine, S., Wagener, N., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.05611 · Cited by 77
Control of a PCM ventilated facade using reinforcement learning techniques
de Gracia, Á., Fernández, C. C., Castell, A., Mateu, C., & Cabeza, L. F. (2015) · Energy and Buildings · doi:10.1016/j.enbuild.2015.06.045 · Cited by 71
Online convex optimization for cumulative constraintsPreprint
Yuan, J., & Lamperski, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.06472 · Cited by 69
Zio, E., & Sansavini, G. (2013) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmca.2012.2207106 · Cited by 68
Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement LearningPreprint
Chen, Y., & Wang, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.02516 · Cited by 68
Robust Reinforcement Learning via Adversarial training with Langevin DynamicsPreprint
Kamalaruban, P., Huang, Y., Hsieh, Y.-P., Rolland, P., Shi, C., & Cevher, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06063 · Cited by 66
Self-citationA Q-Learning Based Charging Scheduling Scheme for Electric Vehicles
Dang, Q., Wu, D., & Boulet, B. (2019) · IEEE Transportation Electrification Conference and Expo · doi:10.1109/itec.2019.8790603 · Cited by 64
Sparse multi-task reinforcement learning
Calandriello, D., Lazaric, A., & Restelli, M. (2015) · Intelligenza Artificiale · doi:10.3233/ia-150080 · Cited by 63
Mohajeri, N., Perera, A. T. D., Coccolo, S., Mosca, L., Guen, M. L., & Scartezzini, J. (2019) · Renewable Energy · doi:10.1016/j.renene.2019.05.033 · Cited by 60
Self-citationGuan, C., Wang, Y., Lin, X., Nazarian, S., & Pedram, M. (2015) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc.2015.7158054 · Cited by 55
Schmidt, M., Moreno, M. V., Schülke, A., Macek, K., Mařík, K., & Pastor, A. G. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.05.002 · Cited by 55
Yu, Z., & Dexter, A. L. (2010) · Control Engineering Practice · doi:10.1016/j.conengprac.2010.01.018 · Cited by 52
Sistani, M. B. N., Akbarzadeh-Tootoonchi, M.-R., Bayaz, M. H. J.-D., & Mashhadi, H. R. (2005) · Energy Conversion and Management · doi:10.1016/j.enconman.2005.08.012 · Cited by 50
Striving for Simplicity in Off-policy Deep Reinforcement LearningPreprint
Agarwal, R., Schuurmans, D., & Norouzi, M. (2019) · arXiv (Cornell University) · Cited by 48
Value-Aware Loss Function for Model-based Reinforcement Learning
Farahmand, A. M., Barreto, A. D. M. S., & Nikovski, D. N. (2017) · PolyPublie (École Polytechnique de Montréal) · Cited by 46
Control of a Realistic Wave Energy Converter Model Using Least-Squares Policy Iteration
Anderlini, E., Forehand, D. I. M., Bannon, E., & Abusara, M. A. (2017) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2017.2696060 · Cited by 40
Dynamic load management for a residential customer; Reinforcement Learning approach
Sheikhi, A., Rayati, M., & Ranjbar, A. M. (2016) · Sustainable Cities and Society · doi:10.1016/j.scs.2016.04.001 · Cited by 38
Sun, J., Zhu, Z., Li, H., Chai, Y., Qi, G., Wang, H., & Hu, Y. H. (2019) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2019.04.011 · Cited by 37
Battery Scheduling in a Residential Multi-Carrier Energy System Using Reinforcement Learning
Mbuwir, B. V., Kaffash, M., & Deconinck, G. (2018) · IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm) · doi:10.1109/smartgridcomm.2018.8587412 · Cited by 35
Zarrabian, S., Belkacemi, R., & Babalola, A. A. (2016) · Electric Power Systems Research · doi:10.1016/j.epsr.2016.06.041 · Cited by 34
Deep Residual Reinforcement Learning
Zhang, S., Böhmer, W., & Whiteson, S. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/icps4845 · Cited by 34
Cao, J., Zhang, W., Xiao, Z., & Hua, H. (2019) · Energies · doi:10.3390/en12081556 · Cited by 33
Policy-Aware Model Learning for Policy Gradient MethodsPreprint
Abachi, R., Ghavamzadeh, M., & Farahmand, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00030 · Cited by 28
Reinforcement learning control approach for autonomous microgrids
Mahmoud, M. S., Abouheaf, M., & Sharaf, A. Μ. (2019) · International Journal of Modelling and Simulation · doi:10.1080/02286203.2019.1655701 · Cited by 26
An adaptive multi-agent-based approach to smart grids control and optimization
Carvalho, M., Perez, C., & Granados, A. (2012) · Energy Systems · doi:10.1007/s12667-012-0054-0 · Cited by 24
Multiagent Stochastic Dynamic Game for Smart Generation Control
Yu, T., Xi, L., Yang, B. F., Xu, Z., & Jiang, L. (2015) · Journal of Energy Engineering · doi:10.1061/(asce)ey.1943-7897.0000275 · Cited by 22
Adaptive control for building energy management using reinforcement learning
Eller, L., Siafara, L. C., & Sauter, T. (2018) · IEEE International Conference on Industrial Technology (ICIT) · doi:10.1109/icit.2018.8352414 · Cited by 22
Intelligent Decision-Making for Smart Home Energy Management
Berlink, H., Kagan, N., & Costa, A. H. R. (2014) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-014-0169-8 · Cited by 21
Deep Robust Kalman FilterPreprint
Shashua, S. D.-C., & Mannor, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02310 · Cited by 20
Poudyal, R. L., Kobayashi, R., Suzuki, T., & Watanabe, M. (2019) · International Journal of Refrigeration · doi:10.1016/j.ijrefrig.2019.07.022 · Cited by 20
Multi-objective energy management for we-energy in Energy Internet using reinforcement learning
Sun, Q., Wang, D., Ma, D., & Huang, B. (2017) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2017.8285243 · Cited by 19
Odonkor, P., & Lewis, K. E. (2018) · Journal of Mechanical Design · doi:10.1115/1.4041629 · Cited by 19
Learning policies for battery usage optimization in electric vehicles
Ermon, S., Xue, Y., Gomes, C. P., & Selman, B. (2013) · Machine Learning · doi:10.1007/s10994-013-5378-z · Cited by 18
Balancing Energy in the Smart Grid Using Distributed Value Function (DVF)
Shirzeh, H., Naghdy, F., Ciufo, P., & Ros, M. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2363844 · Cited by 18
Stochastic Primal-Dual Q-LearningPreprint
Jeong, N., Lee, D., & He, N. (2018) · arXiv (Cornell University) · doi:10.2139/ssrn.5403129 · Cited by 16
Bollenbacher, J., & Rhein, B. (2017) · International Energy and Sustainability Conference (IESC) · doi:10.1109/iesc.2017.8167476 · Cited by 10
Coordinated Multi-Agent Reinforcement Learning for Swarm Battery Control
Ebell, N., & Pruckner, M. (2018) · Conference proceedings - Canadian Conference on Electrical and Computer Engineering · doi:10.1109/ccece.2018.8447851 · Cited by 8
Online reinforcement learning in multi-agent systems for distributed energy systems
Radhakrishnan, B. M., Menon, S. B., Srinivasan, D., & Jain, L. C. (2014) · IEEE Innovative Smart Grid Technologies-Asia · doi:10.1109/isgt-asia.2014.6873894 · Cited by 7
Distributed Multitask Reinforcement Learning with Quadratic Convergence
Tutunov, R., Kim, D., & Ammar, H. B. (2018) · neural information processing systems · Cited by 6
Online convex optimization for demand response
Lesage‐Landry, A., & Taylor, J. A. (2017) · PolyPublie (École Polytechnique de Montréal) · Cited by 3
Sample-Efficient Model-based Actor-Critic for an Interactive Dialogue TaskPreprint
Kudashkina, K., Chockalingam, V., Taylor, G. W., & Bowling, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13657 · Cited by 2
Cited by
Ahmad, T., Madonski, R., Zhang, D., Huang, C., & Mujeeb, A. (2022) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2022.112128 · Cited by 711
Benti, N. E., Chaka, M. D., & Semie, A. G. (2023) · Sustainability · doi:10.3390/su15097087 · Cited by 378
Machine Learning and Deep Learning in Energy Systems: A Review
Forootan, M. M., Larki, I., Zahedi, R., & Ahmadi, A. (2022) · Sustainability · doi:10.3390/su14084832 · Cited by 343
An overview of machine learning applications for smart buildings
Alanne, K., & Sierla, S. (2022) · Sustainable Cities and Society · doi:10.1016/j.scs.2021.103445 · Cited by 280
Evaluation of advanced control strategies for building energy systems
Stoffel, P., Maier, L., Kümpel, A., Schreiber, T., & Müller, D. (2023) · Energy and Buildings · doi:10.1016/j.enbuild.2022.112709 · Cited by 126
“The following comprehensive review articles are recommended to the interested reader ([80– 82]).”
Metadata
- Authors
- Perera, A. T. D., & Kamalaruban, P.
- Venue
- Renewable and Sustainable Energy Reviews
- Publisher
- Elsevier BV
- Published
- 2021-03
- Volume
- 137
- Pages
- 110618
- ISSN
- 1364-0321 (ISSN-L)
- OpenAlex
- W3112388352
- Cited by
- 420
- References
- 183
- FWCI
- 14.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.