Reinforcement learning for demand response: A review of algorithms and modeling techniques

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works

Reinforcement Learning: An Introduction

Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works

Technical Note: Q-Learning

Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277 · see citing works

Q-learning

Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works

Understanding Smart Cities: An Integrative Framework

Chourabi, H., Nam, T., Walker, S., Gil-Garcia, J. R., Mellouli, S., Nahon, K., Pardo, T. A., & Scholl, H. J. (2012) · 2012 45th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2012.615 · Cited by 2,599 · see citing works

Demand response and smart grids – a survey

Siano, P. (2014) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2013.10.022 · Cited by 2,411 · see citing works

A global review of energy consumption, CO 2 emissions and policy in the residential sector (with an overview of the top ten CO 2 emitting countries)

Nejat, P., Jomehzadeh, F., Taheri, M. M., Gohari, M., & Majid, M. Z. A. (2014) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2014.11.066 · Cited by 1,951 · see citing works

Classifier Fitness Based on Accuracy

Wilson, S. W. (1995) · Evolutionary Computation · doi:10.1162/evco.1995.3.2.149 · Cited by 1,395 · see citing works

A Survey on Demand Response Programs in Smart Grids: Pricing Methods and Optimization Algorithms

Vardakas, J. S., Zorba, N., & Verikoukis, C. V. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2341586 · Cited by 951 · see citing works

Demand response in smart electricity grids equipped with renewable energy sources: A review

Aghaei, J., & Alizadeh, M. I. (2012) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2012.09.019 · Cited by 689 · see citing works

Review of building energy modeling for control and operation

Li, X., & Wen, J. (2014) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2014.05.056 · Cited by 632 · see citing works

Demand side management in smart grid: A review and proposals for future direction

Gelažanskas, L., & Gamage, K. A. A. (2013) · Sustainable Cities and Society · doi:10.1016/j.scs.2013.11.001 · Cited by 624 · see citing works

Market Structure and Equilibrium

von Stackelberg, H. (2010) · Springer eBooks · doi:10.1007/978-3-642-12586-7 · Cited by 596 · see citing works

Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle

Xiong, R., Cao, J., & Yu, Q. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.11.072 · Cited by 566 · see citing works

An Application of Reinforcement Learning to Aerobatic Helicopter Flight

Abbeel, P., Coates, A., Quigley, M., & Ng, A. Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0006 · Cited by 551 · see citing works

Review and prospect of integrated demand response in the multi-energy system

Wang, J., Zhong, H., Ma, Z., Xia, Q. L., & Kang, C. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.05.150 · Cited by 522 · see citing works

The Neural Network House: An Environment that Adapts to its Inhabitants

Mozer, M. C. (1998) · Cited by 500 · see citing works

Supervisory and Optimal Control of Building HVAC Systems: A Review

Wang, S., & Ma, Z. (2008) · Science and Technology for the Built Environment · doi:10.1080/10789669.2008.10390991 · Cited by 499 · see citing works

On the Complexity of Solving Markov Decision ProblemsPreprint

Littman, M. L., Dean, T., & Kaelbling, L. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.4971 · Cited by 434 · see citing works

Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus

Wu, J., He, H., Peng, J., Li, Y., & Li, Z. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.104 · Cited by 434 · see citing works

Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle

Liu, T., Hu, X., Li, S. E., & Cao, D. (2017) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2707338 · Cited by 402 · see citing works

PAC model-free reinforcement learning

Strehl, A. L., Li, L. C., Wiewiora, E., Langford, J. C., & Littman, M. L. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143955 · Cited by 391 · see citing works

Demand response for sustainable energy systems: A review, application and implementation strategy

Shariatzadeh, F., Mandal, P., & Srivastava, A. K. (2015) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2015.01.062 · Cited by 354 · see citing works

Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning

Ruelens, F., Claessens, B., Vandael, S., Schutter, B. D., Babuška, R., & Belmans, R. J. M. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2517211 · Cited by 339 · see citing works

A survey of industrial applications of Demand Response

Shoreh, M. H., Siano, P., Shafie-khah, M., Loia, V., & Catalão, J. P. S. (2016) · Electric Power Systems Research · doi:10.1016/j.epsr.2016.07.008 · Cited by 302 · see citing works

Comprehensive analysis of the relationship between thermal comfort and building control research - A data-driven literature review

Park, J. Y., & Nagy, Z. (2017) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.09.102 · Cited by 301 · see citing works

Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning

Kim, B., Zhang, Y., van der Schaar, M., & Lee, J.-W. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2495145 · Cited by 299 · see citing works

A stochastic reinforcement learning algorithm for learning real-valued functions

Gullapalli, V. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90056-q · Cited by 292 · see citing works

Residential Demand Response Using Reinforcement Learning

O’Neill, D. C., Levorato, M., Goldsmith, A. J., et al. (2010) · IEEE International Conference on Smart Grid Communications (SmartGridComm) · doi:10.1109/smartgrid.2010.5622078 · Cited by 290 · see citing works

Incremental multi-step Q-learning

Peng, J., & Williams, R. J. (1996) · Machine Learning · doi:10.1007/bf00114731 · Cited by 289 · see citing works

Optimal control of HVAC and window systems for natural ventilation through reinforcement learning

Chen, Y., Norford, L. K., Samuelson, H. W., & Malkawi, A. M. (2018) · Energy and Buildings · doi:10.1016/j.enbuild.2018.03.051 · Cited by 289 · see citing works

Augmenting Image Classifiers Using Data Augmentation Generative Adversarial Networks

Antoniou, A., Storkey, A., & Edwards, H. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01424-7_58 · Cited by 282 · see citing works

Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning

Hu, Y., Li, W., Xu, K., Zahid, T., Qin, F., & Li, C. (2018) · Applied Sciences · doi:10.3390/app8020187 · Cited by 276 · see citing works

Optimal Demand Response Using Device-Based Reinforcement Learning

Wen, Z., O’Neill, D. C., & Maei, H. R. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2396993 · Cited by 271 · see citing works

Reinforcement learning-based real-time energy management for a hybrid tracked vehicle

Zou, Y., Liu, T., Liu, D., et al. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.03.082 · Cited by 253 · see citing works

Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids

Kofinas, P., Dounis, A. I., & Vouros, G. A. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.017 · Cited by 242 · see citing works

LightLearn: An adaptive and occupant centered controller for lighting based on reinforcement learning

Park, J. Y., Dougherty, T., Fritz, H., & Nagy, Z. (2019) · Building and Environment · doi:10.1016/j.buildenv.2018.10.028 · Cited by 238 · see citing works

An exploratory analysis of California residential customer response to critical peak pricing of electricity

Herter, K., McAuliffe, P., & Rosenfeld, A. H. (2006) · Energy · doi:10.1016/j.energy.2006.01.014 · Cited by 231 · see citing works

A novel dual iterative Q-learning method for optimal battery management in smart residential environments

Wei, Q., Liu, D., & Shi, G. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2361485 · Cited by 231 · see citing works

Machine Learning: ECML 2004

FANIZZI, N., SEMERARO, G., IANNONE, L., & Palmisano, I. (2004) · Lecture notes in computer science · doi:10.1007/b100702 · Cited by 217 · see citing works

Reinforcement Learning of Heuristic EV Fleet Charging in a Day-Ahead Electricity Market

Vandael, S., Claessens, B., Ernst, D., Holvoet, T., & Deconinck, G. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2393059 · Cited by 209 · see citing works

Reinforcement learning for energy conservation and comfort in buildings

Dalamagkidis, K., Kolokotsa, D. D., Kalaitzakis, K. C., & Stavrakakis, G. S. (2006) · Building and Environment · doi:10.1016/j.buildenv.2006.07.010 · Cited by 203 · see citing works

GridLAB-D: An Agent-Based Simulation Framework for Smart Grids

Chassin, D. P., Fuller, J., & Djilali, N. (2014) · Journal of Applied Mathematics · doi:10.1155/2014/492320 · Cited by 202 · see citing works

Action Selection Methods using Reinforcement Learning

Humphrys, M. (1996) · The MIT Press eBooks · doi:10.7551/mitpress/3118.003.0018 · Cited by 197 · see citing works

Fusing TensorFlow with building energy simulation for intelligent energy management in smart cities

Vázquez-Canteli, J. R., Ulyanin, S., Kämpf, J., & Nagy, Z. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2018.11.021 · Cited by 195 · see citing works

Battery Energy Management in a Microgrid Using Batch Reinforcement Learning

Mbuwir, B. V., Ruelens, F., Spiessens, F., & Deconinck, G. (2017) · Energies · doi:10.3390/en10111846 · Cited by 187 · see citing works

Reinforcement learning for optimal control of low exergy buildings

Yang, L., Nagy, Z., Goffin, P., & Schlueter, A. (2015) · Applied Energy · doi:10.1016/j.apenergy.2015.07.050 · Cited by 183 · see citing works

Multiagent Learning: Basics, Challenges, and Prospects

Tuyls, K., & Weiß, G. (2012) · AI Magazine · doi:10.1609/aimag.v33i3.2426 · Cited by 180 · see citing works

Reinforcement Learning-Based Plug-in Electric Vehicle Charging with Forecasted Price

Chis, A., Lundén, J., & Koivunen, V. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2603536 · Cited by 180 · see citing works

Transition to Sustainable Buildings

International Energy Agency (2013) · OECD eBooks · doi:10.1787/9789264202955-en · Cited by 180 · see citing works

Residential Demand Response model and impact on voltage profile and losses of an electric distribution network

Venkatesan, N., Solanki, J. M., & Solanki, S. K. (2012) · Applied Energy · doi:10.1016/j.apenergy.2011.12.076 · Cited by 175 · see citing works

Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice

Ruelens, F., Claessens, B., Quaiyum, S., Schutter, B. D., Babuška, R., & Belmans, R. J. M. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2640184 · Cited by 174 · see citing works

Dynamic Pricing Design for Demand Response Integration in Power Distribution Networks

Nguyen, D. T., Nguyen, H. T., & Le, L. B. (2016) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2015.2510612 · Cited by 167 · see citing works

Coordination of Energy Efficiency and Demand Response

Goldman, C. A., Reid, M., Levy, R., et al. (2010) · doi:10.2172/981732 · Cited by 165 · see citing works

Convolutional Neural Networks for Automatic State-Time Feature Extraction in Reinforcement Learning Applied to Residential Load Control

Claessens, B., Vrancx, P., & Ruelens, F. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2629450 · Cited by 157 · see citing works

A Long-Short Term Memory Recurrent Neural Network Based Reinforcement Learning Controller for Office Heating Ventilation and Air Conditioning Systems

Wang, Y., Velswamy, K., & Huang, B. (2017) · Processes · doi:10.3390/pr5030046 · Cited by 153 · see citing works

Electricity Cost Minimization for a Microgrid With Distributed Energy Resource Under Different Information Availability

Liu, Y., Yuen, C., Hassan, N. U., et al. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2371780 · Cited by 152 · see citing works

Action dependent heuristic dynamic programming for home energy resource scheduling

Fuselli, D., Angelis, F. D., Boaro, M., Squartini, S., Wei, Q., Liu, D., & Piazza, F. (2013) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2012.11.023 · Cited by 149 · see citing works

Distributed Coverage Games for Energy-Aware Mobile Sensor Networks

Zhu, M., & Martı́nez, S. (2013) · SIAM Journal on Control and Optimization · doi:10.1137/100784163 · Cited by 148 · see citing works

An Online Learning Algorithm for Demand Response in Smart Grid

Bahrami, S., Wong, V. W. S., & Huang, J. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2667599 · Cited by 147 · see citing works

Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning

Liu, T., Wang, B., & Yang, C. (2018) · Energy · doi:10.1016/j.energy.2018.07.022 · Cited by 146 · see citing works

Smart Home in Smart Microgrid: A Cost-Effective Energy Ecosystem With Intelligent Hierarchical Agents

Jiang, B., & Fei, Y. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2347043 · Cited by 145 · see citing works

Optimal decarbonization pathways for urban residential building energy services

Leibowicz, B. D., Lanham, C. M., Brozynski, M. T., Vázquez-Canteli, J. R., Castejón, N. C., & Nagy, Z. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.09.046 · Cited by 142 · see citing works

Energy Management in Smart Cities Based on Internet of Things: Peak Demand Reduction and Energy Savings

Mahapatra, C., Moharana, A. K., & Leung, V. C. M. (2017) · Sensors · doi:10.3390/s17122812 · Cited by 139 · see citing works

Autonomous HVAC Control, A Reinforcement Learning Approach

Barrett, E., & Linder, S. P. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23461-8_1 · Cited by 135 · see citing works

Real-time vehicle-to-grid control algorithm under price uncertainty

Shi, W., & Wong, V. W. S. (2011) · IEEE International Conference on Smart Grid Communications (SmartGridComm) · doi:10.1109/smartgridcomm.2011.6102330 · Cited by 135 · see citing works

A Near-Optimal Model-Based Control Algorithm for Households Equipped With Residential Photovoltaic Power Generation and Energy Storage Systems

Wang, Y., Lin, X., & Pedram, M. (2015) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2015.2467190 · Cited by 132 · see citing works

Satisfaction based Q-learning for integrated lighting and blind control

Cheng, Z., Zhao, Q., Wang, F., Jiang, Y., Xia, L., & Ding, J. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.05.067 · Cited by 132 · see citing works

Future Air Conditioning Energy Consumption in Developing Countries and what can be done about it: The Potential of Efficiency in the Residential Sector

McNeil, M. A., & Letschert, V. E. (2008) · Lawrence Berkeley National Laboratory · Cited by 130 · see citing works

Experimental analysis of data-driven control for a building heating system

Costanzo, G. T., Iacovella, S., Ruelens, F., Leurs, T., & Claessens, B. (2016) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2016.02.002 · Cited by 126 · see citing works

Demand Response Architecture: Integration into the Distribution Management System

Mohagheghi, S., Stoupis, J., Wang, Z., et al. (2010) · IEEE International Conference on Smart Grid Communications (SmartGridComm) · doi:10.1109/smartgrid.2010.5622094 · Cited by 122 · see citing works

Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory

Liu, S., & Henze, G. P. (2005) · Energy and Buildings · doi:10.1016/j.enbuild.2005.06.001 · Cited by 120 · see citing works

Battery and ultracapacitor in-the-loop approach to validate a real-time power management method for an all-climate electric vehicle

Xiong, R., Duan, Y., Cao, J., & Yu, Q. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.02.128 · Cited by 118 · see citing works

Impact of residential demand response on power system operation: A Belgian case study

Dupont, B., Dietrich, K., Jonghe, C. D., Ramos, A., & Belmans, R. J. M. (2014) · Applied Energy · doi:10.1016/j.apenergy.2014.02.022 · Cited by 116 · see citing works

Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory

Liu, S., & Henze, G. P. (2005) · Energy and Buildings · doi:10.1016/j.enbuild.2005.06.002 · Cited by 115 · see citing works

Gigawatt-hour scale savings on a budget of zero: Deep reinforcement learning based optimal control of hot water systems

Kazmi, H., Mehmood, F., Lodeweyckx, S., & Driesen, J. (2018) · Energy · doi:10.1016/j.energy.2017.12.019 · Cited by 109 · see citing works

Data-Driven Reinforcement Learning–Based Real-Time Energy Management System for Plug-In Hybrid Electric Vehicles

Qi, X., Wu, G., Boriboonsomsin, K., et al. (2016) · Transportation Research Record Journal of the Transportation Research Board · doi:10.3141/2572-01 · Cited by 108 · see citing works

Intelligent energy management strategy based on hierarchical approximate global optimization for plug-in fuel cell hybrid electric vehicles

Yuan, J., Yang, L., & Chen, Q. (2018) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2018.03.033 · Cited by 108 · see citing works

Demand side management for a residential customer in multi-energy systems

Sheikhi, A., Rayati, M., & Ranjbar, A. M. (2016) · Sustainable Cities and Society · doi:10.1016/j.scs.2016.01.010 · Cited by 106 · see citing works

Evaluation of Reinforcement Learning Control for Thermal Energy Storage Systems

Henze, G. P., & Schoenmann, J. (2003) · Science and Technology for the Built Environment · doi:10.1080/10789669.2003.10391069 · Cited by 102 · see citing works

A Bi-Level Control for Energy Efficiency Improvement of a Hybrid Tracked Vehicle

Liu, T., & Hu, X. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2797322 · Cited by 98 · see citing works

Control strategies for integration of thermal energy storage into buildings: State-of-the-art review

Yu, Z. J., Huang, G., Haghighat, F., et al. (2015) · Energy and Buildings · doi:10.1016/j.enbuild.2015.05.038 · Cited by 95 · see citing works

A Review on Demand Response: Pricing, Optimization, and Appliance Scheduling

Hussain, I., Mohsin, S., Basit, A., et al. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.05.141 · Cited by 88 · see citing works

Dynamic Residential Demand Response and Distributed Generation Management in Smart Microgrid with Hierarchical Agents

Jiang, B., & Fei, Y. (2011) · Energy Procedia · doi:10.1016/j.egypro.2011.10.012 · Cited by 86 · see citing works

A verification of CitySim results using the BESTEST and monitored consumption values

Walter, E., & Kämpf, J. H. (2015) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 85 · see citing works

Heterogeneous Energy Storage Optimization for Microgrids

Qiu, X., Nguyen, T. A., & Crow, M. L. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2461134 · Cited by 82 · see citing works

Demand response of a heterogeneous cluster of electric water heaters using batch reinforcement learning

Ruelens, F., Claessens, B., Vandael, S., et al. (2014) · Power Systems Computation Conference · doi:10.1109/pscc.2014.7038106 · Cited by 81 · see citing works

Generalizable occupant-driven optimization model for domestic hot water production in NZEB

Kazmi, H., D’Oca, S., Delmastro, C., Lodeweyckx, S., & Corgnati, S. P. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.04.108 · Cited by 80 · see citing works

Model-free control of thermostatically controlled loads connected to a district heating network

Claessens, B., Vanhoudt, D., Desmedt, J., et al. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.08.052 · Cited by 77 · see citing works

Using reinforcement learning for demand response of domestic hot water buffers: A real-life demonstration

Somer, O. D., Soares, A., Vanthournout, K., Spiessens, F., Kuijpers, T., & Vossen, K. (2017) · IEEE PES Innovative Smart Grid Technologies Europe · doi:10.1109/isgteurope.2017.8260152 · Cited by 77 · see citing works

Mobility-Aware Vehicle-to-Grid Control Algorithm in Microgrids

Ko, H., Pack, S., & Leung, V. C. M. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2816935 · Cited by 76 · see citing works

Learning Agent for a Heat-Pump Thermostat with a Set-Back Strategy Using Model-Free Reinforcement Learning

Ruelens, F., Iacovella, S., Claessens, B., & Belmans, R. J. M. (2015) · Energies · doi:10.3390/en8088300 · Cited by 74 · see citing works

A multi-grid reinforcement learning method for energy conservation and comfort of HVAC in buildings

Li, B., & Xia, L. (2015) · 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2015.7294119 · Cited by 74 · see citing works

Prediction-Based Multi-Agent Reinforcement Learning in Inherently Non-Stationary Environments

Marinescu, A., Dusparić, I., & Clarke, S. (2017) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3070861 · Cited by 71 · see citing works

Demand response with locational dynamic pricing to support the integration of renewables

Dupont, B., Jonghe, C. D., Olmos, L., & Belmans, R. J. M. (2014) · Energy Policy · doi:10.1016/j.enpol.2013.12.058 · Cited by 70 · see citing works

Machine-Learning Aided Optimal Customer Decisions for an Interactive Smart Grid

Li, D., & Jayaweera, S. K. (2014) · IEEE Systems Journal · doi:10.1109/jsyst.2014.2334637 · Cited by 70 · see citing works

Deep transfer Q-learning with virtual leader-follower for supply-demand Stackelberg game of smart grid

Zhang, X., Bao, T., Yu, T., et al. (2017) · Energy · doi:10.1016/j.energy.2017.05.114 · Cited by 69 · see citing works

Impact of behavior-driven demand response on supply adequacy in smart distribution systems

Zeng, B., Wu, G., Wang, J., et al. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.05.098 · Cited by 69 · see citing works

Demand Response Architectures and Load Management Algorithms for Energy-Efficient Power Grids: A Survey

Law, Y. W., Alpcan, T., Lee, V. C. S., et al. (2012) · doi:10.1109/kicss.2012.45 · Cited by 68 · see citing works

Enabling Cooperative Behavior for Building Demand Response Based on Extended Joint Action Learning

Hurtado, L. A., Mocanu, E., Nguyen, P. H., Gibescu, M., & Kamphuis, R. I. G. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2753408 · Cited by 68 · see citing works

Control of a PCM ventilated facade using reinforcement learning techniques

de Gracia, Á., Fernández, C. C., Castell, A., et al. (2015) · Energy and Buildings · doi:10.1016/j.enbuild.2015.06.045 · Cited by 66 · see citing works

Synthesis of reinforcement learning, neural networks and PI control applied to a simulated heating coil

Anderson, C. W., Hittle, D. C., Katz, A. D., et al. (1997) · Artificial Intelligence in Engineering · doi:10.1016/s0954-1810(97)00004-6 · Cited by 66 · see citing works

Demand-Side Management of Domestic Electric Water Heaters Using Approximate Dynamic Programming

Al-jabery, K. K., Xu, Z., Yu, W., et al. (2016) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2016.2598563 · Cited by 64 · see citing works

Event-Based Optimization Within the Lagrangian Relaxation Framework for Energy Savings in HVAC Systems

Sun, B., Luh, P. B., Jia, Q., & Yan, B. Y. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2015.2455419 · Cited by 62 · see citing works

Evaluation of Reinforcement Learning for Optimal Control of Building Active and Passive Thermal Storage Inventory

Liu, S., & Henze, G. P. (2006) · Journal of Solar Energy Engineering · doi:10.1115/1.2710491 · Cited by 59 · see citing works

Multi-agent residential demand response based on load forecasting

Dusparić, I., Harris, C., Marinescu, A., Cahill, V., & Clarke, S. (2013) · IEEE Conference on Technologies for Sustainability · doi:10.1109/sustech.2013.6617303 · Cited by 59 · see citing works

Multiple agents and reinforcement learning for modelling charging loads of electric taxis

Jiang, C., Jing, Z., Cui, X., et al. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.164 · Cited by 57 · see citing works

Power management for Plug-in Hybrid Electric Vehicles using Reinforcement Learning with trip information

Liu, C., & Murphey, Y. L. (2014) · IEEE Transportation Electrification Conference and Expo · doi:10.1109/itec.2014.6861862 · Cited by 56 · see citing works

Optimizing legacy building operation: The evolution into data-driven predictive cyber-physical systems

Schmidt, M., Moreno, M. V., Schülke, A., et al. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.05.002 · Cited by 53 · see citing works

Online tuning of a supervisory fuzzy controller for low-energy building system using reinforcement learning

Yu, Z., & Dexter, A. L. (2010) · Control Engineering Practice · doi:10.1016/j.conengprac.2010.01.018 · Cited by 52 · see citing works

Stochastic Control for Smart Grid Users With Flexible Demand

Liang, Y., He, L., Cao, X., et al. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2263201 · Cited by 51 · see citing works

Online scheduling of plug-in vehicles in dynamic pricing schemes

Arif, A. I., Babar, M., Ahamed, T. P. I., et al. (2016) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2016.05.001 · Cited by 51 · see citing works

Reinforcement learning-based control of residential energy storage systems for electric bill minimization

Guan, C., Wang, Y., Lin, X., et al. (2015) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc.2015.7158054 · Cited by 50 · see citing works

Accelerating Learning in multi-objective systems through Transfer Learning

Taylor, A., Dusparić, I., Galván, E., Clarke, S., & Cahill, V. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889438 · Cited by 48 · see citing works

Distributed Optimization of Solar Micro-grid Using Multi Agent Reinforcement Learning

Raju, L., Sankar, S., & Milton, R. S. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.02.016 · Cited by 47 · see citing works

Short-term demand response of flexible electric heating systems: The need for integrated simulations

Bruninx, K., Patteeuw, D., Delarue, E. D., Helsen, L., & D’haeseleer, W. D. (2013) · International Conference on the European Energy Market (EEM) · doi:10.1109/eem.2013.6607333 · Cited by 45 · see citing works

Application of fuzzy Q- learning for electricity market modeling by considering renewable power penetration

Salehizadeh, M. R., & Soltaniyan, S. (2015) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2015.12.020 · Cited by 44 · see citing works

Residential demand response: Experimental evaluation and comparison of self-organizing techniques

Dusparić, I., Taylor, A., Marinescu, A., Golpayegani, F., & Clarke, S. (2017) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.07.033 · Cited by 43 · see citing works

Balancing comfort and energy consumption of a heat pump using batch reinforcement learning with fitted Q-iteration

Vázquez-Canteli, J. R., Kämpf, J. H., & Nagy, Z. (2017) · Energy Procedia · doi:10.1016/j.egypro.2017.07.429 · Cited by 42 · see citing works

Echo state network‐based Q‐learning method for optimal battery control of offices combined with renewable energy

Shi, G., Liu, D., & Wei, Q. (2017) · IET Control Theory and Applications · doi:10.1049/iet-cta.2016.0653 · Cited by 41 · see citing works

Adaptive Optimal Control of a Grid-Independent Photovoltaic System

Henze, G. P., & Dodier, R. H. (2003) · Journal of Solar Energy Engineering · doi:10.1115/1.1532005 · Cited by 41 · see citing works

Dynamic load management for a residential customer; Reinforcement Learning approach

Sheikhi, A., Rayati, M., & Ranjbar, A. M. (2016) · Sustainable Cities and Society · doi:10.1016/j.scs.2016.04.001 · Cited by 38 · see citing works

Coordination of Energy Efficiency and Demand Response

Author, N. G. (2010) · doi:10.2172/1219666 · Cited by 37 · see citing works

A learning agent for heat-pump thermostat control

Urieli, D., & Stone, P. (2013) · doi:10.5555/2484920.2485092 · Cited by 36 · see citing works

Maximizing renewable energy use with decentralized residential demand response

Dusparić, I., Taylor, A., Marinescu, A., et al. (2015) · IEEE International Smart Cities Conference (ISC2) · doi:10.1109/isc2.2015.7366212 · Cited by 35 · see citing works

Reinforcement Learning Based Algorithm for the Maximization of EV Charging Station Revenue

Dimitrov, S. I., & Lguensat, R. (2014) · International Conference on Mathematics and Computers in Sciences and in Industry · doi:10.1109/mcsi.2014.54 · Cited by 33 · see citing works

A Novel Blended Real-Time Energy Management Strategy for Plug-in Hybrid Electric Vehicle Commute Trips

Qi, X., Wu, G., Boriboonsomsin, K., et al. (2015) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2015.167 · Cited by 32 · see citing works

Residential Demand Response Algorithms: State-of-the-Art, Key Issues and Challenges

Rajasekhar, B., & Pindoriya, N. M. (2015) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-319-25479-1_2 · Cited by 32 · see citing works

A two-layer networked learning control system using actor–critic neural network

Du, D., & Fei, M. (2008) · Applied Mathematics and Computation · doi:10.1016/j.amc.2008.05.062 · Cited by 30 · see citing works

Fast learning optimiser for real‐time optimal energy management of a grid‐connected microgrid

Tan, Z., Zhang, X., Xie, B., et al. (2018) · IET Generation Transmission & Distribution · doi:10.1049/iet-gtd.2017.1983 · Cited by 28 · see citing works

Iteratively Extending Time Horizon Reinforcement Learning

Ernst, D., Geurts, P., & Wehenkel, L. A. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39857-8_11 · Cited by 27 · see citing works

Optimal bidding of plug-in electric vehicles in a market-based control setup

Vayá, M. G., Roselló, L. B., & Andersson, G. (2014) · Power Systems Computation Conference · doi:10.1109/pscc.2014.7038108 · Cited by 27 · see citing works

On-board stochastic control of Electric Vehicle recharging

Giorgio, A. D., Liberati, F., & Pietrabissa, A. (2013) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2013.6760789 · Cited by 27 · see citing works

Peak shaving of a heterogeneous cluster of residential flexibility carriers using reinforcement learning

Claessens, B., Vandael, S., Ruelens, F., et al. (2013) · doi:10.1109/isgteurope.2013.6695254 · Cited by 26 · see citing works

Implementation of real-time energy management strategy based on reinforcement learning for hybrid electric vehicles and simulation validation

Kong, Z., Zou, Y., & Liu, T. (2017) · PLoS ONE · doi:10.1371/journal.pone.0180491 · Cited by 23 · see citing works

Applying reinforcement learning method to optimize an Energy Hub operation in the smart grid

Rayati, M., Sheikhi, A., & Ranjbar, A. M. (2015) · IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT) · doi:10.1109/isgt.2015.7131906 · Cited by 22 · see citing works

Reinforcement learning aided smart-home decision-making in an interactive smart grid

Ding, L., & Jayaweera, S. K. (2014) · doi:10.1109/igesc.2014.7018632 · Cited by 22 · see citing works

A review of multi-agent systems concepts and research related to building HVAC control

Windham, A., & Treado, S. (2015) · Science and Technology for the Built Environment · doi:10.1080/23744731.2015.1074851 · Cited by 22 · see citing works

An Intelligent Battery Controller Using Bias-Corrected Q-learning

Lee, D., & Powell, W. B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8164 · Cited by 22 · see citing works

Intelligent Decision-Making for Smart Home Energy Management

Berlink, H., Kagan, N., & Costa, A. H. R. (2014) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-014-0169-8 · Cited by 21 · see citing works

A novel grid load management technique using electric water heaters and Q-learning

Al-jabery, K. K., Wunsch, D. C., Xiong, J., et al. (2014) · IEEE International Conference on Smart Grid Communications (SmartGridComm) · doi:10.1109/smartgridcomm.2014.7007742 · Cited by 21 · see citing works

Wireless and Satellite Systems

Pillai, P., Hu, Y. F., Otung, I. E., et al. (2015) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-319-25479-1 · Cited by 20 · see citing works

Scheduling of plug-in electric vehicle battery charging with price prediction

Chis, A., Lundén, J., & Koivunen, V. (2013) · doi:10.1109/isgteurope.2013.6695263 · Cited by 20 · see citing works

Smart charging of electric vehicles using reinforcement learning

Valogianni, K., Ketter, W., & Collins, J. E. (2013) · EUR Research Repository (Erasmus University Rotterdam) · doi:10.5555/2908259.2908266 · Cited by 17 · see citing works

Machine Learning: ECML 2006

Candillier, L., Tellier, I., Torre, F., et al. (2006) · Lecture notes in computer science · doi:10.1007/11871842 · Cited by 15 · see citing works

Learning based bidding strategy for HVAC systems in double auction retail energy markets

Sun, Y., Somani, A., & Carroll, T. E. (2015) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2015.7171177 · Cited by 14 · see citing works

Distributed demand response algorithms against semi-honest adversaries

Zhu, M. (2014) · doi:10.1109/pesgm.2014.6939191 · Cited by 14 · see citing works

Event-based optimization with non-stationary uncertainties to save energy costs of HVAC systems in buildings

Sun, B., Luh, P. B., Jia, Q., et al. (2013) · 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2013.6654055 · Cited by 13 · see citing works

Flexible power consumption management using Q learning techniques in a smart home

Kaliappan, A. T., Sathiakumar, S., & Parameswaran, N. (2013) · IEEE Conference on Clean Energy and Technology (CEAT) · doi:10.1109/ceat.2013.6775653 · Cited by 13 · see citing works

Optimization of plug-in electric vehicle charging with forecasted price

Chis, A., Lundén, J., & Koivunen, V. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178338 · Cited by 12 · see citing works

Structure-aware stochastic load management in smart grids

Zhang, Y., & van der Schaar, M. (2014) · doi:10.1109/infocom.2014.6848212 · Cited by 12 · see citing works

Model-Based Reinforcement Learning in a Complex Domain

Kalyanakrishnan, S., Stone, P., & Liu, Y. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-68847-1_15 · Cited by 12 · see citing works

Market Structure and Equilibrium

Tribe, K. P. (2016) · History of Political Economy · doi:10.1215/00182702-3638767 · Cited by 10 · see citing works

P-MARL: Prediction-Based Multi-Agent Reinforcement Learning for Non-Stationary Environments

Marinescu, A., Dusparić, I., Taylor, A., et al. (2015) · Adaptive Agents and Multi-Agents Systems · doi:10.5555/2772879.2773492 · Cited by 9 · see citing works

Massive 3D models and physical data for building simulation at the urban scale: a focus on Geneva and climate change scenarios

Vázquez-Canteli, J. R., & Kämpf, J. H. (2016) · WIT transactions on ecology and the environment · doi:10.2495/sc160041 · Cited by 9 · see citing works

Market-Based EV Charging Coordination

Dauer, D., Flath, C. M., Ströhle, P., & Weinhardt, C. (2013) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat.2013.97 · Cited by 8 · see citing works

An intelligent Home Energy Management System with classifier system

Sekizaki, S., Hayashida, T., & Nishizaki, I. (2015) · IEEE International Workshop on Computational Intelligence and Applications (IWCIA) · doi:10.1109/iwcia.2015.7449452 · Cited by 8 · see citing works

Optimal self-learning battery control in smart residential grids by iterative Q-learning algorithm

Wei, Q., Liu, D., Shi, G., et al. (2014) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2014.7010630 · Cited by 6 · see citing works

Modelling of Ambient Comfort Affect Reward Based Adaptive Laboratory Climate Controller

Bielskis, A. A., Guseinovienė, E., Drungilas, D., et al. (2013) · Elektronika ir Elektrotechnika · doi:10.5755/j01.eee.19.8.5399 · Cited by 3 · see citing works

Data-driven probabilistic machine learning in sustainable smart energy/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm

Ahmad, T., Madonski, R., Zhang, D., Huang, C., & Mujeeb, A. (2022) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2022.112128 · Cited by 711 · see citing works

Artificial intelligence and machine learning approaches to energy demand-side response: a systematic review

Antonopoulos, I., Robu, V., Couraud, B., Kirli, D., Norbu, S., Kiprakis, A., Flynn, D., Elizondo-Gonzalez, S., & Wattam, S. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109899 · Cited by 656 · see citing works

Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review

Cao, D., Hu, W., Zhao, J., Zhang, G., Zhang, B., Liu, Z., Chen, Z., & Blaabjerg, F. (2020) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2020.000552 · Cited by 460 · see citing works

A Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home Energy Management

Xu, X., Jia, Y., Xu, Y., Xu, Z., Chai, S., & Lai, C. S. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2971427 · Cited by 420 · see citing works

Artificial Intelligence Techniques for Stability Analysis and Control in Smart Grids: Methodologies, Applications, Challenges and Future Directions

Shi, Z., Yao, W., Li, Z., Zeng, L., Zhao, Y., Zhang, R., Tang, Y., & Wen, J. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115733 · Cited by 327 · see citing works

Fusing TensorFlow with building energy simulation for intelligent energy management in smart cities

Vázquez-Canteli, J. R., Ulyanin, S., Kämpf, J., & Nagy, Z. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2018.11.021 · Cited by 195 · see citing works

Tackling climate change with machine learning

Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336 · see citing works

Artificial Intelligence Enabled Demand Response: Prospects and Challenges in Smart Grid Environment

Khan, M. A., Saleh, A. M., Waseem, M., & Sajjad, I. A. (2023) · IEEE Access · doi:10.1109/access.2022.3231444 · Cited by 179 · see citing works

Development and Evaluation of a Smart Charging Strategy for an Electric Vehicle Fleet Based on Reinforcement Learning

Tuchnitz, F., Ebell, N., Schlund, J., & Pruckner, M. (2021) · Applied Energy · doi:10.1016/j.apenergy.2020.116382 · Cited by 175 · see citing works

Reinforcement learning for HVAC control in intelligent buildings: A technical and conceptual review

Al Sayed, K., Boodi, A., Sadeghian Broujeny, R., & Beddiar, K. (2024) · Journal of Building Engineering · doi:10.1016/j.jobe.2024.110085 · Cited by 140 · see citing works

Evaluation of advanced control strategies for building energy systems

Stoffel, P., Maier, L., Kümpel, A., Schreiber, T., & Müller, D. (2023) · Energy and Buildings · doi:10.1016/j.enbuild.2022.112709 · Cited by 126 · see citing works

Coordinated energy management for a cluster of buildings through deep reinforcement learning

Pinto, G., Piscitelli, M. S., Vázquez-Canteli, J. R., Nagy, Z., & Capozzoli, A. (2021) · Energy · doi:10.1016/j.energy.2021.120725 · Cited by 108 · see citing works

Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings

Xie, J., Ajagekar, A., & You, F. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121162 · Cited by 109 · see citing works

Digitalization and the Anthropocene

Creutzig, F., Acemoglu, D., Bai, X., Edwards, P. N., Hintz, M. J., Kaack, L. H., Kilkis, S., Kunkel, S., Luers, A., Milojevic-Dupont, N., et al. (2022) · Annual Review of Environment and Resources · doi:10.1146/annurev-environ-120920-100056 · Cited by 112 · see citing works

Energy management for demand response in networked greenhouses with multi-agent deep reinforcement learning

Ajagekar, A., Decardi-Nelson, B., & You, F. (2024) · Applied Energy · doi:10.1016/j.apenergy.2023.122349 · Cited by 82 · see citing works

Integrating active demand into the distribution system using metaheuristic techniques

Obando‐Paredes, E. D., López‐García, D., & Carvajal‐Quintero, S. X. (2024) · The Journal of Engineering · doi:10.1049/tje2.70005 · Cited by 2 · see citing works

Demand Response for Large Public Building System

He, Z. D., Liu, J. W., & Shi, C. K. (2012) · Advanced materials research · doi:10.4028/www.scientific.net/amr.518-523.4387

Scenario analysis of residential demand response at network peak periods

Gyamfi, S., & Krumdieck, S. P. (2012) · Electric Power Systems Research · doi:10.1016/j.epsr.2012.07.004 · Cited by 54 · see citing works

An Analysis on Power Demand Reduction Effects of Demand Response Systems in the Smart Grid Environment in Korea

Won, J.-R., & Song, K. (2013) · Journal of Electrical Engineering and Technology · doi:10.5370/jeet.2013.8.6.1296 · Cited by 8 · see citing works

The Development of the Automatic Demand Response Systems Based on SEP 2.0 for the Appliances's Energy Reduction on Smart Grid Environments

Jung, J.-U., Kim, S.-H., & Jin, K.-H. (2016) · The Journal of the Korean Institute of Information and Communication Engineering · doi:10.6109/jkiice.2016.20.9.1799

Quantification of Demand Response in Smart Grids

Musti, K. S. S. (2020) · IEEE India Council International Subsections Conference (INDISCON) · doi:10.1109/indiscon50162.2020.00063 · Cited by 22 · see citing works