Residential Load Scheduling With Renewable Generation in the Smart Grid: A Reinforcement Learning Approach
Abstract
The significance and need of demand response (DR) programs is realized by the utility as a means to reduce the additional production cost imposed by the accelerating energy demand. With the development in smart information and communication systems, the price-based DR programs can be effectively utilized for controlling the loads of smart residential buildings. Nowadays, the use of stochastic renewable energy sources like photovoltaic (PV) by a small domestic consumer is increasing. In this paper, a generalized model for the residential load scheduling or load commitment problem (LCP) in the presence of renewable sources for any type of tariff is presented. Reinforcement learning (RL) is an efficient tool that has been used to solve the decision making problem under uncertainty. An RL-based approach to solve the LCP is also proposed. The novelty of this paper lies in the introduction of a comprehensive model with implementable solution considering consumer comfort, stochastic renewable power, and tariff. Simulation experiments are conducted to test the efficacy and scalability of the proposed algorithm. The performance of the algorithm is investigated by considering a domestic consumer with schedulable and nonschedulable appliances along with a PV source. Guidelines are given for choosing the parameters of the load.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Methods like RL can be used to learn how to optimally schedule household appliances to consume energy more efficiently and sustainably.
“Methods like RL can be used to learn how to optimally schedule household appliances to consume energy more efficiently and sustainably [543,679].”
From Tackling climate change with machine learning (2023) · cites this paper as [679]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Power Generation, Operation, and Control
Wood, A. J., & Wollenberg, B. F. (1984) · Cited by 8,337 · see citing works
Power-Electronic Systems for the Grid Integration of Renewable Energy Sources: A Survey
Carrasco, J. M., Franquelo, L. G., Białasiewicz, J. T., Galván-Díez, E., Portillo, R., Martín-Prats, M. A., Leon, J. I., & Moreno-Alfonso, N. (2006) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2006.878356 · Cited by 4,027 · see citing works
Mohsenian-Rad, A.-H., Wong, V. W. S., Jatskevich, J., Schober, R., & Leon‐Garcia, A. (2010) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2010.2089069 · Cited by 2,751 · see citing works
A summary of demand response in electricity markets
Albadi, M. H., & El-Saadany, E. F. (2008) · Electric Power Systems Research · doi:10.1016/j.epsr.2008.04.002 · Cited by 1,884 · see citing works
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 1× · see citing works
“RL has been proposed for the solution of numerous control and optimization problems [30].”
BackgroundOptimal Renewable Resources Mix for Distribution System Energy Loss Minimization
Atwa, Y. M., El-Saadany, E. F., Salama, M. M. A., et al. (2009) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2009.2030276 · Cited by 1,397 · see citing works
Demand side management in smart grid using heuristic optimization
Logenthiran, T., Srinivasan, D., & Shun, T. Z. (2012) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2195686 · Cited by 1,229 · see citing works
Kouro, S., Leon, J. I., Vinnikov, D., et al. (2015) · IEEE Industrial Electronics Magazine · doi:10.1109/mie.2014.2376976 · Cited by 1,197 · see citing works
Temporal Difference Learning and TD-Gammon
Tesauro, G. (1995) · ICGA Journal · doi:10.3233/icg-1995-18207 · Cited by 770 · see citing works
Demand response in smart electricity grids equipped with renewable energy sources: A review
Aghaei, J., & Alizadeh, M. I. (2012) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2012.09.019 · Cited by 689 · see citing works
Optimal Smart Home Energy Management Considering Energy Saving and a Comfortable Lifestyle
Anvari‐Moghaddam, A., Monsef, H., & Rahimi‐Kian, A. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2349352 · Cited by 448 · see citing works
Paterakis, N. G., Erdinç, O., Bakirtzis, A. G., & Catalào, J. P. S. (2015) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2015.2438534 · Cited by 431 · see citing works
Neuro-dynamic programming: an overview
Bertsekas, D. P., & Tsitsiklis, J. N. (2002) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.1995.478953 · Cited by 424 · see citing works
Residential Demand Response: Dynamic Energy Management and Time-Varying Electricity Pricing
Muratori, M., & Rizzoni, G. (2015) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2015.2414880 · Cited by 402 · see citing works
Optimal scheduling of household appliances for demand response
Setlhaolo, D., Xia, X., & Zhang, J. (2014) · Electric Power Systems Research · doi:10.1016/j.epsr.2014.04.012 · Cited by 312 · see citing works
Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning
Kim, B., Zhang, Y., van der Schaar, M., & Lee, J.-W. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2495145 · Cited by 299 · see citing works
Elevator Group Control Using Multiple Reinforcement Learning Agents
Crites, R. H., & Barto, A. G. (1998) · Machine Learning · doi:10.1023/a:1007518724497 · Cited by 275 · see citing works
Autonomous Appliance Scheduling for Household Energy Management
Adika, C. O., & Wang, L. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2271427 · Cited by 258 · see citing works
Optimal Power Management of Residential Customers in the Smart Grid
Guo, Y., Pan, M., & Fang, Y. M. (2012) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2012.25 · Cited by 238 · see citing works
Intelligent Residential Energy Management System for Dynamic Demand Response in Smart Buildings
Arun, S. L., & Selvan, M. P. (2017) · IEEE Systems Journal · doi:10.1109/jsyst.2017.2647759 · Cited by 228 · see citing works
Neural network controller for Active Demand-Side Management with PV energy in the residential sector
Matallanas, E., Castillo-Cagigal, M., Gutiérrez, Á., Monasterio-Huelin, F., Caamaño‐Martín, E., Masa-Bote, D., & Leube, F. J. J. (2011) · Applied Energy · doi:10.1016/j.apenergy.2011.09.004 · Cited by 218 · see citing works
Power Systems Stability Control: Reinforcement Learning Framework
Ernst, D., Glavić, M., & Wehenkel, L. A. (2004) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2003.821457 · Cited by 214 · see citing works
Thathachar, M. A. L., & SASTRY, P. S. (2004) · Springer eBooks · doi:10.1007/978-1-4419-9052-5 · Cited by 174 · see citing works
Residential Demand Response for Renewable Energy Resources in Smart Grid Systems
Park, L., Jang, Y., Cho, S., et al. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2704282 · Cited by 144 · see citing works
Integrating demand-side management into utility planning
Gellings, C. W., & Smith, W. M. (1989) · Proceedings of the IEEE · doi:10.1109/5.29331 · Cited by 135 · see citing works
Real-time Energy Control Approach for Smart Home Energy Management System
Zhou, S., Wu, Z. S., Li, J., et al. (2014) · Electric Power Components and Systems · doi:10.1080/15325008.2013.862322 · Cited by 123 · see citing works
Li, T., & Dong, M. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2550500 · Cited by 87 · see citing works
Parallel Load Schedule Optimization With Renewable Distributed Generators in Smart Grids
Yang, P., Chavali, P., Gilboa, E., et al. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2013.2264728 · Cited by 76 · see citing works
Weck, M., van Hooff, J., & van Sark, W. G. J. H. M. (2016) · International Journal of Energy Research · doi:10.1002/er.3683 · Cited by 50 · see citing works
Application of reinforcement learning for segmentation of transrectal ultrasound images
Sahba, F., Tizhoosh, H. R., & Salama, M. M. A. (2008) · BMC Medical Imaging · doi:10.1186/1471-2342-8-8 · Cited by 47 · see citing works
Davda, A. T., Azzopardi, B., Parekh, B. R., et al. (2014) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2013.2292117 · Cited by 46 · see citing works
Reinforcement Learning-based Energy Trading for MicrogridsPreprint
Xiao, L., Xiao, X., Dai, C., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.06285 · Cited by 32 · see citing works
Load scheduling with Maximum Demand using Binary Particle Swarm Optimization
Remani, T., Jasmin, E. A., & Ahamed, T. P. I. (2015) · International Conference on Technological Advancements in Power and Energy (TAP Energy) · doi:10.1109/tapenergy.2015.7229634 · Cited by 21 · see citing works
Self-citationLearning Automata Algorithms for Load Scheduling
Ali, S. Q., Ahamed, T. P. I., & Malik, N. H. (2013) · Electric Power Components and Systems · doi:10.1080/15325008.2012.742943 · Cited by 21 · see citing works
Self-citationDemand response under real-time pricing for domestic energy system with DGs
Ruan, B., Yang, Q., Fang, X., et al. (2014) · International Conference on Power System Technology (POWERCON) · doi:10.1109/powercon.2014.6993601 · Cited by 14 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home Energy Management
Xu, X., Jia, Y., Xu, Y., Xu, Z., Chai, S., & Lai, C. S. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2971427 · Cited by 420 · see citing works
Tackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336 · see citing works
A multi-agent deep reinforcement learning based energy management for behind-the-meter resources
Wilk, P., Wang, N., & Li, J. (2022) · The Electricity Journal · doi:10.1016/j.tej.2022.107129 · Cited by 9 · see citing works
- Authors
- Remani, T., Jasmin, E. A., & Ahamed, T. P. I.
- Affiliations
- Government Engineering College Thrissur (IN)
- T.K.M Institute of Technology (IN)
- Venue
- IEEE Systems Journal
- Published
- 2019-09
- Volume
- 13
- Issue
- 3
- Pages
- 3283-3294
- ISSN
- 1932-8184 (ISSN-L)
- 1937-9234
- 2373-7816
- OpenAlex
- W2885341446
- Cited by
- 106
- References
- 37
- Citations by year
- 2026: 1 · 2025: 10 · 2024: 14 · 2023: 14 · 2022: 28 · 2021: 19 · 2020: 13 · 2019: 7
- FWCI
- 3.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Systems Journal (publishedVersion)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
- Physical Sciences › Engineering › Electrical and Electronic Engineering › IoT-based Smart Home Systems
- Keywords
- residential load scheduling, demand response, reinforcement learning, photovoltaic generation, smart grid, electricity tariffs, customer comfort, price-based demand response
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
