A Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home Energy Management
Open accessAbstract
This paper proposes a novel framework for home energy management (HEM) based on reinforcement learning in achieving efficient home-based demand response (DR). The concerned hour-ahead energy consumption scheduling problem is duly formulated as a finite Markov decision process (FMDP) with discrete time steps. To tackle this problem, a data-driven method based on neural network (NN) and Q-learning algorithm is developed, which achieves superior performance on cost-effective schedules for HEM system. Specifically, real data of electricity price and solar photovoltaic (PV) generation are timely processed for uncertainty prediction by extreme learning machine (ELM) in the rolling time windows. The scheduling decisions of the household appliances and electric vehicles (EVs) can be subsequently obtained through the newly developed framework, of which the objective is dual, i.e., to minimize the electricity bill as well as the DR induced dissatisfaction. Simulations are performed on a residential house level with multiple home appliances, an EV and several PV panels. The test results demonstrate the effectiveness of the proposed data-driven based HEM framework.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fuzzy Q-learning adaptations allow agents to navigate uncertain partner behaviors or incomplete operational data by adjusting policy confidence based on interaction outcomes.
“When paired with fuzzy Q-learning adaptations [43], agents can navigate uncertain partner behaviors or incomplete operational data by adjusting policy confidence based on interaction outcomes.”
From Explainable AI and Multi-Agent Systems for Energy Management in IoT-Edge Environments: A State of the Art Review (2026) · cites this paper as [43]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855 · see citing works
Solar cells—Operating principles, technology and system applications
(1982) · Solar Energy · doi:10.1016/0038-092x(82)90265-1 · Cited by 1,730 · see citing works
Reinforcement learning for demand response: A review of algorithms and modeling techniques
Vázquez-Canteli, J. R., & Nagy, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.11.002 · Cited by 883 · see citing works
Evaluating MPPT Converter Topologies Using a Matlab PV Model
Walker, G. R. (2001) · QUT ePrints (Queensland University of Technology) · Cited by 871 · see citing works
Koza, J. R. (1992) · Medical Entomology and Zoology · Cited by 825 · see citing works
Rezvanizaniani, S. M., Liu, Z., Chen, Y., & Lee, J. (2014) · Journal of Power Sources · doi:10.1016/j.jpowsour.2014.01.085 · Cited by 688 · see citing works
On-Line Building Energy Optimization Using Deep Reinforcement Learning
Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2834219 · Cited by 649 · see citing works
Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning
Wan, Z., Li, H., He, H., & Prokhorov, D. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2879572 · Cited by 548 · cited 1× · see citing works
“[14], Wan et al. use deep RL algorithms to determine the optimal solutions of the EV charging/discharging scheduling problem.”
A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach
Lu, R., Hong, S. H., & Zhang, X. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.03.072 · Cited by 436 · see citing works
Du, Y., & Li, F. F. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2930299 · Cited by 396 · see citing works
Lu, R., Hong, S. H., & Yu, M. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2909266 · Cited by 390 · see citing works
Shareef, H., Ahmed, M. S., Mohamed, A., & Hassan, E. A. (2018) · IEEE Access · doi:10.1109/access.2018.2831917 · Cited by 385 · see citing works
Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning
Ruelens, F., Claessens, B., Vandael, S., Schutter, B. D., Babuška, R., & Belmans, R. J. M. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2517211 · Cited by 339 · see citing works
Stochastic Optimal Energy Management of Smart Home With PEV Energy Storage
Wu, X., Hu, X., Yin, X., et al. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2606442 · Cited by 307 · see citing works
Reinforcement Learning Approach for Optimal Distributed Energy Management in a Microgrid
Foruzan, E., Soh, L., & Asgarpoor, S. (2018) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2018.2823641 · Cited by 295 · see citing works
Yu, M., & Hong, S. H. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.06.010 · Cited by 220 · see citing works
Value-Difference Based Exploration: Adaptive Control between Epsilon-Greedy and Softmax
Tokic, M., & Palm, G. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24455-1_33 · Cited by 207 · see citing works
Chance Constrained Optimization in a Home Energy Management System
Huang, Y., Wang, L., Guo, W., Kang, Q., & Wu, Q. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2550031 · Cited by 197 · see citing works
Artificial intelligence based smart energy community management: A reinforcement learning approach
Zhou, S., Hu, Z., Gu, W., et al. (2019) · CSEE Journal of Power and Energy Systems · doi:10.17775/cseejpes.2018.00840 · Cited by 190 · see citing works
Fu, W., Wang, K., Li, C., & Tan, J. (2019) · Energy Conversion and Management · doi:10.1016/j.enconman.2019.02.086 · Cited by 175 · see citing works
Rafiei, M., Niknam, T., Aghaei, J., et al. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2807845 · Cited by 169 · see citing works
Luo, F., Kong, W., Ranzi, G., et al. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2915679 · Cited by 152 · see citing works
Online Energy Management for a Sustainable Smart Home With an HVAC Load and Random Occupancy
Yu, L., Jiang, T., & Zou, Y. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2775209 · Cited by 147 · see citing works
A Fast Technique for Smart Home Management: ADP With Temporal Difference Learning
Keerthisinghe, C., Verbič, G., & Chapman, A. C. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2629470 · Cited by 134 · see citing works
Smart Home Energy Management Including Renewable Sources: A QoE-driven Approach
Pilloni, V., Floris, A., Meloni, A., & Atzori, L. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2605182 · Cited by 131 · see citing works
Remani, T., Jasmin, E. A., & Ahamed, T. P. I. (2019) · IEEE Systems Journal · doi:10.1109/jsyst.2018.2855689 · Cited by 106 · see citing works
Hierarchical Energy Management System for Home Microgrids
Luo, F., Ranzi, G., Wang, S., et al. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2884323 · Cited by 101 · see citing works
Demand Response Management for Industrial Facilities: A Deep Reinforcement Learning Approach
Huang, X., Hong, S. H., Yu, M., Ding, Y., & Jiang, J. (2019) · IEEE Access · doi:10.1109/access.2019.2924030 · Cited by 84 · see citing works
Time-Coordinated Multienergy Management of Smart Buildings Under Uncertainties
Sharma, S., Xu, Y. H., Verma, A., et al. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2901120 · Cited by 76 · see citing works
Self-citationPDE Modeling and Control of Electric Vehicle Fleets for Ancillary Services: A Discrete Charging Case
Floch, C. L., Kara, E. C., & Moura, S. J. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2556643 · Cited by 71 · see citing works
Conditional Density Forecast of Electricity Price Based on Ensemble ELM and Logistic EMOS
Chai, S., Xu, Z., & Jia, Y. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2817284 · Cited by 64 · see citing works
Self-citationOptimal control theory and the linear Bellman equation
Kappen, H. J. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511984679.018 · Cited by 54 · see citing works
Social housing and renewable energy: Community energy in a supporting role
McCabe, A., Pojani, D., & van Groenou, A. B. (2018) · Energy Research & Social Science · doi:10.1016/j.erss.2018.02.005 · Cited by 51 · see citing works
Optimally coordinated dispatch of combined‐heat‐and‐electrical network with demand response
Chen, Y., Xu, Y. H., Li, Z., et al. (2019) · IET Generation Transmission & Distribution · doi:10.1049/iet-gtd.2018.6992 · Cited by 48 · see citing works
Self-citationAttar, M., Homaee, O., Falaghi, H., et al. (2017) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2017.09.028 · Cited by 37 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Ahmad, T., Madonski, R., Zhang, D., Huang, C., & Mujeeb, A. (2022) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2022.112128 · Cited by 711 · see citing works
Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review
Cao, D., Hu, W., Zhao, J., Zhang, G., Zhang, B., Liu, Z., Chen, Z., & Blaabjerg, F. (2020) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2020.000552 · Cited by 460 · see citing works
“Reference [105] [106] [107] [108] [109] [110] [111] [112] [113] Field Demand-side management Distribution network Demand-side management Operational control Operational control Operational control Distribution network Demand response Distribution network Algorithm DQN DQN Q-learning DQN MADDPG MADDPG MADDPG + attentio…”
MethodChen, X., Qu, G., Tang, Y., Low, S. H., & Li, N. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3154718 · Cited by 440 · see citing works
“Some Q -learning-based work [133] discretizes the continuous action space to fit the algorithm framework.”
BackgroundEnergetics Systems and artificial intelligence: Applications of industry 4.0
Ahmad, T., Zhu, H., Zhang, D., Tariq, R., Bassam, A., Ullah, F., AlGhamdi, A. S., & Alshamrani, S. S. (2021) · Energy Reports · doi:10.1016/j.egyr.2021.11.256 · Cited by 399 · see citing works
A Review of Technical Standards for Smart Cities
Lai, C. S., Jia, Y., Dong, Z., Wang, D., Tao, Y., Lai, Q. H., Wong, R. T. K., Zobaa, A. F., Wu, R., & Lai, L. L. (2020) · Clean Technologies · doi:10.3390/cleantechnol2030019 · Cited by 352 · see citing works
Self-citationLee, S., & Choi, D. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3035451 · Cited by 237 · see citing works
Multi-agent deep reinforcement learning approach for EV charging scheduling in a smart grid
Park, K., & Moon, I. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.120111 · Cited by 193 · see citing works
“Xu et al. [21] solved the problem of managing electric devices used at home, including EVs, using MADRL.”
MethodMachine learning in photovoltaic systems: A review
Gaviria, J. F., Narváez, G., Guillén, C. A., et al. (2022) · Renewable Energy · doi:10.1016/j.renene.2022.06.105 · Cited by 189 · see citing works
Artificial intelligence techniques for enabling Big Data services in distribution networks: A review
Barja-Martínez, S., Aragüés‐Peñalba, M., Munné‐Collado, Í., Lloret-Gallego, P., Bullich‐Massagué, E., & Villafáfila‐Robles, R. (2021) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2021.111459 · Cited by 187 · see citing works
Qiu, D., Ye, Y., Papadaskalopoulos, D. A., & Štrbac, G. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.116940 · Cited by 164 · see citing works
Ali, A. O., Elmarghany, M. R., Abdelsalam, M. M., Sabry, M. N., & Hamed, A. M. (2022) · Journal of Energy Storage · doi:10.1016/j.est.2022.104609 · Cited by 144 · see citing works
Wang, Y., Qiu, D., & Štrbac, G. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.118575 · Cited by 136 · see citing works
Amer, A. A., Shaban, K. B., & Massoud, A. M. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3198401 · Cited by 138 · see citing works
“Complementary to this work, the authors in [27] introduced a more comprehensive model considering EVs.”
BackgroundNutakki, M., & Mandava, S. (2023) · Engineering applications of artificial intelligence · doi:10.1016/j.engappai.2022.105721 · Cited by 104 · see citing works
Yang, T., Zhao, L., Li, W., Wu, J., & Zomaya, A. Y. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.117335 · Cited by 122 · see citing works
Latent variable models in the era of industrial big data: Extension and beyond
Kong, X., Jiang, X., Zhang, B., Yuan, J., & Ge, Z. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.09.005 · Cited by 126 · see citing works
“In the past two decades, a lot of data-driven modeling and analyzing methods have been proposed for different industrial applications, such as semiconductor manufacturing [15, 16], injection molding [17, 18], smart grid [19, 20], wind energy forecasting [21, 22], predictive maintenance [23, 24], prognostics health man…”
BackgroundRen, K., Liu, J. P., Wu, Z., Liu, X., Nie, Y., & Xu, H. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.122258 · Cited by 121 · see citing works
Multi-agent hierarchical reinforcement learning for energy management
Jendoubi, I., & Bouffard, F. (2023) · Applied Energy · doi:10.1016/j.apenergy.2022.120500 · Cited by 123 · see citing works
Huy, T. H. B., Dinh, H. T., Vo, D. N., & Kim, D. (2023) · Energy Conversion and Management · doi:10.1016/j.enconman.2023.117340 · Cited by 119 · see citing works
Wang, Z., Ikechukwu, O. M., Huang, H., Qiu, C., Hisada, M., & Abdallah, A. B. (2020) · IEEE Access · doi:10.1109/access.2020.3044612 · Cited by 114 · see citing works
“In [35]–[39], intelligent integrated approaches were proposed for efficient demand response.”
BackgroundMethodA review of research on reinforcement learning algorithms for multi-agents
Hu, K., Li, M., Song, Z., Xu, K., Xia, Q., Sun, N., Zhou, P., & Xia, M. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128068 · Cited by 113 · see citing works
Online Optimal Power Scheduling of a Microgrid via Imitation Learning
Gao, S., Xiang, C., Yu, M., Tan, K. T., & Lee, T. H. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3122570 · Cited by 101 · see citing works
Building Energy Management With Reinforcement Learning and Model Predictive Control: A Survey
Zhang, H., Seal, S., Wu, D., Bouffard, F., & Boulet, B. (2022) · IEEE Access · doi:10.1109/access.2022.3156581 · Cited by 105 · see citing works
Smart home energy management systems: Research challenges and survey
Raza, A., Li, J., Ghadi, Y. Y., Adnan, M., & Ali, M. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.02.033 · Cited by 109 · see citing works
Digital Twin-Driven Decision Making and Planning for Energy Consumption
Fathy, Y., Jaber, M., & Nadeem, Z. (2021) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan10020037 · Cited by 100 · see citing works
“For example, the authors in [13] presented a neural network-based method for forecasting the next hour’s energy consumption and Q-learning to decide the best action for appliances that can either delay their usage or alter the mode of operation to save energy.”
MethodAlam, M. M., Rahman, M. H., Ahmed, M. F., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-19147-y · Cited by 98 · see citing works
“Equations (15) and (16) describe the total predicted demand profile and consumed power.”
BackgroundMulti-agent reinforcement learning for resources allocation optimization: a survey
Hady, M. A., Hu, S., Pratama, M., Cao, Z., & Kowalczyk, R. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11340-5 · Cited by 120 · see citing works
A systematic review of machine learning techniques related to local energy communities
Hernandez-Matheus, A., Löschenbrand, M., Berg, K., Fuchs, I., Aragüés‐Peñalba, M., Bullich‐Massagué, E., & Sumper, A. (2022) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2022.112651 · Cited by 97 · see citing works
Machine learning applications in energy systems: current trends, challenges, and research directions
Aslam, S., Aung, P. P., Rafsanjani, A. S., & Majeed, A. P. P. A. (2025) · Energy Informatics · doi:10.1186/s42162-025-00524-6 · Cited by 112 · see citing works
“ML models perform various tasks in energy applications, such as storing data [41], visualizing data [42], analyzing data [43], capturing data [44], updating data, etc.”
Deep Reinforcement Learning-Based Demand Response for Smart Facilities Energy Management
Lu, R., Bai, R., Luo, Z., Jiang, J., Sun, M., & Zhang, H. (2021) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2021.3104596 · Cited by 92 · see citing works
Huang, C., Zhang, H., Wang, L., et al. (2022) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2021.000394 · Cited by 92 · see citing works
Electricity Theft Detection Using Deep Reinforcement Learning in Smart Power Grids
El-Toukhy, A. T., Badr, M. M., Mahmoud, M. M. E. A., Srivastava, G., Fouda, M. M., & Alsabaan, M. (2023) · IEEE Access · doi:10.1109/access.2023.3284681 · Cited by 92 · see citing works
“…of DL approaches since it has the capability of efficiently incrementing the learning of the model by retraining the model using new consumption patterns or newly traced attacks, without forgetting what has already been learned or requiring complete retraining of the model from scratch [28], [31].”
BackgroundMulti-Agent Reinforcement Learning for Intelligent V2G Integration in Future Transportation Systems
Dong, J., Yassine, A., Armitage, A., & Hossain, M. S. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3284756 · Cited by 97 · see citing works
“The work in [15] develops an RL framework using a finite Markov decision process (FMDP) to achieve optimal scheduling of EVs.”
MethodQiu, D., Wang, Y., Zhang, T., Sun, M., & Štrbac, G. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3166215 · Cited by 89 · see citing works
“To model the multiagent system (MAS), the most straightforward MARL method is independent RL (IRL), which has been applied to control EV schedules in a home energy management system [22].”
MethodA Reinforcement-Learning-Based Secure Demand Response Scheme for Smart Grid System
Kumari, A., & Tanwar, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3090305 · Cited by 86 · see citing works
“Here, we compare the proposed Q-SDRM scheme with multiagent RL (MARL) approach [38] and HEM using RL and artificial neural network (RL-ANN) [8].”
MethodGoh, H. H., Huang, Y., Lim, C. S., Zhang, D., Liu, H., Dai, W., Kurniawan, T. A., & Rahman, S. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3179567 · Cited by 87 · see citing works
“Another home energy management framework is studied in [20], in which the framework merges extreme learning machine and multi-agent Q -learning.”
MethodSmart Home Energy Management: VAE-GAN Synthetic Dataset Generator and Q-Learning
Razghandi, M., Zhou, H., Erol‐Kantarci, M., et al. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3288824 · Cited by 83 · see citing works
“For example, a correlated Q-learning-based approach is proposed in [5] for microgrid energy management, and a multi-agent reinforcement learning method is introduced in [21] to minimize the energy cost of different smart home devices.”
MethodDistributionally Robust Hydrogen Optimization With Ensured Security and Multi-Energy Couplings
Zhao, P., Gu, C., Hu, Z., et al. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.3005991 · Cited by 79 · see citing works
“However, the abundant renewable generation poses operation and security challenges to power systems [1], [2].”
BackgroundLai, B.-C., Chiu, W., & Tsai, Y.-P. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3157026 · Cited by 77 · see citing works
LSTM-based Energy Management for Electric Vehicle Charging in Commercial-building Prosumers
Zhou, H., Zhou, Y., Hu, J., et al. (2021) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2020.000501 · Cited by 75 · see citing works
“References [15-21] have provided valuable insights for our study.”
BackgroundMethodResultDeep Reinforcement Learning for Real-Time Energy Management in Smart Home
Wei, G., Chi, M., Liu, Z., Ge, M., Li, C., & Liu, X. (2023) · IEEE Systems Journal · doi:10.1109/jsyst.2023.3247592 · Cited by 81 · see citing works
“Q-learning is a classical model-free RL algorithm, which was adopted in different cases [9], [10], [11], [12], [13] to reduce the energy cost of the smart home.”
MethodAlvarez‐Alvarado, M. S., Apolo-Tinoco, C., Ramirez-Prado, M. J., Alban-Chacón, F. E., Pico, N., Aviles-Cedeno, J., Recalde, A. A., Moncayo-Rea, F., Velásquez, W., & Rengifo, J. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109149 · Cited by 71 · see citing works
Zheng, Y., Yan, Z., Chen, K., Sun, J., Xu, Y. H., & Liu, Y. (2021) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2021.3062700 · Cited by 70 · see citing works
“…many problems in power systems can be tackled by leveraging DRL models, including electric vehicle scheduling [11]–[12], price-based demand response [13]–[14], load frequency control [15], voltage control [16], [17] and emergency control [18], [38]–[40], and network reconfiguration [19]–[20].”
BackgroundSelf-citationShort-term district power load self-prediction based on improved XGBoost model
Cao, W., Liu, Y., Mei, H., et al. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106826 · Cited by 70 · see citing works
Shuvo, S. S., & Yılmaz, Y. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3158814 · Cited by 68 · see citing works
“Hence, the possible actions for the AC remain turn on or off instead of different power ratings, (iv) We include EV battery depletion, which is overlooked in [26].”
BackgroundMethodResultInfluentialNovel Architecture of Energy Management Systems Based on Deep Reinforcement Learning in Microgrid
Lee, S., Seon, J., Sun, Y. G., Kim, S. H., Kyeong, C., Kim, D. I., & Kim, J. Y. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3317096 · Cited by 72 · see citing works
“In [23], Q-learning-based EMS has been proposed to reduce the electricity bill and DR-induced dissatisfaction cost.”
BackgroundMethodInfluentialA review of solar and wind energy forecasting: From single-site to multi-site paradigm
Verdone, A., Panella, M., Santis, E. D., & Rizzi, A. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.126016 · Cited by 69 · see citing works
Sierla, S., Pourakbari‐Kasmaei, M., & Vyatkin, V. V. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104174 · Cited by 67 · see citing works
A review of machine learning applications in IoT-integrated modern power systems
Farhoumandi, M., Zhou, Q., & Shahidehpour, M. (2020) · The Electricity Journal · doi:10.1016/j.tej.2020.106879 · Cited by 69 · see citing works
A safe reinforcement learning approach for multi-energy management of smart home
Ding, H., Xu, Y. H., Chew, B. S. H., et al. (2022) · Electric Power Systems Research · doi:10.1016/j.epsr.2022.108120 · Cited by 65 · see citing works
Self-citationModel-Based and Data-Driven HVAC Control Strategies for Residential Demand Response
Kou, X., Du, Y., Li, F. F., et al. (2021) · IEEE Open Access Journal of Power and Energy · doi:10.1109/oajpe.2021.3075426 · Cited by 64 · see citing works
“When considering multiple households, most data-driven algorithms have to deal with the exponential combination of binary control actions, which easily results in an out-of-memory issue when the number of households is large [21].”
BackgroundRaza, A., Li, J.-Z., Adnan, M., & Ahmad, I. (2024) · Results in Engineering · doi:10.1016/j.rineng.2024.102188 · Cited by 65 · see citing works
Michailidis, P., Michailidis, I., & Kosmatopoulos, E. B. (2025) · Energies · doi:10.3390/en18071724 · Cited by 72 · see citing works
Chen, T., Cui, Q., Gao, C., et al. (2021) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.12179 · Cited by 60 · see citing works
“In [21], it also uses multi-agent reinforcement learning to solve an energy management problem, meanwhile leveraging the extreme learning machine for prediction of electricity price and solar photovoltaic generation.”
MethodMulti-agent-based decentralized residential energy management using Deep Reinforcement Learning
Kumari, A., Kakkar, R., Tanwar, S., et al. (2024) · Journal of Building Engineering · doi:10.1016/j.jobe.2024.109031 · Cited by 58 · see citing works
An Advanced Satisfaction-Based Home Energy Management System Using Deep Reinforcement Learning
Forootani, A., Rastegar, M., & Jooshaki, M. (2022) · IEEE Access · doi:10.1109/access.2022.3172327 · Cited by 57 · see citing works
“References [7] and [10]-[11] deal with residential energy management via Q-Learning, a prevalent model-free algorithm.”
BackgroundMethodInfluentialLv, X., Li, X., & Xu, C. (2022) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2022.11.220 · Cited by 56 · see citing works
Yu, P., Zhang, H., Song, Y., Hui, H., & Chen, G. (2023) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2023.3237888 · Cited by 58 · see citing works
“Xu et al. [20] adopt DRL to schedule home energy consumption considering uncertain PV generation.”
BackgroundLi, S., Hu, W., Cao, D., Zhang, Z., Huang, Q. P., Chen, Z., & Blaabjerg, F. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3139650 · Cited by 56 · see citing works
“…performance of the proposed approach, five benchmarks, including uncontrolled approach; independent-SAC (i-SAC) approach based on the framework of [48]; multiagent-SAC (MA-SAC) approach [49]; multiactor-attention-critic (MAAC) approach [22]; centralized-SAC (c-SAC) approach based on the framework…”
BackgroundMethodChen, S.-J., Chiu, W., & Liu, W.-J. (2021) · IEEE Access · doi:10.1109/access.2021.3132962 · Cited by 55 · see citing works
“In [22], [23], and [25], the authors formulated a DR problem as a single objective RL (SORL) problem.”
BackgroundA systematic survey on demand response management schemes for electric vehicles
Kakkar, R., Agrawal, S. S., & Tanwar, S. (2024) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2024.114748 · Cited by 56 · see citing works
Abdelaal, G., Gilany, M. I., Elshahed, M., Sharaf, H. M., & El’Gharably, A. (2021) · IEEE Access · doi:10.1109/access.2021.3068421 · Cited by 55 · see citing works
“The relevant researches of the second category have been reported in [27]–[37].”
BackgroundDeep reinforcement learning-based joint load scheduling for household multi-energy system
Zhao, L., Yang, T., Li, W., et al. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.119346 · Cited by 54 · see citing works
Economic-Effective Multi-Energy Management Considering Voltage Regulation Networked With Energy Hubs
Zhao, P., Gu, C., Cao, Z., et al. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.3025861 · Cited by 54 · see citing works
“INTEGRATED energy systems (IES) are attracting increasing research attention due to the flexibility to coordinate and contemplate multi-energy infrastructures among each subsystem with promising storage technologies [1]–[3].”
BackgroundRecent Techniques Used in Home Energy Management Systems: A Review
Gomes, I. L. R., Bot, K., Ruano, M. G., et al. (2022) · Energies · doi:10.3390/en15082866 · Cited by 53 · see citing works
Smart Home’s Energy Management Through a Clustering-Based Reinforcement Learning Approach
Zenginis, I., Vardakas, J. S., Koltsaklis, N. E., et al. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3152586 · Cited by 49 · see citing works
“The Q-learning algorithm is applied in [22]–[24] for the schedul-”
MethodDinh, H. T., Lee, K.-H., & Kim, D. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.119382 · Cited by 48 · see citing works
Yavuz, M., & Kıvanç, Ö. C. (2024) · IEEE Access · doi:10.1109/access.2024.3370922 · Cited by 48 · see citing works
“Xu and et al. proposed a multi-agent Q-Learning HEMS, scheduling appliances and EVs to minimize energy costs approximately by 44.53% for short-term period [85].”
BackgroundInfluentialSeo, G., Yoon, S., Kim, M., Mun, C., & Hwang, E. (2021) · IEEE Access · doi:10.1109/access.2021.3094466 · Cited by 47 · see citing works
“This framework was inspired by [7], [20], [35].”
BackgroundMethodAlmughram, O., Slama, S. A. B., & Zafar, B. A. (2023) · Applied Sciences · doi:10.3390/app13095539 · Cited by 46 · see citing works
A Survey of Multi-Agent Systems for Smartgrids
İzmirlioğlu, Y., Pham, L. H., Son, T. C., & Pontelli, E. (2024) · Energies · doi:10.3390/en17153620 · Cited by 47 · see citing works
Jawad, S., & Liu, J. (2020) · Energies · doi:10.3390/en13133371 · Cited by 46 · see citing works
A Comprehensive Review of Digital Twin Technology in Building Energy Consumption Forecasting
Boukaf, M., Fadli, F., & Meskin, N. (2024) · IEEE Access · doi:10.1109/access.2024.3498107 · Cited by 43 · see citing works
“Specifically, a novel approach using a neural network-based method and Q-learning to determine the optimal actions for appliances, allowing them to change their operation modes or delay usage to save energy, was introduced by [124].”
A Blockchain-Enabled Demand Management and Control Framework Driven by Deep Reinforcement Learning
Ma, R., Yi, Z., Xiang, Y., Shi, D., Xu, C., & Wu, H. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3146631 · Cited by 41 · see citing works
“To optimally control each load in response to the time-varying electricity price, a multiagent RL approach is used in [12] to make optimal scheduling decisions.”
MethodCui, C., & Xue, J. W. (2024) · Energy · doi:10.1016/j.energy.2024.130505 · Cited by 41 · see citing works
de Lima, T. D., Faria, P., & Vale, Z. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.115251 · Cited by 46 · see citing works
Reinforcement Learning for Electric Vehicle Charging Management: Theory and Applications
Michailidis, P., Michailidis, I., & Kosmatopoulos, E. B. (2025) · Energies · doi:10.3390/en18195225 · Cited by 51 · see citing works
Volt-VAR-Pressure Optimization of Integrated Energy Systems With Hydrogen Injection
Zhao, P., Lu, X., Cao, Z., et al. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.3028530 · Cited by 41 · see citing works
Overview of Distributed Machine Learning Techniques for 6G Networks
Muscinelli, E., Shinde, S. S., & Tarchi, D. (2022) · Algorithms · doi:10.3390/a15060210 · Cited by 41 · see citing works
“Voltage control [42], energy management [43], demand predictions [44], transient stability enhancement, and resilience enhancement [45] are some of the main areas of power systems where distributed learning is succinctly used.”
BackgroundChen, X., Qu, G., Tang, Y., Low, S. H., & Li, N. (2021) · CaltechAUTHORS (California Institute of Technology) · Cited by 42 · see citing works
Lu, J., Mannion, P., & Mason, K. (2022) · IET Smart Grid · doi:10.1049/stg2.12068 · Cited by 39 · see citing works
Li, Z., Xie, X., Cheng, Z., et al. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108768 · Cited by 39 · see citing works
Wang, J., Li, L., & Zhang, J. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108885 · Cited by 39 · see citing works
Dong, L., Lin, H., Ji, Q., Zhang, T., Zhang, S., & Pu, T. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.123870 · Cited by 40 · see citing works
Intelligent strategies for microgrid protection: A comprehensive review
Peter, N., Gupta, P., & Goel, N. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124901 · Cited by 40 · see citing works
Implementation of home energy management system based on reinforcement learning
Haq, E. U., Lyu, C., Xie, P., et al. (2021) · Energy Reports · doi:10.1016/j.egyr.2021.11.170 · Cited by 39 · see citing works
Self-citationSecure Multi-Party Household Load Scheduling Framework for Real-Time Demand-Side Management
Cheng, L., Zang, H., Wei, Z., & Sun, G. (2022) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2022.3221081 · Cited by 39 · see citing works
“Due to the high online computation speed, RL methods are considered feasible for decentralized real-time DSM [36], [37].”
MethodMulti-agent DRL-based data-driven approach for PEVs charging/discharging scheduling in smart grid
Wan, Y., Qin, J., Ma, Q., Fu, W., & Wang, S. (2022) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2022.01.016 · Cited by 38 · see citing works
Tapaskar, R. P., Revankar, P. P., & Ganachari, S. V. (2024) · World Electric Vehicle Journal · doi:10.3390/wevj15060222 · Cited by 36 · see citing works
Decentralized energy management system for smart microgrids using reinforcement learning
Darshi, R., Shamaghdari, S., Jalali, A., et al. (2023) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.12796 · Cited by 35 · see citing works
Next Generation of Multi-Agent Driven Smart City Applications and Research Paradigms
Arora, A., Jain, A., Yadav, D., Hassija, V., Chamola, V., & Sikdar, B. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3310528 · Cited by 32 · see citing works
Multi-Agent Reinforcement Learning for Smart Community Energy Management
Wilk, P., Wang, N., & Li, J. (2024) · Energies · doi:10.3390/en17205211 · Cited by 36 · see citing works
Rehman, U. U., Faria, P., Gomes, L., et al. (2025) · Process Integration and Optimization for Sustainability · doi:10.1007/s41660-025-00506-x · Cited by 35 · see citing works
“For instance, Xu et al. (2020) focus on the technical aspects of HEMS models but overlook user behavior modeling entirely.”
A real-time automated scheduling algorithm with PV integration for smart home prosumers
Sharda, S., Sharma, K., & Singh, M. (2021) · Journal of Building Engineering · doi:10.1016/j.jobe.2021.102828 · Cited by 35 · see citing works
Jahangir, H., Gougheri, S. S., Vatandoust, B., Golkar, M. A., Golkar, M. A., Ahmadian, A., & Hajizadeh, A. (2021) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2021.3100994 · Cited by 35 · see citing works
Meta-Reinforcement Learning-Based Transferable Scheduling Strategy for Energy Management
Xiong, L., Tang, Y. S., Liu, C., Mao, S., Meng, K., Dong, Z. Y., & Qian, F. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3240702 · Cited by 35 · see citing works
“algorithms, some of which are also considered in scheduling problem of energy storage system and shiftable loads [28], [29].”
BackgroundInfluentialSaberi, H., Zhang, C., & Dong, Z. Y. (2024) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2024.3386896 · Cited by 33 · see citing works
“In [15], with extreme learning machine for uncertainty prediction, a multi-agent RL method for decision making under multiple uncertainties is developed.”
MethodEfficient and Private Scheduling of Wireless Electric Vehicles Charging Using Reinforcement Learning
Hossain, M. B., Pokhrel, S. R., & Vu, H. L. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3230012 · Cited by 32 · see citing works
“The values of GA and ML parameters are chosen (based on successful empirical findings [30], [37]) for tractability and to guarantee the maximization of the cumulative reward of the underlying dynamical system.”
MethodContrastive Identity-Aware Learning for Multi-Agent Value Decomposition
Liu, S., Zhou, Y., Song, J., Zheng, T., Chen, K., Zhu, T., Feng, Z., & Song, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i10.26370 · Cited by 32 · see citing works
Si, C., Tao, Y., Qiu, J., et al. (2021) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-020-01266-5 · Cited by 32 · see citing works
- Authors
- Xu, X., Jia, Y., Xu, Y., Xu, Z., Chai, S., & Lai, C. S.
- Affiliations
- Nanyang Technological University (SG)
- Southern University of Science and Technology (CN)
- Hong Kong Polytechnic University (HK)
- Guangdong University of Technology (CN)
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2020-07
- Volume
- 11
- Issue
- 4
- Pages
- 3201-3211
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W3005335721
- Semantic Scholar
- 8d06a30a270ff6dc763458dcedeea500fbf249e0
- Cited by
- 420
- References
- 37
- Citations by year
- 2026: 46 · 2025: 77 · 2024: 70 · 2023: 71 · 2022: 74 · 2021: 61 · 2020: 21
- FWCI
- 19.50
- Type
- journal article
- Language
- en
- Open access
- green
- bura.brunel.ac.uk
- Licence
- Locations
- IEEE Transactions on Smart Grid (publishedVersion)
- Brunel University Research Archive (BURA) (Brunel University London) (submittedVersion)
- Funders
- SUSTech Faculty Startup Funding (Y01236135, Y01236235)
- Young Talent Program (2018KQNCX223)
- Nanyang Assistant Professorship from Nanyang Technological University, Singapore
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
- Physical Sciences › Engineering › Electrical and Electronic Engineering › IoT-based Smart Home Systems
- Keywords
- home energy management systems, demand response, multi-agent reinforcement learning, Q-learning, finite Markov decision processes, extreme learning machine, uncertainty prediction, solar photovoltaic generation, electric vehicle scheduling, home appliance scheduling, electricity price forecasting, cost minimization, user dissatisfaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
