Optimising residential electric vehicle charging under renewable energy: Multi‐agent learning in software simulation and hardware‐in‐the‐loop evaluation
Open accessAbstract
The integration of intermittent renewable energy sources coupled with the increasing demand of electric vehicles (EVs) poses new challenges to the electrical grid. To address this, many solutions based on demand response have been presented. These solutions are typically tested only in software-based simulations. In this paper, we present the application in hardware-in-the-loop (HIL) of a recently proposed algorithm for decentralised EV charging, prediction-based multi-agent reinforcement learning (P-MARL), to the problem of optimal EV residential charging under intermittent wind power and variable household baseload demands. P-MARL is an approach that can address EV charging objectives in a demand response aware manner, to avoid peak power usage while maximising the exploitation of renewable energy sources. We first train and test our algorithm in a residential neighbourhood scenario using GridLAB-D, a software power network simulator. Once agents learn optimal behaviour for EV charging while avoiding peak power demand in the software simulator, we port our solution to HIL while emulating the same scenario, in order to decrease the effects of agent learning on power networks. Experimental results carried out in a laboratory microgrid show that our approach makes full use of the available wind power, and smooths grid demand while charging EVs for their next day's trip, achieving a peak-to-average ration of 1.67, down from 2.24 in the baseline case. We also provide an analysis of the additional demand response effects observed in HIL, such as voltage drops and transients, which can impact the grid and are not observable in the GridLAB-D software simulation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There has been limited work in real-time control algorithm validation, but with control algorithms different than AIMD.
“There has also been limited work in real-time control algorithm validation, but with control algorithms different than AIMD [13].”
From Controller Hardware-in-the-loop (C-HIL) Testing of Decentralized EV-Grid Integration (2022) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Farhangi, H. (2010) · IEEE Power and Energy Magazine · doi:10.1109/mpe.2009.934876 · Cited by 2,967 · see citing works
Demand Side Management: Demand Response, Intelligent Energy Systems, and Smart Loads
Pálenský, P., & Dietrich, D. (2011) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2011.2158841 · Cited by 2,959 · see citing works
Demand side management in smart grid using heuristic optimization
Logenthiran, T., Srinivasan, D., & Shun, T. Z. (2012) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2195686 · Cited by 1,229 · see citing works
Developing Theory Through Simulation Methods
Davis, J., Eisenhardt, K. M., & Bingham, C. B. (2007) · Academy of Management Review · doi:10.5465/amr.2007.24351453 · Cited by 1,168 · see citing works
McArthur, S. D. J., Davidson, E. M., Catterson, V. M., Dimeas, A., Hatziargyriou, N. D., Ponci, F., & Funabashi, T. (2007) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2007.908471 · Cited by 1,094 · see citing works
A genetic algorithm for the Flexible Job-shop Scheduling Problem
Pezzella, F., Morganti, G., & Ciaschetti, G. (2007) · Computers & Operations Research · doi:10.1016/j.cor.2007.02.014 · Cited by 1,019 · see citing works
A Survey on Demand Response Programs in Smart Grids: Pricing Methods and Optimization Algorithms
Vardakas, J. S., Zorba, N., & Verikoukis, C. V. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2341586 · Cited by 951 · see citing works
Demand Response as a Market Resource Under the Smart Grid Paradigm
Rahimi, F. A., & Ipakchi, A. (2010) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2010.2045906 · Cited by 853 · see citing works
Dependable Demand Response Management in the Smart Grid: A Stackelberg Game Approach
Maharjan, S., Zhu, Q., Zhang, Y., Gjessing, S., & Başar, T. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2223766 · Cited by 784 · see citing works
Multi-agent systems in a distributed smart grid: Design and implementation
Pipattanasomporn, M., Feroze, H., & Rahman, S. (2009) · IEEE PES Power Systems Conference and Exposition · doi:10.1109/psce.2009.4840087 · Cited by 724 · see citing works
Demand response experience in Europe: Policies, programmes and implementation
Torriti, J., Hassan, M., & Leach, M. A. (2009) · Energy · doi:10.1016/j.energy.2009.05.021 · Cited by 583 · see citing works
Optimal Charging of Electric Vehicles in Low-Voltage Distribution Systems
Richardson, P., Flynn, D., & Keane, A. (2012) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2011.2158247 · Cited by 566 · see citing works
GridLAB-D: An open-source power systems modeling and simulation environment
Chassin, D. P., Schneider, K. P., & Gerkensmeyer, C. (2008) · IEEE PES Transmission and Distribution Conference and Exhibition · doi:10.1109/tdc.2008.4517260 · Cited by 429 · see citing works
Optimal residential community demand response scheduling in smart grid
Nan, S., Zhou, M. Q., & Li, G. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.06.066 · Cited by 319 · see citing works
Serban, I., & Marinescu, C. (2013) · IEEE Transactions on Power Electronics · doi:10.1109/tpel.2013.2283298 · Cited by 281 · see citing works
Self-citationOptimal Demand Response Using Device-Based Reinforcement Learning
Wen, Z., O’Neill, D. C., & Maei, H. R. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2396993 · Cited by 271 · see citing works
Demand Response Management in the Smart Grid in a Large Population Regime
Maharjan, S., Zhu, Q., Zhang, Y., et al. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2431324 · Cited by 224 · see citing works
GridLAB-D: An Agent-Based Simulation Framework for Smart Grids
Chassin, D. P., Fuller, J., & Djilali, N. (2014) · Journal of Applied Mathematics · doi:10.1155/2014/492320 · Cited by 202 · see citing works
Jeon, J.-H., Kim, J., Kim, H., et al. (2010) · IEEE Transactions on Power Electronics · doi:10.1109/tpel.2010.2078518 · Cited by 182 · see citing works
Simulating Cyber-Physical Energy Systems: Challenges, Tools and Methods
Pálenský, P., Widl, E., & Elsheikh, A. M. G. (2013) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmcc.2013.2265739 · Cited by 161 · see citing works
Faria, P., Soares, J., Vale, Z., Morais, H. G. V., & Sousa, T. M. (2013) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2235866 · Cited by 137 · see citing works
A flexible distributed framework for realising electric and plug-in hybrid vehicle charging policies
Stüdli, S., Crisostomi, E., Middleton, R. H., & Shorten, R. N. (2012) · International Journal of Control · doi:10.1080/00207179.2012.679970 · Cited by 113 · see citing works
A Power Hardware-in-the-Loop Platform With Remote Distribution Circuit Cosimulation
Palmintier, B. S., Lundstrom, B., Chakraborty, S., Williams, T., Schneider, K. P., & Chassin, D. P. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2367462 · Cited by 104 · see citing works
Stabilization of Power Hardware-in-the-Loop simulations of electric energy systems
Viehweider, A., Lauss, G. F., & Lehfuß, F. (2011) · Simulation Modelling Practice and Theory · doi:10.1016/j.simpat.2011.04.001 · Cited by 100 · see citing works
Multi-Agent Optimal Allocation of Energy Storage Systems in Distribution Systems
Zheng, Y. J., Hill, D. J., & Dong, Z. Y. (2017) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2017.2705838 · Cited by 100 · see citing works
Robust Broadcast-Communication Control of Electric Vehicle Charging
Turitsyn, K. S., Sinitsyn, N. A., Backhaus, S., et al. (2010) · IEEE International Conference on Smart Grid Communications (SmartGridComm) · doi:10.1109/smartgrid.2010.5622044 · Cited by 93 · see citing works
Distributed nonlinear control of mobile autonomous multi-agents
Liu, T., & Jiang, Z. (2014) · Automatica · doi:10.1016/j.automatica.2014.02.023 · Cited by 89 · see citing works
Residential electrical demand forecasting in very small scale: An evaluation of forecasting methods
Marinescu, A., Harris, C., Dusparić, I., et al. (2013) · doi:10.1109/se4sg.2013.6596108 · Cited by 82 · see citing works
Self-citationRezkallah, M., Hamadi, A., Chandra, A., et al. (2015) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2015.2436333 · Cited by 81 · see citing works
Weckx, S., D’hulst, R., Claessens, B., & Driesen, J. (2014) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2014.2345886 · Cited by 79 · see citing works
Prediction-Based Multi-Agent Reinforcement Learning in Inherently Non-Stationary Environments
Marinescu, A., Dusparić, I., & Clarke, S. (2017) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3070861 · Cited by 71 · see citing works
Self-citationIntelligent Demand Response Scheme for Customer Side Load Management
Dam, Q. B., Mohagheghi, S., & Stoupis, J. (2008) · doi:10.1109/energy.2008.4781013 · Cited by 68 · see citing works
Wind power smoothing using demand response of electric vehicles
Raoofat, M., Saad, M., Lefebvre, S., Asber, D., Mehrjedri, H., & Lenoir, L. (2018) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2017.12.017 · Cited by 65 · see citing works
Hajibandeh, N., Shafie‐khah, M., Osório, G. J., et al. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.12.076 · Cited by 51 · see citing works
Accelerating Learning in multi-objective systems through Transfer Learning
Taylor, A., Dusparić, I., Galván, E., Clarke, S., & Cahill, V. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889438 · Cited by 48 · see citing works
Self-citationStudies in power hardware in the loop (PHIL) simulation using real-time digital simulator (RTDS)
Dargahi, M., Ghosh, A., Ledwich, G. F., et al. (2012) · IEEE International Conference on Power Electronics, Drives and Energy Systems · doi:10.1109/pedes.2012.6484500 · Cited by 43 · see citing works
Residential demand response: Experimental evaluation and comparison of self-organizing techniques
Dusparić, I., Taylor, A., Marinescu, A., Golpayegani, F., & Clarke, S. (2017) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.07.033 · Cited by 43 · see citing works
Self-citationEvaluating Policies for Parking Lots Handling Electric Vehicles
Babić, J., Carvalho, A., Ketter, W., & Podobnik, V. (2017) · IEEE Access · doi:10.1109/access.2017.2777098 · Cited by 39 · see citing works
Al‐Hammouri, A. T., Nordström, L., Chenine, M., et al. (2012) · IEEE Power and Energy Society General Meeting (PESGM) · doi:10.1109/pesgm.2012.6344949 · Cited by 36 · see citing works
Maximizing renewable energy use with decentralized residential demand response
Dusparić, I., Taylor, A., Marinescu, A., et al. (2015) · IEEE International Smart Cities Conference (ISC2) · doi:10.1109/isc2.2015.7366212 · Cited by 35 · see citing works
Self-citationHardware-in-the-Loop Wind Turbine Simulation Platform for a Laboratory Feeder Model
Hardy, T. D., & Jewell, W. T. (2014) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2014.2320600 · Cited by 25 · see citing works
Set point control for charging of electric vehicles on the distribution network
Harris, C., Dusparić, I., Galván, E., et al. (2014) · doi:10.1109/isgt.2014.6816415 · Cited by 18 · see citing works
A self-supporting approach to EV agent participation in smart grid
Habibidoost, M., & Bathaee, S. M. T. (2018) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2018.01.003 · Cited by 18 · see citing works
Examining system-wide impacts of solar PV control systems with a power hardware-in-the-loop platform
Williams, T., Fuller, J., Schneider, K. P., et al. (2014) · Conference record of the IEEE Photovoltaic Specialists Conference · doi:10.1109/pvsc.2014.6925336 · Cited by 9 · see citing works
A hardware-in-the-loop simulator for demand response energy management in industrial facilities
Luo, Z., Alam, M., Hong, S. H., et al. (2015) · Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES) · doi:10.1109/mscpes.2015.7115396 · Cited by 8 · see citing works
Supporting the dynamic frequency response in microgrids by means of active loads
Serban, I., & Ion, C. P. (2016) · Proceedings of the Annual Conference of the IEEE Industrial Electronics Society · doi:10.1109/iecon.2016.7793493 · Cited by 8 · see citing works
Self-citationEfficient Predictive Demand Response using Laguerre functions
McNamara, P., & McLoone, S. (2013) · IEEE Power & Energy Society General Meeting (PESGM) · doi:10.1109/pesmg.2013.6672963 · Cited by 3 · see citing works
Proceedings of the 44th IEEE conference on Decision and Control, December 2005 Seville, Spain
Iftime, O., Sandovici, & Golo (2005) · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Ren W, Steurer M, Woodruff S. Accuracy evaluation in power hardware-in-the-loop simulation center for advanced power sys- tems. In: Summer Computer Simulation Conference; 2007; San Diego, CA, USA:489-493.
- Watkins CJ, Dayan P. Q-learning. Machine learning. 1992;8:279-292.
- 1099114x, 2019, 8, · DOI
Cited by
Sorted by citations
Controller Hardware-in-the-loop (C-HIL) Testing of Decentralized EV-Grid Integration
Flack, C., Ucer, E., Smith, C. P., & Kisacikoglu, M. (2022) · 2022 IEEE Power & Energy Society General Meeting (PESGM) · doi:10.1109/pesgm48719.2022.9917135 · Cited by 13 · see citing works
- Authors
- Marinescu, A., Taylor, A., Clarke, S., Serban, I., & Marinescu, C.
- Affiliations
- Trinity College Dublin (IE)
- Transilvania University of Brașov (RO)
- Corresponding author
- Marinescu, Andrei
- Venue
- International Journal of Energy Research
- Publisher
- Wiley
- Published
- 2019-05-08
- Volume
- 43
- Issue
- 8
- Pages
- 3853-3868
- ISSN
- 0363-907X (ISSN-L)
- 1099-114X
- OpenAlex
- W2944633713
- Cited by
- 9
- References
- 50
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 2 · 2022: 3 · 2021: 2
- FWCI
- 0.45
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- International Journal of Energy Research (publishedVersion)
- Funders
- Royal Irish Academy (Charlemont Grant 2018)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Optimal Power Flow Distribution
- Keywords
- electric vehicle charging, residential EV charging, multi-agent reinforcement learning, demand response, hardware-in-the-loop, wind power integration, peak shaving, microgrids, voltage drop, GridLAB-D
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
