The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning
Open accessAbstract
Driven by the recent advances in electric vehicle (EV) technologies, EVs have become important for smart grid economy. When EVs participate in demand response program which has real-time pricing signals, the charging cost can be greatly reduced by taking full advantage of these pricing signals. However, it is challenging to determine an optimal charging strategy due to the existence of randomness in traffic conditions, user's commuting behavior, and the pricing process of the utility. Conventional model-based approaches require a model of forecast on the uncertainty and optimization for the scheduling process. In this paper, we formulate this scheduling problem as a Markov Decision Process (MDP) with unknown transition probability. A model-free approach based on deep reinforcement learning is proposed to determine the optimal strategy for this problem. The proposed approach can adaptively learn the transition probability and does not require any system model information. The architecture of the proposed approach contains two networks: a representation network to extract discriminative features from the electricity prices and a Q network to approximate the optimal action-value function. Numerous experimental results demonstrate the effectiveness of the proposed approach.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For load monitoring and electric vehicle charging scheduling, transfer learning [200] and deep reinforcement learning [215] methods are proposed.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [215]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Critical Review of Recurrent Neural Networks for Sequence LearningPreprint
Lipton, Z. C., Berkowitz, J. A., & Elkan, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00019 · Cited by 3,159
Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network
Kong, W., Dong, Z. Y., Jia, Y., Hill, D. J., Xu, Y. H., & Zhang, Y. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2753802 · Cited by 2,806
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Research works printed but not linked: 5
- bellman, 1958, Dynamic Programming
- 2018, Grid-Connected Renewable Energy Systems
- goodfellow, 2016, Deep Learning, 1
- 2017, The Egallon How Much Cheaper is it to Drive on Electricity?
- 2017, California Independent System Operator
Cited by
Sorted by citations
Items 201–300 of 547
Preemptive scheduling of EV charging for providing demand response services
Shao, S., Harirchi, F., Dave, D. N., & Gupta, A. K. (2022) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2022.100986 · Cited by 26
“[31] developed a deep reinforcement learning algorithm to determine the scheduling strategy given the EVs can charge and discharge at the station to shave the peak in grid loads.”
BackgroundKorkas, C. D., Tsaknakis, C. D., Kapoutsis, A. C., & Kosmatopoulos, E. B. (2024) · Energies · doi:10.3390/en17153694 · Cited by 25
Jendoubi, I., & Bouffard, F. (2022) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2022.100919 · Cited by 25
Ma, S., Liu, H., Wang, N., Huang, L., & Goh, H. H. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121838 · Cited by 25
Li, T. K., Xiao, Y., & Song, L. (2021) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2021.3073979 · Cited by 24
“For example, reinforcement learning (RL) is utilized for online scheduling of building energy [12] and electric vehicle charging systems [13].”
BackgroundMachine Learning for Solving Charging Infrastructure Planning Problems: A Comprehensive Review
Deb, S. (2021) · Energies · doi:10.3390/en14237833 · Cited by 25
“In [95,96], the authors used multiagent reinforcement learning for charging scheduling and proposed a dynamic pricing strategy.”
BackgroundReal-Time Charging Scheduling and Optimization of Electric Buses in a Depot
Verbrugge, B., Rauf, A., Rasool, H., Abdel-Monem, M., Geury, T., Baghdadi, M. E., & Hegazy, O. (2022) · Energies · doi:10.3390/en15145023 · Cited by 24
“Therefore, in [26], the authors developed a deep reinforcement learning (RL)-based approach to determine an optimal strategy for a real-time charging scheduling problem which does not require any system model information.”
BackgroundScheduling EV Charging Having Demand With Different Reliability Constraints
Shao, S., Sartipizadeh, H., & Gupta, A. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3279070 · Cited by 24
Hybrid Energy Storage System Optimization With Battery Charging and Swapping Coordination
Chen, X., Yang, Y. L., Song, J., Wang, J., & He, G. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3292189 · Cited by 24
“Wan et al. [22] developed a model-free algorithm based on reinforcement learning to obtain the optimal charging schedules of EVs, and demonstrated the effectiveness of the proposed approach by numerous experiments.”
MethodXiao, Q., Zhang, R., Wang, Y., Shi, P., Wang, X., Chen, B., Fan, C., & Chen, G. (2024) · Energy Reports · doi:10.1016/j.egyr.2024.10.056 · Cited by 23
Managing Distributed Flexibility Under Uncertainty by Combining Deep Learning With Duality
Τσαούσογλου, Γ., Mitropoulou, K., Steriotis, K., Paterakis, N. G., Pinson, P., & Varvarigos, E. (2021) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2021.3086846 · Cited by 23
“In [24], a penalty term is designed to teach a NN to also respect the local constraints of the agents.”
BackgroundMenos-Aikateriniadis, C., Sykiotis, S., & Georgilakis, P. S. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.110255 · Cited by 23
Flexibility estimation of electric vehicles and its impact on the future power grid
Wu, J., Li, L., Zhang, J.-F., & Xiao, B.-Y. (2025) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2024.110435 · Cited by 25
Yang, B. F., Zhou, Y., Yan, Y., Su, S., Li, J., Yao, W., Li, H., Gao, D., & Wang, J. (2025) · Protection and Control of Modern Power Systems · doi:10.23919/pcmp.2024.000297 · Cited by 22
Self-citationAn, D., Cui, F., & Kang, X. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2023.1273820 · Cited by 22
“This method ultimately resulted in the optimal charging strategy for reducing charging costs (Wan et al., 2018). innovatively used a long short-term memory (LSTM) network to extract electricity price features.”
MethodXie, J., Vorobev, P., Yang, R., & Nguyen, H. D. (2024) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2024.3460486 · Cited by 24
“Similarly, [8] presents a model-free deep reinforcement learning approach designed to manage uncertainties such as traffic conditions and user behavior.”
Optimal Energy Management Strategy for Smart Home with Electric Vehicle
Yi, Y., Verbič, G., & Chapman, A. C. (2021) · 2021 IEEE Madrid PowerTech · doi:10.1109/powertech46648.2021.9494787 · Cited by 16
“Deep Q-learning and Deep double Q-leaning were proposed in [12] and [13] respectively to minimize the EV charging cost in a smart home.”
BackgroundAutonomous charging of electric vehicle fleets to enhance renewable generation dispatchability
Bayani, R., Manshadi, S. D., Liu, G., Wang, Y., & Dai, R. (2021) · CSEE Journal of Power and Energy Systems · doi:10.17775/cseejpes.2020.04000 · Cited by 21
“The algorithm makes use of deep neural networks to reach an approximation of the Q value an is similar to the one used in [30].”
BackgroundMethodSupervised assisted deep reinforcement learning for emergency voltage control of power systems
Li, X., Wang, X., Zheng, X., Dai, Y., Yu, Z., Zhang, J. J., Bu, G., & Wang, F. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.12.043 · Cited by 21
Rouzbahani, H. M., Karimipour, H., & Lei, L. (2022) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2022.102712 · Cited by 21
A constrained DRL-based bi-level coordinated method for large-scale EVs charging
Ming, F., Gao, F., Liu, K., & Li, X. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.120381 · Cited by 23
Maity, A., & Sarkar, S. (2023) · International Journal of Green Energy · doi:10.1080/15435075.2023.2276174 · Cited by 25
“With the introduction of vehicle-to-grid (V2G) technology, Wan et al. (2018) and Li, Wan, and He (2019) have proposed reinforcement learning-based EV charging-discharging scheduling methods.”
BackgroundAli, A., Aftab, A., Akram, M. N., Awan, S., Muqeet, H. A., & Arfeen, Z. A. (2024) · Sustainability · doi:10.3390/su16052156 · Cited by 24
Prediction-Based EV-PV Coordination Strategy for Charging Stations Using Reinforcement Learning
Sun, F., Diao, R., Zhou, B., Lan, T., Mao, T., Su, S., Cheng, H., Meng, D., & Lü, S. (2023) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2023.3326433 · Cited by 20
“In [20], the historical electricity price is included into the input information of the RL agent to tackle the uncertainty from time-varying real-time price.”
BackgroundLiu, H., Yao, Y., Li, T., Du, M., Wang, X., Li, H., & Li, M. (2024) · Journal of Energy Storage · doi:10.1016/j.est.2024.112526 · Cited by 20
Self-citationZhang, F., Yang, Q., & An, D. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3288277 · Cited by 19
“In [14], authors adopted the DQN method to learn real-time charging decisions under random commuting behavior and dynamic energy price.”
MethodSi, C., Chau, K. T., Liu, W., & Hou, Y. (2025) · Journal of Energy Storage · doi:10.1016/j.est.2025.117500 · Cited by 20
Egan, D., Zhu, Q., & Prucka, R. G. (2023) · Energies · doi:10.3390/en16083450 · Cited by 18
Li, B., Li, J., & Han, M. (2023) · IEEE Access · doi:10.1109/access.2023.3296783 · Cited by 18
“In [48], authors propose an optimal EV charging strategy through DRL.”
BackgroundXu, H., Yu, Z., Zheng, Q., Hou, J., Wei, Y., & Zhang, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2949480 · Cited by 19
“When γ = 0, only the immediate reward is considered [21].”
BackgroundMethodInfluentialOn large action space in EV charging scheduling optimization
Jiang, Z., Jia, Q., & Guan, X. (2021) · Science China Information Sciences · doi:10.1007/s11432-020-3106-7 · Cited by 19
Colak, A., & Fescioglu-Unver, N. (2024) · Energy · doi:10.1016/j.energy.2024.133637 · Cited by 18
Kavousighahfarokhi, A., Hannan, M. A., Abu, S. M., Ker, P. J., Wong, R. T., & Jang, G. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.115904 · Cited by 22
Deep reinforcement learning-aided autonomous navigation with landmark generators
Wang, X., Sun, Y., Xie, Y., Bin, J., & Xiao, J. H. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1200214 · Cited by 23
Moniruzzaman, M., Yassine, A., & Hossain, M. S. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3345015 · Cited by 17
“The SOC boundaries deemed suitable for each EV and MCS correspond to 0.2 and 0.8 for each of their respective battery capacities [45].”
MethodInfluentialSun, C., Wen, X., Lu, Z., Zhang, J., & Chen, X. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3025721 · Cited by 18
“Unlike the schemes of charging scheduling for parked EVs that referred to the planning of charging durations [3]–[5], the charging scheduling schemes for on-the-move EVs mainly focus on the selections and navigations of CSs to charge.”
BackgroundOperational challenges and solution approaches for low voltage distribution grids — A review
Kotsonias, A., Hadjidemetriou, L., Asprou, M., & Panayiotou, C. G. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.111258 · Cited by 23
Lei, Y., Zhao, L., Gu, J., & Wang, J. (2025) · Energy · doi:10.1016/j.energy.2025.137051 · Cited by 17
Digital twin-driven deep reinforcement learning for real-time optimisation in dynamic AGV systems
Lee, D., Kang, Y.-S., & Noh, S. D. (2025) · International Journal of Production Research · doi:10.1080/00207543.2025.2543491 · Cited by 22
“To address these limitations, Deep Reinforcement Learning (DRL) has been introduced by integrating deep neural networks with RL, enabling more efficient handling of high-dimensional data and complex state representations (Guo et al. 2020; Hu et al.2020; Wan et al. 2018).”
Bang, H., & Malikopoulos, A. A. (2022) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc53348.2022.9867307 · Cited by 17
“, deep reinforcement learning [19], [20], mixed-integer linear programming [21]–[23], and a graph search algorithm [24].”
BackgroundMulti-objective optimal scheduling of charging stations based on deep reinforcement learning
Cui, F., Lin, X., Zhang, R., & Yang, Q. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2022.1042882 · Cited by 17
“Based on DRL, Chis et al. (2017) and Li et al. (2022) combined neural networks and DRL, which effectively reduced the charging cost.”
BackgroundInfluentialSaba, I., Tariq, M. H., Ullah, M., & Poor, H. V. (2023) · IET Smart Grid · doi:10.1049/stg2.12110 · Cited by 20
Li, J. Q., Li, H., Huang, T., Zheng, L., Ji, L., & Yin, S. (2023) · IEEE Transactions on Industrial Cyber-Physical Systems · doi:10.1109/ticps.2023.3294874 · Cited by 17
“Additionally, in contrast to previous researches [24], [25] that just take into account charging,”
BackgroundMethodResultSelf-citationGradient-Free Accelerated Event-Triggered Scheme for Constrained Network Optimization in Smart Grids
Hu, C., Zhang, X., & Wu, Q. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3315207 · Cited by 17
“A deep RL-based model-free method was investigated in [6] for the EV charging/discharging scheduling problem.”
MethodA Computing Budget Allocation Method for Minimizing EV Charging Cost Using Uncertain Wind Power
Jiang, Z., Jia, Q., & Guan, X. (2020) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.2995914 · Cited by 17
Intelligent Trainer for Dyna-Style Model-Based Deep Reinforcement Learning
Dong, L., Li, Y., Zhou, X., Wen, Y., & Guan, K. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3008249 · Cited by 20
Ultrafast Active Response Strategy against Malfunction Attack on Fault Current Limiter
Wei, F., Wan, Z., He, H., & Lin, X. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2960459 · Cited by 17
“The reason to adopt the LSTM network is that it has strong ability to model the time dependencies of time-series data [22], and has achieved promising results in the smart grid applications [23], [24].”
BackgroundSelf-citationZeng, P., Cui, S., Song, C., Wang, Z., & Li, G. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-06982-3 · Cited by 16
Zhou, P., Xu, Z., Zhu, X., Zhao, J., Song, C., & Shao, Z. (2023) · Information Sciences · doi:10.1016/j.ins.2023.02.084 · Cited by 16
Sui, Q., Li, F., Lin, X., Wei, F., Wu, C., Li, Z., & Xu, H. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2021.107933 · Cited by 15
Li, H., Han, B., Li, G., Wang, K., Xu, J. Q., & Khan, M. W. (2023) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.13047 · Cited by 15
Self-citationReinforcement Learning for Charging Scheduling in a Renewable Powered Battery Swapping Station
Renga, D., Spoturno, F., & Meo, M. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3404108 · Cited by 15
“Some works in the literature exploit deep RL methods to properly schedule the charging of a single EV, based on Q-learning [20], [21], [22] or safe deep RL approaches [23].”
BackgroundChen, W., Liu, D., & Cao, J. (2024) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2024.3453990 · Cited by 15
“The work in [7] also puts forward a model-free RL based method to optimize charging cost through bidirectional power flow.”
Optimizing capacitive deionization operation using dynamic modeling and reinforcement learning
Lee, S., Shim, J., Kim, H. H., Yun, N., Son, M., & Cho, K. H. (2025) · Desalination · doi:10.1016/j.desal.2025.118626 · Cited by 15
Chargym: An EV Charging Station Model for Controller Benchmarking
Karatzinis, G. D., Korkas, C. D., Terzopoulos, M., Tsaknakis, C. D., Stefanopoulou, A., Michailidis, I., & Kosmatopoulos, E. B. (2022) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-08341-9_20 · Cited by 14
Machine Learning-based Electric Vehicle User Behavior Prediction
Lilhore, A., Prasad, K. K., & Agarwal, V. (2023) · IEEE IAS Global Conference on Renewable Energy and Hydrogen Technologies (GlobConHT) · doi:10.1109/globconht56829.2023.10087780 · Cited by 14
“1 and reduces the transmission losses caused by the charging of EVs [3].”
BackgroundInfluentialCASA: cost-effective EV charging scheduling based on deep reinforcement learning
Zhang, A., Liu, Q., Liu, J., & Cheng, L. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09530-3 · Cited by 14
“Similarly, the study [25] address the EV charging scheduling problem using a model-free DRL approach.”
Ghasemi, A., Shojaeighadikolaei, A., & Hashemi, M. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3389653 · Cited by 24
“Recently, LSTM has been integrated into RL frameworks in several studies [27], [28], [29], [30], [31], [32].”
BackgroundMethodInfluentialBlockchain and Quantum Machine Learning Driven Energy Trading for Electric Vehicles
Kashyap, P. K., Dohare, U., Kumar, M., & Kumar, S. (2024) · Ad Hoc Networks · doi:10.1016/j.adhoc.2024.103632 · Cited by 14
Solar Power driven EV Charging Optimization with Deep Reinforcement Learning
Sykiotis, S., Menos-Aikateriniadis, C., Doulamis, A. D., Doulamis, N. D., & Georgilakis, P. S. (2022) · International Conference on Energy Transition in the Mediterranean Area (SyNERGY MED) · doi:10.1109/synergymed55767.2022.9941452 · Cited by 14
Foundation for adaptive charging solutions: Optimised use of electric vehicle charging capacity
Simolin, T., Rauma, K., Rautiainen, A., Järventausta, P., & Rehtanz, C. (2021) · IET Smart Grid · doi:10.1049/stg2.12043 · Cited by 14
Mobile battery energy storage system control with knowledge‐assisted deep reinforcement learning
Zhao, H. Q., Liu, Z., Mai, X., Zhao, J., Qiu, J., Liu, G., Dong, Z. Y., & Ghias, A. M. Y. M. (2022) · Energy Conversion and Economics · doi:10.1049/enc2.12075 · Cited by 13
Mechanism design of EVs fast charging rights for enhanced vehicle-to-grid regulation
Qian, T., Xu, Y., Jin, X., & Hu, Q. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124392 · Cited by 13
Machine Learning for Solving Charging Infrastructure Planning: A Comprehensive Review
Deb, S. (2021) · International Conference on Smart Grid and Smart Cities (ICSGSC) · doi:10.1109/icsgsc52434.2021.9490407 · Cited by 15
Wang, C., Chu, S., Yu, H., Ying, Y., & Chen, R. (2021) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2021.107724 · Cited by 13
Reinforcement Learning Environment for Cyber-Resilient Power Distribution System
Sahu, A., Venkatraman, V., & Macwan, R. (2023) · IEEE Access · doi:10.1109/access.2023.3282182 · Cited by 16
“…applied for decision support and control in a variety of power systems applications, such as voltage control [1]; frequency control [2]; energy management systems, such as optimal power flow [3]; electric vehicle charging scheduling [4]; battery management [5]; and residential load control [6].”
BackgroundDistributed Volt-VAR Optimization based on Multi-Agent Deep Reinforcement Learning
Li, H., Wang, Z., & He, H. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534348 · Cited by 13
Self-citationLiu, C., Xu, Z., Zhou, J., Yuan, Y., Ma, K., & Yuan, M. (2026) · Applied Energy · doi:10.1016/j.apenergy.2026.127463 · Cited by 14
Wei, H., Wu, J., Lv, C., Yang, S., Zhang, H., & Al‐Haddad, K. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3432075 · Cited by 12
“This integration paves the way for innovative advancements: (1) Real-time Predictive Charging: Har-nessing the predictive prowess of machine learning, the EV infrastructure is primed to foresee charging demands and align them with the grid’s energy provisioning [220].”
Ye, Y., Qiu, D., Ward, J., & Abram, M. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/48 · Cited by 12
“For instance, only seven discrete power levels are assumed for the EV charging in [ Wan et al. , 2019 ] and the action of the EES is limited to three options: fully charging, fully discharging or idle in [Chen and Su, 2018].”
BackgroundMethodDeep Reinforcement Learning for Economic Energy Scheduling in Data Center Microgrids
Yang, X., Wang, Y., He, H., Sun, C., & Zhang, Y. (2019) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm40551.2019.8974083 · Cited by 12
“Some existing works have verified its effectiveness in building energy optimization [11], electric vehicle scheduling [12], and in our previous work [13].”
BackgroundSelf-citationHe, J., Tang, J., Wang, N., Deng, Z., & Xiang, T. (2025) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2025.3641983 · Cited by 11
“With their superior e ffi ciency and zero-emission characteristics, EVs are rapidly replacing internal combustion engine vehicles, addressing the urgent need to decarbonize the transportation sector [2].”
Data analytics in the electricity market: a systematic literature review
Imani, M. H., Bompard, E. F., Colella, P., & Huang, T. (2023) · Energy Systems · doi:10.1007/s12667-023-00576-1 · Cited by 11
“To minimize the charging costs of vehicles considering the real-time price signals, [73] formulated the charging strategies as a Markov Decision Process (MDP).”
A DOD-SOH balancing control method for dynamic reconfigurable battery systems based on DQN algorithm
Yang, X., Liu, P., Liu, F., Liu, Z., Wang, D., Zhu, J., & Wei, T. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2023.1333147 · Cited by 11
Bayesian inference-based spatiotemporal modeling with interim activities for EV charging etiquette
Chen, B., Chen, Z., Liu, X. C., & Yi, Z. (2024) · Transportation Research Part D Transport and Environment · doi:10.1016/j.trd.2024.104060 · Cited by 14
Economical Electric Vehicle Charging Scheduling via Deep Imitation Learning
Huang, Z.-H., Wang, J., Fan, X., Yue, R., Xiang, C., & Gao, S. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3434734 · Cited by 11
“This problem is commonly known as EV charging scheduling (EVCS) in the literature [5], [6], [7], [8], [9], [10].”
BackgroundMethodResultInfluentialWang, H., Song, J., & Li, X. (2024) · CHAIN · doi:10.23919/chain.2024.000009 · Cited by 12
Peng, L., Lan, H., Yuan, Q., Jiang, Z., Cao, S., Ding, J., Wei, Q., Fan, J., Quan, C., Zhang, N., et al. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.125405 · Cited by 11
Abed, A. M., & AlArjani, A. S. (2022) · Energies · doi:10.3390/en15196992 · Cited by 11
Home Energy Management System based on Deep Reinforcement Learning Algorithms
Kahraman, A., & Yang, G. (2022) · IEEE PES Innovative Smart Grid Technologies Europe · doi:10.1109/isgt-europe54678.2022.9960575 · Cited by 11
“extracting the features from the LSTM network to deal with the uncertainty of electricity prices and user behaviour [6].”
BackgroundData-Driven Battery-Lifetime-Aware Scheduling for Electric Bus Fleets
Li, S., He, S., Wang, S., He, T., & Chen, J. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3369810 · Cited by 11
“[22] propose a model-free approach based on deep reinforcement learning to determine the optimal EV fleet charging strategy.”
BackgroundMethodAI-powered Energy Internet Towards Carbon Neutrality: Challenges and OpportunitiesPreprint
Li, C. (2021) · doi:10.36227/techrxiv.14787573.v1 · Cited by 13
“DQN 2018 [141] EV charging scheduling EV charging management is solved by deep reinforcement learning framework which involves in a Markov decision process with unknown transition probability.”
BackgroundMethodSkarlis, S. A., Molos, T., Ayfantopoulou, G., Nikiforiadis, A., & Bakouros, L. (2023) · Research in Transportation Business & Management · doi:10.1016/j.rtbm.2023.101040 · Cited by 10
A Progressive Planning Scheme for Dynamic Expansion of Electric Vehicle Charging Resources
Huang, Q., Zhou, G., Wang, H., Dong, Q., & Jia, Y. (2024) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2024.3410322 · Cited by 10
Lee, H., Kim, J., & Park, H. (2025) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2025.110539 · Cited by 10
Deep Reinforcement Learning for Distribution System Operations: A Tutorial and Survey
Glover, D., Krishnamoorthy, G., Ren, H., Dubey, A., & Gebremedhin, A. H. (2025) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3599840 · Cited by 10
“Ding et al. [62] and Wan et al. [92] discuss a real-time charging scheduling strategy formulated as an MDP, aiming to maximize the profit for the ADN system operator while mitigating system voltage deviations.”
Li, H., Li, G., Li, S. Y., Han, B., Wang, K., & Xu, J. (2023) · Asia Conference on Power and Electrical Engineering (ACPEE) · doi:10.1109/acpee56931.2023.10135734 · Cited by 10
“In [6] and [7], a deep Q-learning approach for EV charging control scheduling is proposed.”
BackgroundSelf-citationHe, S.-H., Wang, Y., Han, S., Zou, S.-F., & Miao, F. (2022) · IEEE/RJS International Conference on Intelligent RObots and Systems · doi:10.1109/iros55552.2023.10342342 · Cited by 18
He, S., Han, S., & Miao, F. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342263 · Cited by 10
“RL-based methods formulate the problem as a Markov Decision Process (MDP) and apply RL algorithms to find the optimal balancing policy [12], [13].”
BackgroundMethodA Review of Machine Learning Applications in Electricity Market Studies
Mohammadi, S., Hesamzadeh, M. R., Vafamehr, A., & Ferdowsi, F. (2020) · International Colloquium on Smart Grid Metrology (SMAGRIMET) · doi:10.23919/smagrimet48809.2020.9264022 · Cited by 10
“Wan et al. addressed the charging strategy of electric vehicles in [67] to minimize the charging costs of vehicles considering the real-time price signals.”
BackgroundQureshi, U. B., Ghosh, A., & Panigrahi, B. K. (2021) · IEEE International Conference on Sustainable Energy and Future Electric Transportation (SEFET) · doi:10.1109/sefet48154.2021.9375717 · Cited by 11
“[9], [10] have proposed the centralised charging scheduling while the authors of [11], [12] have put forward the decentralised or distributed control for making the charging decision.”
BackgroundLearning-based airborne sensor task assignment in unknown dynamic environments
Jing, H., Wang, Y., Liang, Y., Hu, J., & Yan, S. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104747 · Cited by 10
Heendeniya, C. B., & Nespoli, L. (2022) · Energy Informatics · doi:10.1186/s42162-022-00197-5 · Cited by 10
“…use-cases such as charging load forecasting (e.g., Zhang et al. 2021; Zhu et al. 2019), fleet assignment (e.g., Shi et al. 2020), charging station recommendation (e.g., Blum et al. 2021), and charging management (e.g., Chang et al. 2019; Wan et al. 2019; Ding et al. 2020; Dorokhova et al. 2021).”
BackgroundChen, S., Liu, Y., Guo, Z., Luo, H., Zhou, Y., Qiu, Y., Zhou, B., & Zang, T. (2023) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.12806 · Cited by 11
Li, Z., Sun‡, Y., Yang, H., Wang, S., Shen, Y., Wang, X., Zhang, K., & Anvari‐Moghaddam, A. (2023) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2023.109444 · Cited by 9
Zhang, Y., Li, K., Du, C., Cai, W., Lu, Y., & Feng, Y. (2024) · Journal of Energy Storage · doi:10.1016/j.est.2024.110474 · Cited by 9
Mishra, S., Choubey, A., Devarasetty, S. V., Sharma, N., & Misra, R. (2024) · Cluster Computing · doi:10.1007/s10586-024-04494-4 · Cited by 9
“Nonetheless, the methodologies examined in prior works [10, 11, 38] exhibit drawbacks stemming from extended temporal dependencies and substantial training time expenditures.”
InfluentialFang, D., Tang, H. J., Hatziargyriou, N. D., Zhang, T., Chen, W., & Zhang, Q. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.123453 · Cited by 9
Pan, G., Guan, X., Jiang, H., Liu, Y., Wu, H., Chen, H., Ohtsuki, T. O., & Han, Z. (2024) · Computer Networks · doi:10.1016/j.comnet.2024.110872 · Cited by 9
Yang, Y., Liang, J., & Feng, J. (2024) · Systems · doi:10.3390/systems12110472 · Cited by 9
Metadata
- Authors
- Wan, Z., Li, H., He, H., & Prokhorov, D.
- Affiliations
- University of Rhode Island (US)
- Shenyang Institute of Automation (CN)
- Toyota Motor North America Research & Development (United States) (US)
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2019-09
- Volume
- 10
- Issue
- 5
- Pages
- 5246-5257
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W2899639849
- Cited by
- 548
- References
- 46
- Citations by year
- 2026: 61 · 2025: 71 · 2024: 83 · 2023: 95 · 2022: 87 · 2021: 74 · 2020: 57 · 2019: 20
- FWCI
- 14.43
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- IEEE Transactions on Smart Grid (publishedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- SIA OpenIR (Chinese Academy of Sciences) (submittedVersion)
- Funders
- Office of Naval Research (N00014-18-1-2396)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Power System Optimization
- Keywords
- electric vehicle charging scheduling, real-time pricing, demand response, deep reinforcement learning, model-free learning, unknown transition probabilities, Markov decision process, Q-network, electricity price forecasting, smart grid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
