The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning
Open accessAbstract
Driven by the recent advances in electric vehicle (EV) technologies, EVs have become important for smart grid economy. When EVs participate in demand response program which has real-time pricing signals, the charging cost can be greatly reduced by taking full advantage of these pricing signals. However, it is challenging to determine an optimal charging strategy due to the existence of randomness in traffic conditions, user's commuting behavior, and the pricing process of the utility. Conventional model-based approaches require a model of forecast on the uncertainty and optimization for the scheduling process. In this paper, we formulate this scheduling problem as a Markov Decision Process (MDP) with unknown transition probability. A model-free approach based on deep reinforcement learning is proposed to determine the optimal strategy for this problem. The proposed approach can adaptively learn the transition probability and does not require any system model information. The architecture of the proposed approach contains two networks: a representation network to extract discriminative features from the electricity prices and a Q network to approximate the optimal action-value function. Numerous experimental results demonstrate the effectiveness of the proposed approach.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For load monitoring and electric vehicle charging scheduling, transfer learning [200] and deep reinforcement learning [215] methods are proposed.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [215]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Critical Review of Recurrent Neural Networks for Sequence LearningPreprint
Lipton, Z. C., Berkowitz, J. A., & Elkan, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00019 · Cited by 3,159
Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network
Kong, W., Dong, Z. Y., Jia, Y., Hill, D. J., Xu, Y. H., & Zhang, Y. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2753802 · Cited by 2,806
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Research works printed but not linked: 5
- bellman, 1958, Dynamic Programming
- 2018, Grid-Connected Renewable Energy Systems
- goodfellow, 2016, Deep Learning, 1
- 2017, The Egallon How Much Cheaper is it to Drive on Electricity?
- 2017, California Independent System Operator
Cited by
Sorted by citations
Items 301–400 of 547
Yu, K., Yan, P., Liu, Y., Chen, Z., & Kong, X. T. R. (2025) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2025.104006 · Cited by 9
A Survey on Deep Learning Applications for Electric Vehicles in Micro Grids
Zadeh, P. T., Joudaki, M., & Ansari, A. (2021) · doi:10.1109/iot52625.2021.9469715 · Cited by 9
He, G., Chen, X., Yang, Y. P., Wang, J., & Song, J. (2022) · IEEE/IAS Industrial and Commercial Power System Asia (I&CPS Asia) · doi:10.1109/icpsasia55496.2022.9949714 · Cited by 9
“Existing algorithms involve exact algorithms [10], heuristics [11], [12], metaheuristics [13] and deep reinforcement learning algorithms [14], [15].”
MethodA self-sustained EV charging framework with N-step deep reinforcement learning
Sykiotis, S., Menos-Aikateriniadis, C., Doulamis, A. D., Doulamis, N. D., & Georgilakis, P. S. (2023) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2023.101124 · Cited by 9
Li, B., Li, J., & Li, Z. (2024) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2024.08.285 · Cited by 8
Beyazıt, M. A., Salehizadeh, M. R., Demirel, E., Taşçıkaraoğlu, A., & Liu, J. J. (2025) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2025.102008 · Cited by 8
One-Shot Unsupervised Domain Adaptation for Object Detection
Wan, Z., Li, L., Li, H., He, H., & Ni, Z. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207244 · Cited by 8
“DNN can extract high-level from input data and has been widely used in many real-world applications [1], [3], [12].”
BackgroundSelf-citationAuza, A., Asadi, E., Chenari, B., & da Silva, M. G. (2023) · Energies · doi:10.3390/en16134983 · Cited by 8
Electric vehicle controllers for sustainable energy management: recommendations and challenges
Gunjal, M., Mathew, L., & Letha, S. S. (2023) · Environment Development and Sustainability · doi:10.1007/s10668-023-03561-x · Cited by 8
Li, B., Li, J., & Jian, B. (2023) · IEEE Access · doi:10.1109/access.2023.3330161 · Cited by 8
“Authors in [34] and [35] adopt the DRL algorithm to decide the dynamic pricing for EV charging.”
MethodLuo, Y., Zhu, J., Chen, J., Wu, R., Huang, H., Liu, W., & Liu, M. (2024) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2024.3395497 · Cited by 8
“To fill these gaps, the SED problem is re-formulated as the Markov Decision Process (MDP) in [17], [18], [19], [20], [21], [22], [23], [24], [25].”
BackgroundMethodInfluentialSECaaS-Based Partially Observable Defense Model for IIoT Against Advanced Persistent Threats
Zhang, Z., Ding, C., Li, Y., Yu, J., & Li, J. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3422870 · Cited by 8
“For uncertain environments, deep reinforcement learning has achieved promising results recently in many complex decision-making applications, such as games [30], smart power grids [31], autonomous driving [32], and security protection [33].”
Ma, Q., Tong, X., Huang, J., Wang, P., & Li, J. (2021) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.12293 · Cited by 8
Reinforcement Learning Based EV Charging Scheduling: A Novel Action Space Representation
Qian, K., Adam, R., & Brehm, R. (2021) · IEEE PES Innovative Smart Grid Technologies - Asia (ISGT Asia) · doi:10.1109/isgtasia49270.2021.9715603 · Cited by 8
“Note that the RL-based solutions in [4], [10] apply only for an individual EV.”
BackgroundMethodReinforcement learning-based demand-side management by smart charging of electric vehicles
Ozcelik, M. B., Kesici, M., Aksoy, N., & Genc, I. V. M. (2022) · Electrical Engineering · doi:10.1007/s00202-022-01597-2 · Cited by 8
“Although machine learning algorithms [6–12] or fuzzy logic control [13] can be utilized for the solving EV charging decision problems, RL algorithms can be considered as the most effective solution methods for the solution of such problems [14–21].”
National-scale bi-directional EV fleet control for ancillary service provision
Nespoli, L., Wiedemann, N., Süel, E., Xin, Y., Raubal, M., & Medici, V. (2023) · Energy Informatics · doi:10.1186/s42162-023-00281-4 · Cited by 8
“For example, Valogianni et al. (2013); Wan et al. (2018); Tuchnitz et al. (2021); Dang et al. (2019); Li et al. (2019) train a reinforcement learning (RL) agent to decide on charging behavior.”
BackgroundSultana, A., Moniruzzaman, M., & Zhao, L. Y. (2023) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2023-fall60731.2023.10333697 · Cited by 8
“The concept of charging behavior is significantly inspired by the approach taken in [26].”
MethodEconomic real-time energy trading services for electric vehicles with uncertain mobility demand
Merfeld, T., & Meisel, S. (2022) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2022.130639 · Cited by 8
Guo, K., Chen, Y., & Yang, Q. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.127096 · Cited by 7
Yang, A., Sun, H., & Zhang, X. (2021) · Journal of Engineering Science and Technology Review · doi:10.25103/jestr.143.12 · Cited by 7
Dolatabadi, A., Abdeltawab, H. H., & Mohamed, Y. A. I. (2024) · IEEE Access · doi:10.1109/access.2024.3352032 · Cited by 7
“There are, for example, only five and seven levels of discrete control actions for lithium-ion batteries and EVs in [22] and [23], respectively.”
Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning
Erüst, A. C., & Taşçıkaraoğlu, F. Y. (2024) · IET Intelligent Transport Systems · doi:10.1049/itr2.12588 · Cited by 7
Electric Vehicle Charging Scheduling Algorithm Based on Online Multi-objective Optimization
Hong, T., Cao, J., Zhao, W., & Lu, M. (2021) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc51323.2021.9498595 · Cited by 7
Day-ahead scheduling based on reinforcement learning with hybrid action space
Cao, J., Dong, L., & Sun, C. (2022) · Journal of Systems Engineering and Electronics · doi:10.23919/jsee.2022.000064 · Cited by 7
Deep Reinforcement Learning for Direct Load Control in Distribution Networks
Bahrami, S., Chen, Y. C., & Wong, V. W. S. (2020) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm41954.2020.9281703 · Cited by 7
“To address this challenge, there have been some efforts on applying deep reinforcement learning for load control in microgrids [6], residential buildings [7]–[9], and electric vehicles charging stations [10], [11].”
BackgroundMethodPrediction-Based EV-PV Coordination Strategy for Charging Stations Using Reinforcement Learning
Sun, F., Su, S., Diao, R., Cheng, H., Meng, D., & Lü, S. (2022) · IEEE Transportation Electrification Conference and Expo, Asia-Pacific (ITEC Asia-Pacific) · doi:10.1109/itecasia-pacific56316.2022.9942164 · Cited by 7
“In [10] and [11], influence of electricity price on drivers’ behavior is considered as key factors in RL-based EV control strategies.”
BackgroundWang, Y., Wu, Q., Li, Z., Tao, S., Xie, S., Zhang, X., & Chan, W. K. (2025) · IEEE Transactions on Energy Markets Policy and Regulation · doi:10.1109/tempr.2025.3558414 · Cited by 6
“By incorporating deep neural networks to mitigate the curse of dimensionality, DRL provides effective solutions for large-scale optimization and complex problems [20], [21], [22], [23].”
Rehman, A. U. (2026) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2026.116733 · Cited by 6
Prajeesh, C. B., Krishna Priya, R., Pillai, A. S., Khwaja, A. S., & Anpalagan, A. (2025) · e-Prime - Advances in Electrical Engineering Electronics and Energy · doi:10.1016/j.prime.2025.100926 · Cited by 6
An electrical vehicle-assisted demand response management system: A reinforcement learning method
Li, D., Yang, Q., Ma, L., Wang, Y., Zhang, Y., & Liao, X. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2022.1071948 · Cited by 6
“For example, Wan et al. (2019) proposed a model-free approach to determine the real-time optimal schedules based on deep reinforcement learning.”
BackgroundReinforcement learning-based scheduling of multi-battery energy storage system
Cheng, G., Dong, L., Yuan, X., & Sun, C. (2023) · Journal of Systems Engineering and Electronics · doi:10.23919/jsee.2023.000036 · Cited by 6
A Modified Incentive-based Demand Response Model using Deep Reinforcement Learning
Xu, H., Kuang, W., Lu, J., & Hu, Q. (2020) · Asia-Pacific Power and Energy Engineering Conference (APPEEC) · doi:10.1109/appeec48164.2020.9220364 · Cited by 6
“The DQN introduced in [22] make use of DNN to approximate the value function in order to solve the continuous state or action problem.”
MethodAdversarial Attack for Deep Reinforcement Learning Based Demand Response
Wan, Z., Li, H., Shuai, H., Sun, Y. L., & He, H. (2021) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm46819.2021.9637826 · Cited by 6
“For instance, Wan et al. [16] develop a DRL-based method to determine the optimal policy for electric vehicle (EV) charging management.”
BackgroundMethodSelf-citationAgent-Based Energy Sharing Mechanism Using Deep Deterministic Policy Gradient Algorithm
Kuang, Y., Wang, X., Zhao, H., Huang, Y., Chen, X., & Wang, X. (2020) · Energies · doi:10.3390/en13195027 · Cited by 6
“Furthermore, the DQN algorithm has been applied to some complex scenarios such as electric vehicle charge scheduling [31], the optimal management of the operation and maintenance of power system [32], energy trading game among smart grids [33], management of a hybrid energy generation system [34,35], and HVAC optimal…”
MethodAI-powered Energy Internet Towards Carbon Neutrality: Challenges and OpportunitiesPreprint
Li, C. (2021) · doi:10.36227/techrxiv.14787573 · Cited by 6
Lahariya, M., Sadeghianpourhamami, N., & Develder, C. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4090508 · Cited by 6
Xia, Y., Cheng, Z., Zhang, J., & Chen, X. (2025) · World Electric Vehicle Journal · doi:10.3390/wevj16030184 · Cited by 5
“Based on this feature, numerous researchers have employed DRL to study orderly charging within charging stations [27–30].”
Research and Application of Safe Reinforcement Learning in Power System
Li, J., Wang, X., Chen, S., & Yan, D. (2023) · Asia Conference on Power and Electrical Engineering (ACPEE) · doi:10.1109/acpee56931.2023.10135995 · Cited by 5
Smart Management of Electric Vehicle Chargers Through Reinforcement Learning
Abdullah, H. M., Gastli, A., & Ben‐Brahim, L. (2024) · International Conference on Smart Grid and Renewable Energy (SGRE) · doi:10.1109/sgre59715.2024.10428720 · Cited by 5
Zheng, Y., Miao, X., Wang, H., Li, X., & Hansen, T. M. (2025) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2025.3577381 · Cited by 5
“Wan et al. [16] proposed an augmented Lagrangian-based safe off-policy deep RL (DRL) algorithm for the carbon-oriented optimal scheduling of EV aggregators in a distribution network.”
Joshi, A., Tipaldi, M., & Glielmo, L. (2025) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2025.101790 · Cited by 5
Multi-Agent DDPG-Based Multi-Device Charging Scheduling for IIoT Smart Grids
Zeng, H., Huang, Y., Zhan, K., Yu, Z., Zhu, H., & Li, F. (2025) · Sensors · doi:10.3390/s25175226 · Cited by 5
“Single-agent Deep Q-Network (DQN) approaches [19], while effective for individual EV optimization, suffer from the curse of dimensionality when extended to multi-agent settings and cannot handle continuous charging power control effectively.”
Load Scheduling for an Electric Water Heater With Forecasted Price Using Deep Reinforcement Learning
Cao, J., Dong, L., & Xue, L. (2020) · Chinese Automation Congress · doi:10.1109/cac51589.2020.9326475 · Cited by 5
“Several articles employ RL approach for optimization of load scheduling [9]-[13].”
MethodA Review of Deep Reinforcement Learning Applications in Power System Parameter Estimation
Wang, Y., Chai, B., Lu, W., & Zheng, X. (2021) · International Conference on Power System Technology (POWERCON) · doi:10.1109/powercon53785.2021.9697763 · Cited by 5
“In [18], EV charging control was scheduled through discrete DRL version Deep Q network (DQN) approach.”
MethodCollaborative optimal dispatch of microgrid and electric vehicles based on the Stackelberg game
Bo, W., Zhang, W., Du, R., Wang, D., Zhao, Y., Li, Q., & Zhang, Z. (2022) · Frontiers in Energy Research · doi:10.3389/fenrg.2022.974282 · Cited by 5
“At the same time, the scheduling of the charging load has been considered from the perspectives of both time and space (Wan et al., 2018; Yang et al., 2020; Wang et al., 2021).”
BackgroundOptimal Scheduling for EV Charging Stations with Varying Power Limits Based on MADDPG Algorithm
Fang, C., Zhu, S., Zhang, L., & Huang, J. (2023) · Power System and Green Energy Conference (PSGEC) · doi:10.1109/psgec58411.2023.10255946 · Cited by 5
“In [5], to deal with uncertainties of EV users’ behaviors and real-time electricity price, a deep Q network (DQN) was used to schedule the EV charging.”
MethodAn Efficient Distributed Multi-Agent Reinforcement Learning for EV Charging Network Control
Shojaeighadikolaei, A., & Hashemi, M. (2023) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton58177.2023.10313469 · Cited by 5
“Similarly, in [13], the authors utilized two networks, including a representation network to extract discriminative features from the electricity prices and a Q-network to approximate the optimal charging/discharging power for an individual EV.”
MethodElectric vehicle charging design: The factored action based reinforcement learning approach
Truong, V. B., & Le, L. B. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.122737 · Cited by 6
Multi-Agent DRL-Based Resource Scheduling and Energy Management for Electric Vehicles
Zhang, Z., Yu, C., & Tian, B. (2024) · Electronics · doi:10.3390/electronics13163311 · Cited by 6
Robust Frequency Control of Power Systems Under Time-varying Loads
Wang, Z., He, H., & Sun, Y. L. (2019) · North American Power Symposium (NAPS) · doi:10.1109/naps46351.2019.9000232 · Cited by 5
“The influence of renewable energy resources and electric vehicles is studies in [2] and [3].”
BackgroundSelf-citationLocational marginal price forecasting using Transformer-based deep learning network
Liao, S., Wang, Z., Luo, Y., & Liang, H. (2021) · Chinese Control Conference (CCC) · doi:10.23919/ccc52363.2021.9549619 · Cited by 5
“Moreover, the real-time scheduling of electric vehicles on the demand side also uses this framework [12].”
BackgroundConstrained Double Deep Q-learning Network for EVs Charging Scheduling with Renewable Energy
Ming, F., Gao, F., Liu, K., Wu, J., Xu, Z., & Li, W. (2020) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case48305.2020.9216999 · Cited by 5
“In [11]-[13], the EV charging problem was modeled as MDP without transition probability and DQN was used to get the optimal charging policy.”
MethodReinforcement Learning for Electric Vehicle Charging using Dueling Neural NetworksPreprint
Gokhale, G., Claessens, B., & Develder, C. (2021) · Preprints.org · doi:10.20944/preprints202103.0592.v1 · Cited by 5
“While [14] studies the charging of a single EV, Lee et 104 al.”
BackgroundMethodElectric vehicles coordination for grid balancing using multi-objective Harris Hawks Optimization
Pop, C. B., Cioara, T., Chifu, V. R., Anghel, I. M., & Bellesini, F. (2024) · Energy Reports · doi:10.1016/j.egyr.2024.08.049 · Cited by 4
Li, Y., Dong, B., Wang, X., & Qiu, Y. L. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.127032 · Cited by 4
Park, J.-W., & Yang, H.-S. (2026) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2026.102244 · Cited by 4
Zhao, Q., Wu, W., Cheng, N., Guo, G., & Han, Y. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3567097 · Cited by 4
“In recent years, DRL has found extensive application in various domains, including wireless communication [19], traffic planning [20], fire detection in smart cities [21], charging management in Vehicle-grid integration [22] and charging aggregation in smart grids [23], [24].”
Hu, H., Zhao, D. F., Zockaie, A., & Ghamami, M. (2025) · Journal of Management in Engineering · doi:10.1061/jmenea.meeng-7010 · Cited by 4
Li, D., Yang, Q., Ma, L., Peng, Z., & Liao, X. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2022.1071973 · Cited by 4
Shokati, M., Mohammadi, P., & Amirinian, A. (2024) · doi:10.1109/icemd64575.2024.10963615 · Cited by 4
Parametric study of adaptive reinforcement learning for battery operations in microgrids
Panda, D. K., Das, S., & Abusara, M. A. (2025) · Renewable Energy · doi:10.1016/j.renene.2025.123250 · Cited by 4
“…and optimization algorithm and has been widely used in the management of microgrids [3],[4],[5],[6],[7],[8], electric vehicles (EVs) charging [9],[10],[11], energy management in hybrid electric vehicles [12],[13],[14],[15],[16] and demand-side management (DSM) for scheduling appliances usage.”
Lin, Y., Yan, L., Hui, H., Yang, Q., Zhou, J., Chen, Y., Chen, X., & Wen, J. (2025) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2025.3577145 · Cited by 4
“The Deep-Q-Network (DQN) algorithm [11], [12], [13] is a classical discrete control method that combines Q-learning and DL.”
An application of reinforcement learning to residential energy storage under real-time pricing
Brock, E., Bruckstein, L., Connor, P., Nguyen, S., Kerestes, R. J., & Abdelhakim, M. (2021) · IEEE PES Innovative Smart Grid Technologies - Asia (ISGT Asia) · doi:10.1109/isgtasia49270.2021.9715712 · Cited by 4
Deep Reinforcement Learning based Applications in Smart Power Systems
Chen, Y., Peng, X., Xu, X., & Wu, H. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1881/2/022051 · Cited by 4
“Similarly, a model-free DRL method is proposed in [17] to determine the optimal strategy for real-time electric vehicle charging/discharging scheduling without any system model information.”
MethodRe-Routing Strategy of Connected and Automated Vehicles Considering Coordination at Intersections
Bang, H., & Malikopoulos, A. A. (2023) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc55779.2023.10156555 · Cited by 4
Real-Time Control of Electric Autonomous Mobility-on-Demand Systems via Graph Reinforcement Learning
Singhal, A., Gammelli, D., Luke, J., Gopalakrishnan, K., Helmreich, D., & Pavone, M. (2024) · European Control Conference (ECC) · doi:10.23919/ecc64448.2024.10591098 · Cited by 3
“For example, Wan et al. [9] consider charge scheduling for personal electric vehicles.”
BackgroundZhang, M., Wang, M., Li, F., Wang, L., & Zhang, H. (2025) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2025.3593892 · Cited by 3
“Statistical methods typically include Monte Carlo simulations [16], fuzzy linear regression [17], quantile linear regression [18], ARIMA models [19], and Kalman filters [20].”
Lao, B., Zhang, X., Liu, D., & Zou, Y. (2025) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2025.102087 · Cited by 3
Liu, C., Yang, X., Xiaoyan, L., & Qin, C. (2026) · World Electric Vehicle Journal · doi:10.3390/wevj17010047 · Cited by 3
“The primary drawbacks are low computational efficiency and the “curse of dimensionality” in large-scale systems [35,36].”
Yu, Z., Yang, C., & Wang, Q. (2026) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2026.117375 · Cited by 7
Shen, X., Li, J., Yu-jun, Y., Jianlin, T., Xiaoming, L., & Bin, Q. (2023) · Frontiers in Energy Research · doi:10.3389/fenrg.2023.1338807 · Cited by 3
“A deep reinforcement learning based approach was constructed to address optimal charging scheduling under uncertain electric prices (Wan et al., 2019).”
MethodBlatiak, A., Qiu, D., Štrbac, G., & Papadaskalopoulos, D. A. (2024) · International Conference on Smart Energy Systems and Technologies (SEST) · doi:10.1109/sest61601.2024.10694447 · Cited by 3
“Deep RL (DRL), which uses deep neural networks, is applied to the EV charging problem in [10] using a Deep Q-network (DQN) algorithm to produce a revenue maximizing charging schedule.”
BackgroundMethodEntropy-based dynamic ensemble classication algorithm for imbalanced data stream with concept drift
Gong, J.-C., & Dong, M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0311133 · Cited by 3
Greenhouse gas emissions trends and fleet renewal of ride-hailing in Toronto, Canada
Vieira, J. P. B., Saleh, M., & Hatzopoulou, M. (2025) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2025.146252 · Cited by 3
Edge Intelligence for Plug-in Electrical Vehicle Charging Service
Zhang, Y., Hong, F., Wang, Y., Zhi, L., Zhou, Y., Chang, Z., & Chen, G. Y. (2021) · IEEE Network · doi:10.1109/mnet.011.2000587 · Cited by 3
Li, X., Luchun, Y., Lihua, H., & Wang, C. (2022) · Applied Ocean Research · doi:10.1016/j.apor.2022.103132 · Cited by 3
Scheduling Electric Vehicle Fleets as a Virtual Battery under Uncertainty using Quantile Forecasts
Brinkel, N., Hu, J., Visser, L., van Sark, W. G. J. H. M., & AlSkaif, T. A. (2022) · IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm) · doi:10.1109/smartgridcomm52983.2022.9961004 · Cited by 3
“The work in [7] & [8] proposed a stochastic optimization method to schedule a single EV under uncertainty.”
MethodOptimal Dispatching of Active Distribution Network Based on Improved Genetic Algorithm
Luo, N., Liu, J., & Zhang, P. (2022) · IEEE International Conference on Frontiers Technology of Information and Computer (ICFTIC) · doi:10.1109/icftic57696.2022.10075109 · Cited by 3
“In reference [6], the random black hole particle swarm algorithm is introduced and the universal gravitation algorithm is improved to realize the micro-grid optimization solution of the battery and transferable load.”
MethodModel improvement and scheduling optimization for multi-vehicle charging planning in IoV
Qian, J.-H., Zhao, Y., & Huang, W. (2023) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2023.128826 · Cited by 3
Deep Reinforcement Learning-based Electric Vehicle Charging Optimization with Renewable Generations
Duan, Y., Liu, Z., Wang, X., Wang, X., Sun, Q., & Zhao, Z. (2023) · IEEE International Electrical and Energy Conference (CIEEC) · doi:10.1109/cieec58067.2023.10166611 · Cited by 3
Zamzam, T., Shaban, K. B., & Massoud, A. M. (2023) · Sensors · doi:10.3390/s23167216 · Cited by 3
Matare, T. N., & Folly, K. A. (2023) · doi:10.1109/gtd49768.2023.00073 · Cited by 3
Peng, Z., Yang, Q., Li, D., Zhang, F., & Song, P. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113450 · Cited by 5
Distributed V2G Dispatching via LSTM Network within Cloud-Edge Collaboration Framework
Shang, Y., Li, Z., Shao, Z., & Jian, L. (2021) · IEEE/IAS Industrial and Commercial Power System Asia (I&CPS Asia) · doi:10.1109/icpsasia52756.2021.9621448 · Cited by 3
“Because of the stochastic features of PEV charging and RES output, two methods, i.e. mathematical optimization and artificial intelligence (AI), are usually utilized for V2G dispatching [6].”
MethodPlug-in electric vehicles demand modeling in smart grids
Jahangir, H., & Konstantinou, C. (2021) · ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS) · doi:10.1145/3450267.3451998 · Cited by 3
“Most of the existing studies develop a Monte Carlo (MC) based simulation method, i.e., a scenario-based approach which relies on generating a large number of samples from the search space to model the PEV travel behavior [4, 6].”
Analysis of EV Charging Coordination Efficiency in Presence of Cheating Customers
Keçeci, C., Ismail, M., & Serpedin, E. (2021) · IEEE Access · doi:10.1109/access.2021.3128399 · Cited by 3
“In [21], the charging coordination mechanism is formulated by a Markov decision process with unknown transition probability.”
BackgroundLi, Y., Shang, X., & Huang, R. H. (2022) · Recent Advances in Electrical & Electronic Engineering (Formerly Recent Patents on Electrical & Electronic Engineering) · doi:10.2174/2352096515666220513104656 · Cited by 3
Menos-Aikateriniadis, C., Sykiotis, S., & Georgilakis, P. S. (2023) · IET conference proceedings. · doi:10.1049/icp.2023.0007 · Cited by 3
“Both works [11, 12] use a Deep Q-learning (DQN) algorithm to optimize EV charging scheduling for residential DR, considering past electricity prices.”
MethodBang, H., & Malikopoulos, A. A. (2021) · arXiv (Cornell University) · Cited by 3
Deep Reinforcement Learning and Energy Price Prediction
Yadav, D., Mekhilef, S., Singh, B., & Rawa, M. J. H. (2022) · Wiley eBooks · doi:10.1002/9781119893998.ch11 · Cited by 3
An Intelligent Reinforcement‐Learning‐Based Load Shedding to Prevent Voltage Instability
Aghdam, P. A., Khoshkhoo, H., & Akbari, A. (2025) · doi:10.1002/9781394334599.ch16 · Cited by 3
Shojaeighadikolaei, A., Talata, Z., & Hashemi, M. (2025) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2025.3580678 · Cited by 2
“Approaches, such as DQN [10], [11]; Bayesian neural networks [12]; A2C [13], [14]; DDPG [15]; and SAC [16] have been applied, with LSTM networks used to handle the temporal uncertainty in renewable energy sources and prices [10], [11], [15].”
Mohsenimanesh, A. A., McNevin, C., & Entchev, E. (2025) · World Electric Vehicle Journal · doi:10.3390/wevj16110603 · Cited by 4
“[11] This paper predicts electricity prices using a supervised learning approach to formulate the real-time EV charging and discharging scheduling problem.”
Li, H., Liu, Y., Yan, J., Gao, J., Yang, X., & Naili, M. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228849 · Cited by 2
“The emergence of these autonomous systems promises transformative capabilities in addressing complex tasks across various critical infrastructure domains [9], such as smart transportation [10], smart grid and energy systems [11], [12], and robotics and smart manufacturing [13]–[16], among others.”
Self-citationCaballé, J., Segovia, P., Ocampo‐Martínez, C., & Quijano, N. (2025) · Control Engineering Practice · doi:10.1016/j.conengprac.2025.106669 · Cited by 2
“…RL and DRL have gained traction due to their ability to learn effective policies from data without requiring fully specified system dynamics (Wan et al., 2019, Li et al., 2020, Abdullah et al., 2021, Wang et al., 2021, Zhang et al., 2021, Li et al., 2022b, Zhao et al., 2024, Xie et al.,…”
Gauchotte, R., Oulamara, A., Ghogho, M., & Oudani, M. (2026) · European Journal of Operational Research · doi:10.1016/j.ejor.2026.01.015 · Cited by 3
Wen, C., Ma, X.-Z., Jia, F., Yan, X.-H., Meng, Z., Song, Z.-H., & Luo, Z.-H. (2026) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2026.111517 · Cited by 2
He, C., & Li, J. (2026) · Energies · doi:10.3390/en19092238 · Cited by 2
Kwon, K.-B., & Zhu, H. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.437 · Cited by 2
Exploring Electric Vehicles Energy Flexibility in Buildings
Dias, D. V., Lopes, R. A., & Martins, J. F. (2022) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-07520-9_13 · Cited by 2
Metadata
- Authors
- Wan, Z., Li, H., He, H., & Prokhorov, D.
- Affiliations
- University of Rhode Island (US)
- Shenyang Institute of Automation (CN)
- Toyota Motor North America Research & Development (United States) (US)
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2019-09
- Volume
- 10
- Issue
- 5
- Pages
- 5246-5257
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W2899639849
- Cited by
- 548
- References
- 46
- Citations by year
- 2026: 61 · 2025: 71 · 2024: 83 · 2023: 95 · 2022: 87 · 2021: 74 · 2020: 57 · 2019: 20
- FWCI
- 14.43
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- IEEE Transactions on Smart Grid (publishedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- SIA OpenIR (Chinese Academy of Sciences) (submittedVersion)
- Funders
- Office of Naval Research (N00014-18-1-2396)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Power System Optimization
- Keywords
- electric vehicle charging scheduling, real-time pricing, demand response, deep reinforcement learning, model-free learning, unknown transition probabilities, Markov decision process, Q-network, electricity price forecasting, smart grid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
