Cited by 548Electric Vehicles and InfrastructureSmart Grid Energy ManagementElectric Power System Optimization

The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.

Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning

Open access

Wan, Zhiqiang, Li, Hepeng, He, Haibo, Prokhorov, Danil·2019·IEEE Transactions on Smart Grid

DOI 10.1109/tsg.2018.2879572

Open this paper in CoChat — free account

Abstract

Driven by the recent advances in electric vehicle (EV) technologies, EVs have become important for smart grid economy. When EVs participate in demand response program which has real-time pricing signals, the charging cost can be greatly reduced by taking full advantage of these pricing signals. However, it is challenging to determine an optimal charging strategy due to the existence of randomness in traffic conditions, user's commuting behavior, and the pricing process of the utility. Conventional model-based approaches require a model of forecast on the uncertainty and optimization for the scheduling process. In this paper, we formulate this scheduling problem as a Markov Decision Process (MDP) with unknown transition probability. A model-free approach based on deep reinforcement learning is proposed to determine the optimal strategy for this problem. The proposed approach can adaptively learn the transition probability and does not require any system model information. The architecture of the proposed approach contains two networks: a representation network to extract discriminative features from the electricity prices and a Q network to approximate the optimal action-value function. Numerous experimental results demonstrate the effectiveness of the proposed approach.

Integrity signals

Registry

Crossref · 10.1109/tsg.2018.2879572

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
LSTM: A Search Space Odyssey
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Critical Review of Recurrent Neural Networks for Sequence LearningPreprint
Lipton, Z. C., Berkowitz, J. A., & Elkan, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00019 · Cited by 3,159
Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network
Kong, W., Dong, Z. Y., Jia, Y., Hill, D. J., Xu, Y. H., & Zhang, Y. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2753802 · Cited by 2,806
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259
Efficient BackProp
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333

Show all 46 references

Research works printed but not linked: 5

bellman, 1958, Dynamic Programming
2018, Grid-Connected Renewable Energy Systems
goodfellow, 2016, Deep Learning, 1
2017, The Egallon How Much Cheaper is it to Drive on Electricity?
2017, California Independent System Operator

Sorted by citations

Items 401–500 of 547

DeepDLP: Deep Reinforcement Learning based Framework for Dynamic Liner Trade Pricing
Li, X., Hu, Y., Bai, Y., Gao, X., & Chen, G. (2023) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1109/imcom56909.2023.10035599 · Cited by 2
“[22] Nevertheless, they design fully-connected network in the Q-value training process.”
Background
Real-Time Electric Vehicle Intelligent Charging Scheduling Strategy in Real Traffic Scenarios
Yue, Y., Pan, G., & Zhu, J. (2023) · IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM) · doi:10.1109/wowmom57956.2023.00042 · Cited by 2
Research on Multi-objective Charging Scheduling Strategy for Electric Vehicles: A Deep Reinforcement Learning Approach
Peng, Z., Yang, Q., Li, D., & Zhang, F. (2023) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac59482.2023.10401584 · Cited by 2
“Wan et al. [14] improved the deep Q-leaning algorithm and proposed a new architecture to solve the EV demand response problem.”
Method
Real-time Control Strategy for Wind-solar-storage Industrial Park Based on Variational Modal Decomposition
Wang, Q., Chen, Y., Cai, X., Meng, Z., & Song, D. (2023) · International Conference on Intelligent Power and Systems (ICIPS) · doi:10.1109/icips59254.2023.10405061 · Cited by 2
“Literature [7,8] used reinforcement learning algorithms for scheduling microgrids and electric vehicles respectively based on price signals.”
Method
Optimal differentiated threshold characterization for multi-task stochastic deadline scheduling with queuing
Jin, J., & Xu, Y. (2024) · Automatica · doi:10.1016/j.automatica.2024.111545 · Cited by 2
Improved Prediction of Electric Vehicle User Behavior with Machine Learning based Analysis
Prasad, K. K., Mali, A., & Agarwal, V. (2023) · IEEE International Conference on Smart Technologies for Power, Energy and Control (STPEC) · doi:10.1109/stpec59253.2023.10431211 · Cited by 2
“Machine Learning (ML) techniques, including deep reinforcement learning, can be applied to determine the optimal strategy for EV charge scheduling, taking into account real-time pricing and charging costs [2].”
Background
Out-of-Distribution-Aware Electric Vehicle Charging
Li, T., & Sun, C. (2024) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2024.3434750 · Cited by 2
“Classic learning-based scheduling algorithms [4], [5], [20] may encounter high variability issues due to the changes in environments.”
Dynamic Selection of Electric Vehicle Charging Stations Using Deep Reinforcement Learning
Bachiri, K., Yahyaouy, A., Malek, M., Rogovschi, N., Bennani, Y., & Gualous, H. (2024) · International Conference on Intelligent Systems and Computer Vision (ISCV) · doi:10.1109/iscv60512.2024.10620096 · Cited by 2
“Mnih et al [16] by leveraging deep learning, DRL algorithms can derive optimal policies from high-dimensional data without prior knowledge of the system’s dynamics, significantly enhancing decision-making in variable conditions [17]–[19], [20]–[22].”
Background
Community-driven Smart EV charging With Multi-Agent Deep Reinforcement Learning
Sykiotis, S., Athanasoulias, S., Temenos, N., Rallis, I., Doulamis, A. D., & Doulamis, N. D. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650988 · Cited by 2
“Additionally, most works depend on simulated data instead of real-world measurements, which can lead to inaccurate representation of household consumption patterns, given that human activity can often cause outlier values not captured in the simulation [5]–[8].”
Background
Intelligent MPPT Controller for PV Fed Grid-Tied EV Charging with Power Quality Improvement using Neural Network Controller
Kumar, V. R., Sameer, T. C., & Prakash, S. (2024) · International Conference on Circuit, Power and Computing Technologies · doi:10.1109/iccpct61902.2024.10672823 · Cited by 2
Deep Deterministic Policy Gradient Data-Driven Approach for EV Charging and Discharging Scheduling Problem
Alabbas, S., & Kwasinski, A. (2024) · International Telecommunications Energy Conference (INTELEC) · doi:10.1109/intelec60315.2024.10678971 · Cited by 2
“A deep reinforcement Q-network (DQN) algorithm, which incorporates deep neural networks with Q-learning algorithm, is utilized in [4] to propose an optimal solution to the scheduling of EV charging and discharging processes to minimize the charging cost while considering the randomness in…”
Method
Optimal Coordination of Electric Vehicles for Grid Services using Deep Reinforcement Learning
Das, A., & Wu, D. (2024) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm51994.2024.10688956 · Cited by 2
“In [11], the authors propose a deep Q network (DQN)- based RL method to schedule EV’s charging/discharging cost-effectively while fulﬁlling user’s driving demand.”
Smart Grid-Driven Intelligent EV Charging Scheduling Strategy Based on Deep Q-Networks
Zhan, K., Huang, Y., Su, J., Xu, M., Nong, L., & Zeng, H. (2024) · IEEE International Conference on Information, Communication and Networks (ICICN) · doi:10.1109/icicn62625.2024.10761617 · Cited by 2
“ The penalty that denotes the amount discharged at the current time is greater than the battery's residual capacity, of at the departure time ( 0 t l = ) when factor  , called the mileage anxiety term[12], is used to penalize EVs that are not fully charged at departure.”
Transient Stability Data Driven Special Protection Scheme Using Deep Reinforcement Learning
Jung, Y., Kang, S., Kim, K., Woo, H., Choi, S., Jang, G., Song, S., & Yoon, M. (2024) · International Conference on Renewable Energy Research and Applications (ICRERA) · doi:10.1109/icrera62673.2024.10815587 · Cited by 2
Graph RL-Based Route Planning Method for EVs with Synchronous Charge Station Selection
Ma, Y., Chen, W., Ding, F., Fan, Y., & Peng, J. (2024) · CAA International Conference on Vehicular Control and Intelligence (CVCI) · doi:10.1109/cvci63518.2024.10830115 · Cited by 2
Time-of-Use Pricing Aware Battery Swapping Station Charging Scheduling via Deep Reinforcement Learning
Chen, H., Gu, C., Wan, D., Yu, B., & Xiang, C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1542-1_8 · Cited by 2
Smart Energy Management: From Conventional Optimization to Generative AI Techniques
Mongaillard, T., Lasaulce, S., & Varma, V. S. (2025) · Lecture notes in intelligent transportation and infrastructure · doi:10.1007/978-3-031-72959-1_15 · Cited by 2
Optimizing Plug-in Electric Vehicle Charging with Dyna-Q Reinforcement Learning
Arvind, S. K., Rayan, S. R., Satheesh, R., & Rani, G. S. A. (2025) · International Conference on Power, Control and Computing Technologies (ICPC2T) · doi:10.1109/icpc2t63847.2025.10958689 · Cited by 2
“Batteries have grown in popularity as the world’s focus has shifted to renewable energy [2].”
Dynamic Machine Learning-Based Simulation for Preemptive Supply-Demand Balancing Amid EV Charging Growth in the Jamali Grid 2025–2060
Tampubolon, J. V., Dalimi, R., & Sudiarto, B. (2025) · World Electric Vehicle Journal · doi:10.3390/wevj16070408 · Cited by 2
A Deep Reinforcement Learning Method for Multi-objective Transmission Switching
Lin, D., Wang, J., Zhao, T., & Yue, M. (2025) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm52009.2025.11225721 · Cited by 2
“A Q-learning based DRL method is developed to determine the optimal charging strategy for electric vehicle charging scheduling [10].”
An Optimal V2G Charging and Discharging schedule for Electric Vehicles in Sweden’s Day-Ahead and FCR-D Markets
Malik, S. M., Jung, D., Sundström, C., & Hellgren, J. (2025) · IEEE PES Innovative Smart Grid Technologies Europe · doi:10.1109/isgteurope64741.2025.11305294 · Cited by 2
“A deep Q-learning (DQL) algorithm is proposed in [11] which optimizes the EV charging by minimizing the user cost and utilizes long short-term memory in extracting price signal features.”
Optimal EV Recharging Navigation Strategy for Long-Distance Travel using Multiple Objective Optimization
Zunnurain, I., & Sang, Y. (2021) · North American Power Symposium (NAPS) · doi:10.1109/naps52732.2021.9654743 · Cited by 2
Role of Energy Management Systems (EMS) in the Development of Eco Cities: A System Oriented Review Introducing an Agent Perspective
Benrabia, I., Moulik, B., & Söffker, D. (2023) · Springer eBooks · doi:10.1007/978-3-658-39438-7_24 · Cited by 2
Approaches for Sustainable Urban Mobility Futures
Gall, T., Vallet, F., Madrigal, L. M. R., Hörl, S., Abdin, A., Chouaki, T., & Puchinger, J. (2023) · Sustainable urban futures · doi:10.1007/978-3-031-45795-1_3 · Cited by 2
MACHINE LEARNING APPLICATIONS IN POWER SYSTEMS
Wang, X. (2020) · SMU Scholar (Southern Methodist University) · Cited by 2
Deriving Fast AC OPF Solutions via Proximal Policy Optimization for Secure and Economic Grid Operation.Preprint
Zhou, Y., Zhang, B., Xu, C., Lan, T., Diao, R., Shi, D., Wang, Z., & Lee, W. (2020) · arXiv (Cornell University) · Cited by 2
Double Deep Q-learning Based Real-Time Optimization Strategy for MicrogridsPreprint
Shuai, H., Ai, X., Fang, J., Yao, W., & Wen, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12545 · Cited by 2
“Realizing the advantages of the DRL approach, some researchers made the effort to use the DRL to solve the optimal operation/control problems in power industry in the past few years [31]–[35], including the electrical vehicle (EV) charging/discharging scheduling [32], building energy management [33], [34], and power s…”
Method
Enhanced Resilience in Battery Charging through Co-Simulation with Reinforcement LearningPreprint
Seyedi, M., Sheida, K., Siner, S., & Ferdowsi, F. (2024) · doi:10.36227/techrxiv.170846720.02245839/v1 · Cited by 2
Research on Control Method of Electric Vehicle in Residential Area Based on Soft Actor-Critic
Yu, H., Dou, X., Hu, W., & Zhang, K. (2023) · Asia Energy and Electrical Engineering Symposium (AEEES) · doi:10.1109/aeees56888.2023.10114146 · Cited by 1
“[12] proposes a DRL real-time scheduling method that considers the randomness of EV load user behavior and the volatility of real-time electricity price.”
Method
A Review of Artificial Intelligence-Based Prognostic and Health Management Systems for Lithium - In Batteries in Electric Vehicles
Oladimeji, J., & Ogunniyi, O. (2023) · International Journal of Science and Research (IJSR) · doi:10.21275/sr231121155106 · Cited by 1
Multi-agent Deep Reinforcement Learning Based Approach for Power Command Response Combining Fast and Normal Charging Piles
Li, H., Li, G., & Li, S. Y. (2024) · Power System and Green Energy Conference (PSGEC) · doi:10.1109/psgec62376.2024.10721196 · Cited by 1
“In [5], a locally charging control strategy is proposed based on deep Q-learning considering the uncertainty of the renewable generation.”
Self-citation
Optimizing Electric Vehicle Charging Schedules: Minimizing Charging Time and Travel Distance
Kanchana, W., Singh, J. G., Ongsakul, W., & Maya, P. (2024) · International Conference and Utility Exhibition on Energy, Environment and Climate Change (ICUE) · doi:10.1109/icue63019.2024.10795558 · Cited by 1
Reinforcement learning-based dynamic pricing models for electric vehicle charging stations
Chen, Q., & Folly, K. A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-323-95201-9.00012-3 · Cited by 1
Integration of Electric Vehicles as Non-Wires Alternatives: A Virtual Power Line Approach
Lee, Y. H., Rho, S., Lee, J., & Won, D.-J. (2025) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2025.3573858 · Cited by 1
“Wan et al. [7] proposed model-free EV optimal scheduling through Q -learning.”
A novel multi-objective optimization model of electric vehicles’ path planning and charging scheduling based on users’ comprehensive satisfaction
Shi, J., Li, Z., Wei, Y., Huang, X., & Gao, S. (2025) · Proceedings of the Institution of Mechanical Engineers Part I Journal of Systems and Control Engineering · doi:10.1177/09596518251346046 · Cited by 1
“19 It is of great superiority in cost-efficient scheduling and reducing travel distance.”
Smart Pricing for Smart Charging: A Deep Reinforcement Learning Framework for Residential EV Infrastructure
Pergamalis, C., Tsampasis, E. G., Gkonis, P. K., & Elias, C. N. (2026) · Future Internet · doi:10.3390/fi18050241 · Cited by 1
A Mamba State-Space Sequence Model for AI-Driven Dynamic Aggregation and Predictive Control of Electric Vehicle Clusters in Vehicle-to-Grid Energy Management
Tang, J., Zhou, X., & Yan, Q. (2026) · Electronics · doi:10.3390/electronics15112380 · Cited by 1
Optimizing electric vehicle charging routes with deep reinforcement learning considering driver preferences
Mishra, S., Choubey, A., Dhankhar, H., Devarasetty, S. V., & Misra, R. (2026) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2026.102353 · Cited by 1
An intelligent reinforcement learning framework for sequential decision-making
Kuyu, Y. Ç. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115800 · Cited by 1
A Multi-Objective Deep Reinforcement Learning Approach for EV Charging Optimization Considering Transformer Lifespan
Wang, J., & Du, C. (2026) · Electronics · doi:10.3390/electronics15143100 · Cited by 1
Advanced supervisory and distributed control frameworks in modern industrial and energy applications
Eltwab, A. A. A., & Abdelsalam, M. M. (2026) · Journal of Electrical Systems and Information Technology · doi:10.1186/s43067-026-00381-7 · Cited by 1
AI-Enabled Power Electronics, Electrical Machines, and Energy Management Systems for High-Efficiency Sustainable Energy Conversion
Gros, I.-C., Popa, D.-C., Valasutean, E., Cotor, S.-I., & Szabó, L. (2026) · Electronics · doi:10.3390/electronics15163672 · Cited by 1
Auditing Single-Agent Reinforcement Learning for EV Charging Assignment: A Protocol-Amended Comparison of Trained, Untrained, and Heuristic Policies
Moumni, N.-E., Alaoui, R., & Kiouach, D. (2026) · Algorithms · doi:10.3390/a19100838 · Cited by 1
Application of Reinforcement Learning to Power System Operation
Takayama, S. (2021) · IEEJ Transactions on Power and Energy · doi:10.1541/ieejpes.141.608 · Cited by 1
Evolutionary-Based Neural Architecture Search for an Efficient CAES and PV Farm Joint Operation Strategy Using Deep Reinforcement Learning
Dolatabadi, A., Abdeltawab, H. H., & Mohamed, Y. A. I. (2023) · doi:10.1109/gtd49768.2023.00085 · Cited by 1
Introduction: A Brief History of Deep Learning and Its Applications in Power Systems
Li, F. F., & Du, Y. (2023) · Power electronics and power systems · doi:10.1007/978-3-031-45357-1_1 · Cited by 2
Artificial Intelligence of Things for industrial Visual sensing systems in HP's factories
Le, D. V., Zhou, S., Yang, J. Q., Chen, J., Ho, D., & Tan, R. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13812-6.00003-8 · Cited by 1
Multi-Agent RL Framework for EV Charging Scheduling Driven by Energy Costs and User Preferences
Tsaknakis, C. D., Korkas, C. D., Michailidis, I., & Kosmatopoulos, E. B. (2023) · International Conference on Electrical, Communication, and Computer Engineering (ICECCE) · doi:10.1109/icecce61019.2023.10442662 · Cited by 1
“Furthermore, [15], [16] used DQN and SARSA algorithms, two Q-Learning-based techniques, to address the optimal scheduling problem.”
Method
Multi-agent Deep Reinforcement Learning Based Multi-objective Charging Control for Electric Vehicle Charging Station
Gong, J., Fu, W., Kang, Y., Qin, J., & Xiao, F. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-3328-6_23 · Cited by 1
Active Synchronous Rectifier based Parallel Interleaved Converter for EV Battery Charging
Karunakaran, M., Sridevi, R., Suresh, V., & Prabu, V. K. D. (2024) · International Conference on Circuit, Power and Computing Technologies · doi:10.1109/iccpct61902.2024.10672840 · Cited by 1
Transactive Electric Vehicle Agent: A Deep Reinforcement Learning Approach
Sharma, S., Battula, S., & Singh, S. N. (2024) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm51994.2024.10688919 · Cited by 1
“However, in [10], [11], charge/discharge schedules are determined using forecasted price information, and in [12], time-of-use prices are used for the same.”
Background
Reinforcement Learning-Based Approach for EMT Automation of Large-Scale PV Plants
Xia, Q., Kurte, K. R., & Debnath, S. (2024) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm51994.2024.10689173 · Cited by 1
EV Scheduling Model in DR Based on DRL with Prediction Accuracy as the Optimization Objective
Yu, S., Xu, B., Ge, X., Wang, F., & Chen, Q. (2024) · IEEE/IAS Industrial and Commercial Power System Asia (I&CPS Asia) · doi:10.1109/icpsasia61913.2024.10761549 · Cited by 1
“The objective of reinforcement learning algorithms is to find a policy that maximizes the reward [15] [16].”
A Review of Electric Vehicle Charging Technologies and Beyond
Wei, H., Tang, Y., Jiang, K., Chen, J., Liu, Q., & Galea, M. (2024) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin58382.2024.10774378 · Cited by 1
V2PCP: Toward Online Booking Mechanism for Private Charging Piles
Lu, X., Sun, J., Lou, J., Ji, Y., Wu, C., Zhao, W., Xue, G., Luo, Y., Cheng, F., & Li, J. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3516828 · Cited by 1
“For example, reinforcement learning-based strategies require extensive data and long training periods [18], [19], [20], [21], [22], [23], [24], [25], [26].”
Optimizing Charging Control for Medium and Heavy-Duty Electric Vehicles Using Deep Reinforcement Learning
Barba, L., Ahmed, S., Alamaniotis, M., & Gatsis, N. (2025) · IEEE Texas Power and Energy Conference (TPEC) · doi:10.1109/tpec63981.2025.10906943 · Cited by 1
Effect of Maintenance and Failure on Electrical Vehicles Charging Scheduling at an Airport
Godec, P., & McKeever, S. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-87345-4_8 · Cited by 1
Improved spatiotemporal decomposition algorithm for real-time decentralized optimal scheduling of MG with energy storage
Luo, Y., & Zhu, J. (2025) · Journal of Energy Storage · doi:10.1016/j.est.2025.117121 · Cited by 1
Machine Learning-Based Strategy for the Regulated Charging of Plug-In Electric Vehicles
Alkawaz, A. N., Zaidan, A. N., & Yaakub, M. F. (2025) · Al-Iraqia Journal of Scientific Engineering Research · doi:10.58564/ijser.4.2.2025.318 · Cited by 1
Coordinating Distributed Energy Resources with Nodal Pricing in Distribution Networks: a Game-Theoretic Approach
Brock, E., Li, J., Lavaei, J., & Sojoudi, S. (2025) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc57313.2025.11312874 · Cited by 1
“The linear term cancels trivially and the second step applies Lemma (1).”
Influential
Reward Function Identification of GENCOs
Chen, Q., Guo, H., Zheng, K., & Wang, Y. (2021) · Springer eBooks · doi:10.1007/978-981-16-4975-2_13 · Cited by 1
Power Dispatching Strategy of Electric Vehicle Charging Station Based on Reinforcement Learning and Heuristic Priority
An, D., & Zhang, T. (2023) · Asia Energy and Electrical Engineering Symposium (AEEES) · doi:10.1109/aeees56888.2023.10114166 · Cited by 1
A Hybrid Planning System for Smart Charging of Electric Fleets
Garg, K., Narayanan, A., Misra, P., Vasan, A., Bandhu, V., & Das, D. (2022) · International Conference on AI-ML Systems (AIMLSystems) · doi:10.1145/3564121.3564125 · Cited by 1
“Since the planning requires quick charging decisions that need to be taken as soon as EVs arrive and depart, most solutions use some form of heuristic or RL [6, 29].”
Background
Energy System Analysis and Modeling
Wu, D., Fan, J., Lu, S., Wang, J., Zhu, Y., & Hu, H. (2023) · Engineering Applications of Computational Methods · doi:10.1007/978-981-99-4550-4_2 · Cited by 1
DEEP LEARNING FOR DATA-DRIVEN DECISION-MAKING
Wan, Z. (2020) · Digital Commons - URI (University of Rhode Island) · doi:10.23860/wan-zhiqiang-2020 · Cited by 1
Self-citation
Energy Scheduling in IoE-Enabled Smart Grids Using Probabilistic Delayed Double Deep Q-Learning (P3DQL) AlgorithmPreprint
Rouzbahani, H. M., Karimipour, H., & Lei, L. (2022) · doi:10.36227/techrxiv.19282859 · Cited by 1
2019 Index IEEE Transactions on Smart Grid Vol. 10
(2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2951931 · Cited by 1
Two-Stage Data-Driven Optimal Energy Management and Dynamic Real-Time Operation in Networked Microgrid Based Deep Reinforcement Learning ApproachPreprint
Sepehrzad, R., Ghafourian, J., Hedayatnia, A., Al-Durrad, A., & Anvari‐Moghaddam, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4742334 · Cited by 1
Reinforcement Learning-Based Energy Management for Hybrid Engineering VehiclesPreprint
Liu, H., Yao, Y., Li, T., Du, M., Wang, X., Li, H., & Li, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4778979 · Cited by 1
Self-citation
Day-ahead Scheduling Method for Electricity Markets Using Neural Networks
WATANABE, F., Kawaguchi, T., Ishizaki, T., Takenaka, H., Nakajima, T. Y., & Imura, J. (2020) · Transactions of the Society of Instrument and Control Engineers · doi:10.9746/sicetr.56.57
A Novel Model for VPP to Participate in Capacity Market Auction
Kuang, Y., Wang, X., Li, N., Zhao, H., & Qian, T. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1585/1/012032
“Case 2: except for conventional generation capacity and external capacity, there are also RES, EVs and DRRs to participate in capacity market auction.”
BackgroundInfluential
MULTI-VIEW ROBUST REPRESENTATION LEARNING
Li, L. (2021) · doi:10.23860/li-lusi-2021
Efficient Representation for Electric Vehicle Charging Station Operations using Reinforcement Learning
Kwon, K.-B., & Zhu, H. (2021) · Journal of the Association for Information Systems · doi:10.26153/tsw/16960
Energy Scheduling in IoE-Enabled Smart Grids Using Probabilistic Delayed Double Deep Q-Learning (P3DQL) AlgorithmPreprint
Rouzbahani, H. M., Karimipour, H., & Lei, L. (2022) · doi:10.36227/techrxiv.19282859.v1
Abstracts from the 9th DACH+ Conference on Energy Informatics
(2020) · Energy Informatics · doi:10.1186/s42162-020-00113-9
Autonomous PEV Charging Scheduling Using Deep-Q Network and Dyna-Q Reinforcement Learning
Wang, F. (2021) · doi:10.32920/ryerson.14661024
Autonomous PEV Charging Scheduling Using Deep-Q Network and Dyna-Q Reinforcement Learning
Wang, F. (2021) · doi:10.32920/ryerson.14661024.v1
An Advanced Simulation Tool for Electricity Spot Market Using Agent-Based ModelPreprint
Yun, J., Yang, M. Y., Liu, N., Zhang, C., & Feng, C. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4102933
Data-driven operation and control for power systems with high-level renewable energy resources
Yan, Z. (2021) · doi:10.32657/10356/161731
A Multi‐agent Reinforcement Learning for Home Energy Management
Lai, C. S., Lai, L. L., & Lai, Q. H. (2022) · doi:10.1002/9781119790402.ch9
Probabilistic Revenue Analysis for Electric Vehicle Charging
Santoyo, C., & Coogan, S. (2022) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta49430.2022.9966167
“, bidding pricing models [1], [2], scheduling models [3], [4], etc.”
Background
Extensions to PEVs Charging/Discharging Scheduling
Qin, J., Wan, Y., Li, F., Kang, Y., & Fu, W. (2023) · Studies in systems, decision and control · doi:10.1007/978-981-19-8594-2_5
Knowledge Assisted Deep Reinforcement Learning for Electric Vehicle Charging Control
Zai, R., Guo, Y., Liu, Q., Sun, H., Wu, Q., & Li, X. (2022) · IEEE Conference on Energy Internet and Energy System Integration (EI2) · doi:10.1109/ei256261.2022.10116271
“In [10], [11] and [12], the authors all make balance between users’ charging requirements and costs.”
BackgroundMethod
Fuzzy Rule-Based Coordinated EV Charging Management
Salah, A., Mohareb, O. A., Brosi, F., & Reuss, H.-C. (2024) · Proceedings · doi:10.1007/978-3-658-45018-2_10
Proteus-Based Simulation and Analysis of Full Bridge Hf Inverter and Bidirectional Dc-Dc Converter for Ev Charging SystemsPreprint
Shyamalagowri, M., & Jothippriya, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4883825
Energy Scheduling Model for Photovoltaic 5G Base Station Based on Deep Constrained Deterministic Policy Gradient
Liu, M., Ning, Y., Liu, Z., Guo, X., & Wang, X. (2024) · Chinese Control Conference (CCC) · doi:10.23919/ccc63176.2024.10661960
“The common settings for the other learning parameters are adopted, which have been mostly used in other methods [12] .”
Method
A Systematic Survey on the Application of Artificial Intelligence (ai) Baseline Networks on Grid Computing Techniques - Challenges, Novelty and Prospects
Gonten, F. N., Jakawa, J. N., Emmanuel, D. U., Pandok, D. A., Maikano, P. C., & ICT Unit, P. S. P. (2024) · International Journal of Soft Computing and Engineering · doi:10.35940/ijsce.d3643.14040924
Position Papers of the 19th Conference on Computer Science and Intelligence Systems
(2024) · Annals of Computer Science and Information Systems · doi:10.15439/978-83-969601-9-1
Enhancing Electric Vehicle Charging Infrastructure: A Framework for Efficient Charging Point ManagementPreprint
Prajeesh, C. B., Krishna Priya, R., Pillai, A. S., Khwaja, A. S., & Anpalagan, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5051579
Collaborative Dispatching Method for Wind-solar Generation and Electric Vehicle Considering Passenger Convenience
Chen, W., Wu, X., Tian, H., Ye, Y., Zhao, E., & Guo, Y. (2024) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2024-3419
A Novel Hybrid Price Mechanism for Regulating the Coupled Power Distribution Network and Transportation Network Based on Feedback Control Method
Li, B., & Li, J. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-981-96-1379-3_21
Optimization strategy for large-scale electric vehicle charging and discharging based on deep reinforcement learning
Cheng, Z., Xia, Y., & Zhang, J. (2025) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2993/1/012064
“For example, in [5], an optimal charging and discharging strategy was proposed, while in [6], charging pricing was integrated into the optimization process to increase station revenue.”
Bayesian Network Based Evidential Reasoning for EV Charging Navigation Considering Dynamic Multiple Attributes
Zheng, J., Cao, Z., Li, Z., & Wu, Q. (2025) · IET Renewable Power Generation · doi:10.1049/rpg2.70083
Optimal Scheduling of Electric Vehicle Integrated Microgrids Basing a Long Short Term Memory and Deep Deterministic Policy Gradient ApproachPreprint
Xie, H., Ren, L., Liu, F., Mehran, K., & Jiao, X. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5396316
An Optimal Two-Stage Scheduling Strategy for Electric Vehicle Charging Stations Based on Deep Reinforcement Learning
Huang, X., Hu, C., & Zhang, J. (2025) · IEEE Conference on Industrial Electronics and Applications (ICIEA) · doi:10.1109/iciea65512.2025.11148471
“[9] proposes a model-free real-time EVs charging scheduling method based on reinforcement learning, which can adaptively learn transition probabilities.”
An Intelligent Control Framework for High-Power EV Fast Charging via Contrastive Learning and Manifold-Constrained Optimization
Tian, H., Yan, T., Dai, G., Wang, M., & Zhao, X. (2025) · World Electric Vehicle Journal · doi:10.3390/wevj16100562
Optimizing Urban Charging Infrastructure for Electric Vehicles: A Data-Driven Approach to Boost Adoption and Reduce Costs
Hu, H., & Zhao, D. F. (2025) · ASCE International Conference on Computing in Civil Engineering (i3CE) · doi:10.1061/9780784486115.046
Deep Reinforcement Learning-based Research on Electric Vehicle Charging and Discharging Scheduling
Liu, X., Qin, H., & Wang, L. (2025) · International Conference on Electrical Engineering and Mechatronics Technology (ICEEMT) · doi:10.1109/iceemt66565.2025.11281115
“The DQN (Deep Q Network) algorithm was used to introduce the LSTM (Long Short-Term Memory) neural network [7], so as to realize the function of extracting the characteristics of price signals.”
Balancing Airport Grid Load: The Role of Smart EV Charging, Solar, and Batteries
Godec, P., & McKeever, S. (2025) · Winter Simulation Conference · doi:10.1109/wsc68292.2025.11338899
“Deep reinforcement learning (DRL) further enhances this by using neural networks, overcoming high-dimensional state space limitations (Jin and Xu 2021; Wan et al. 2019).”
Deep Learning based Estimation of Charging Time in Solar-Powered Electric Vehicle Charging Stations
Chakraborty, E., Chaurasia, A., Kundu, B., Kundu, S., Mukhopadhyay, M., & Datta, S. (2025) · International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN) · doi:10.1109/icrcicn68210.2025.11364929
“DRL based control frameworks such as those presented by Wan et al. [7] and Zhang et al. [8] have shown strong capability in balancing user demand with grid constraints through model-free decision optimization.”

Authors: Wan, Z., Li, H., He, H., & Prokhorov, D.
Affiliations: University of Rhode Island (US)
Shenyang Institute of Automation (CN)
Toyota Motor North America Research & Development (United States) (US)
ORCID: Wan, Zhiqiang
Li, Hepeng
He, Haibo
Prokhorov, Danil
Venue: IEEE Transactions on Smart Grid
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019-09
Volume: 10
Issue: 5
Pages: 5246-5257
ISSN: 1949-3053 (ISSN-L)
1949-3061
DOI: 10.1109/tsg.2018.2879572
OpenAlex: W2899639849
Cited by: 548
References: 46
Citations by year: 2026: 61 · 2025: 71 · 2024: 83 · 2023: 95 · 2022: 87 · 2021: 74 · 2020: 57 · 2019: 20
FWCI: 14.43
Type: journal article
Language: en
Open access: green
digitalcommons.uri.edu
Licence: cc-by
ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Transactions on Smart Grid (publishedVersion)
Journal of Media Literacy Education (submittedVersion, cc-by)
SIA OpenIR (Chinese Academy of Sciences) (submittedVersion)
Funders: Office of Naval Research (N00014-18-1-2396)
Topics: Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Power System Optimization
Keywords: electric vehicle charging scheduling, real-time pricing, demand response, deep reinforcement learning, model-free learning, unknown transition probabilities, Markov decision process, Q-network, electricity price forecasting, smart grid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks