Cited by 548Electric Vehicles and InfrastructureSmart Grid Energy ManagementElectric Power System Optimization

The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.

Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning

Open access

Wan, Zhiqiang, Li, Hepeng, He, Haibo, Prokhorov, Danil·2019·IEEE Transactions on Smart Grid

DOI 10.1109/tsg.2018.2879572

Open this paper in CoChat — free account

Abstract

Driven by the recent advances in electric vehicle (EV) technologies, EVs have become important for smart grid economy. When EVs participate in demand response program which has real-time pricing signals, the charging cost can be greatly reduced by taking full advantage of these pricing signals. However, it is challenging to determine an optimal charging strategy due to the existence of randomness in traffic conditions, user's commuting behavior, and the pricing process of the utility. Conventional model-based approaches require a model of forecast on the uncertainty and optimization for the scheduling process. In this paper, we formulate this scheduling problem as a Markov Decision Process (MDP) with unknown transition probability. A model-free approach based on deep reinforcement learning is proposed to determine the optimal strategy for this problem. The proposed approach can adaptively learn the transition probability and does not require any system model information. The architecture of the proposed approach contains two networks: a representation network to extract discriminative features from the electricity prices and a Q network to approximate the optimal action-value function. Numerous experimental results demonstrate the effectiveness of the proposed approach.

Integrity signals

Registry

Crossref · 10.1109/tsg.2018.2879572

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
LSTM: A Search Space Odyssey
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Critical Review of Recurrent Neural Networks for Sequence LearningPreprint
Lipton, Z. C., Berkowitz, J. A., & Elkan, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00019 · Cited by 3,159
Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network
Kong, W., Dong, Z. Y., Jia, Y., Hill, D. J., Xu, Y. H., & Zhang, Y. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2753802 · Cited by 2,806
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259
Efficient BackProp
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333

Show all 46 references

Research works printed but not linked: 5

bellman, 1958, Dynamic Programming
2018, Grid-Connected Renewable Energy Systems
goodfellow, 2016, Deep Learning, 1
2017, The Egallon How Much Cheaper is it to Drive on Electricity?
2017, California Independent System Operator

Sorted by citations

Items 501–547 of 547

Peak Shaving and Solar Utilization for Sustainable Campus EV Charging Using Reinforcement Learning Approach
Abdullah, H. M., Gastli, A., Ben‐Brahim, L., & Islam, S. (2026) · Sustainability · doi:10.3390/su18062737
Adaptive congestion management for EV charging in synchrophasor enabled ADNs: A decentralised distributed approach with concordance framework
Gopinathan, N., & Shanmugam, P. K. (2026) · Energy Reports · doi:10.1016/j.egyr.2026.109238
Bi-level Online Framework for Dynamic Energy Coordination and Priority-Based Charging in EV Stations
Xiong, H., Huang, J., & Lu, H. (2025) · Asian Conference on Frontiers of Power and Energy (ACFPE) · doi:10.1109/acfpe67232.2025.11451102
“In [11], a multi-agent DRL algorithm controls energy flows within EV infrastructure, optimizing resource utilization at charging points, while [12] presents a model-free, real-time scheduling via DRL-driven charging coordination.”
Real-Time Balancing Control of Reconfigurable Battery Packs Using Reinforcement Learning
Irshayyid, A., Yang, W., & Chen, J. S. (2026) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2026.3681552
“For example, DRL has been applied to online optimization of building energy management systems [31], tra ﬃ c ﬂow control [29], EV energy management strategies [32], charging scheduling [33], and control of fuel cell-battery systems [34].”
Electric Vehicle Bidirectional Charging Control with Deep Reinforcement Learning
Gei, C., Scherer, J., & Radchenko, G. (2025) · International Conference on Electrical, Computer and Energy Technologies (ICECET) · doi:10.1109/icecet63943.2025.11472556
“We assume that the consumer is a price taker and the the drawn energy does not affect the price [7]–[13].”
Influential
Multiobjective Optimization of Electricity–Gas–Heat Integrated Energy Systems Using Deep Reinforcement Learning
Li, F., Liu, L. X., Li, D., & Pan, S. (2026) · Journal of Energy Engineering · doi:10.1061/jleed9.eyeng-6477
Electric Vehicle Charging Scheduling Optimization Model Based on Reinforcement Learning
Li, P., Wang, W., Fang, X., Yang, Y., Zhang, X., Li, S., Long, Y., & Hu, X. (2025) · International Conference on Artificial Intelligence, Electrical and Electronic Engineering (AI3E) · doi:10.1109/ai3e69313.2025.00019
“Reference [15] employs deep Q-learning and models charging as a Markov Decision Process to progressively learn optimal charging strategies.”
Sustainable EV adoption with clustering and predictive modelling for optimal charging infrastructure in the West Midlands and North East UK
Cavus, M., Wang, S., Deb, S., Sharma, A., Bell, M., & Dissanayake, D. A. (2026) · Scientific Reports · doi:10.1038/s41598-026-43106-6
“Traditional methods rely on static models and rule-based charging management, making them less adaptive to real-time demand variations 31 .”
Reactive–Active Power Coordination Control of Grid-Forming V2G Charging Stations for Distribution Network Voltage Regulation
Xiao, F., Li, H., & Zhang, K. (2026) · World Electric Vehicle Journal · doi:10.3390/wevj17050252
Self-citation
Factored deep reinforcement learning framework for electric vehicle charging scheduling
Ye, T., & Tian, Z. (2026) · Journal of Energy Storage · doi:10.1016/j.est.2026.124285
AI-Driven Optimisation of Electric Vehicle Battery Systems: A Comprehensive Review
Singh, A., Bhardwaj, D., Sachan, A. P. S., Singh, A., Rani, M., & Harchand, N. (2026) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom70271.2026.11525997
Cooperative Multi-Agent Reinforcement Learning with Graph Attention and Price Forecasting for Real-Time Charging Coordination of Battery Electric Bus Fleets with Vehicle-to-Grid CapabilityPreprint
Iqbal, M. A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6874544
Risk-Aware Energy Management Strategy of Electric Vehicle Charging Station Based on TransQR-DQNPreprint
Zhang, X., Lai, S., Zhang, X., Li, L., Li, X., & Chen, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6874496
Electric vehicle charging and discharging scheduling strategy considering differences in users' range anxiety
Wei, X., Yi, H., Yi, H. M., Pang, A., & Yang, W. (2026) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2026.102361
A Comprehensive Review of Consumer Models in Price-Based Demand Response and Their Applications to Electric Vehicles
Li, Q., Fan, S., Zhou, L., Wang, Z., & Qi, P. (2026) · Energies · doi:10.3390/en19122809
Explainable Transformer-Based Demand Forecasting and Hybrid Anomaly Detection for Electric Vehicle Charging Networks: A Large-Scale Study Using ACN-DataPreprint
Salehin, M. U. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6956671
Behavioral Reinforcement Learning for Decentralized EV Fleet Charging: Reliability–Cost–Coordination Trade-offs Under Split Control Authority
Pok, P., Woo, S., & Yeo, H. (2026) · ACM Sustainability Week · doi:10.1145/3765611.3815391
Dynamic potential graphs guided real-time coordination for large-scale EV charging stations
Yang, Y., Cui, Q., Wang, Y., & Wang, B. (2026) · Applied Energy · doi:10.1016/j.apenergy.2026.128219
A distributed approach for electric vehicle smart charging scheduling using blockchain
Mangini, A. M., Roccotelli, M., Volpe, G., Estil-les, M. D. C., & Fanti, M. P. (2026) · Applied Energy · doi:10.1016/j.apenergy.2026.128264
A hierarchical reinforcement learning-based vehicle-to-grid dispatch architecture for car parks: Integrating proximal policy optimisation and a large language model within a dual-agent framework
Cao, J., Xu, X., Yao, W., Xue, F., & Long, C. (2026) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2026.105158
Artificial intelligence-driven embedded sensing, digital twin, and modular electrified vehicle systems for intelligent sustainable mobility
Waheed, H., & Abro, G. E. M. (2026) · Discover Vehicles · doi:10.1007/s44465-026-00025-0 · Cited by 1
“…have proven most effective for continuous action spaces: deep deterministic policy gradient (DDPG) [59], twin-delayed DDPG (TD3) [60], soft actor-critic (SAC) [61], and proximal policy optimisation (PPO) [62] have all been applied to BESS dispatch and EV charging scheduling [50, 63–66].”
Interpretable Price-Clearing Coordination for Capacity-Constrained Electric-Vehicle ChargingPreprint
Elsayed, S., & Aziz, K. A. (2026) · Research Square · doi:10.21203/rs.3.rs-10107999/v1
Advanced Computational Methods for Renewable Energy Forecasting and Electric Vehicle Scheduling: A Literature Review
Harak, V., & Samadhiya, A. (2026) · International Conference on Electrical and Electronics for Sustainable Innovations (ICEESI) · doi:10.1109/iceesi68020.2026.11583675
“Wan et al. (2015) proposed a Q-learning based approach to EV charging scheduling, that can adjust sliding system parameters and the user’s taste in cooperation [34].”
Multi-agent Reinforcement Learning-Based Dynamic EV Charging Scheduler for Smart Grids with Renewable Energy Integration
Rishitha, M., Nohitha, M., Raj, R. D. A., Yanamala, R. M. R., & Pallakonda, A. (2026) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-026-11426-4
Analyzing and Mitigating Coordinated Load Redistribution and Topology Attacks on Smart Grids using Deep Learning
Dong, M., Chen, M., & Wang, Z. (2026) · Journal of Electrical Engineering and Technology · doi:10.1007/s42835-026-02844-3
“For instance, ref [17 ] analyzed the effect of stealthy topology attacks and recommended a detection methodology using state estimation residuals.”
Degradation-aware and vehicle-to-grid incentivized scheduling for energy storage system and electric vehicle coordination using deep reinforcement learning
Liu, C., Xu, Z., Huang, H., Zhou, J., Yuan, Y., & Ma, K. (2026) · Journal of Energy Storage · doi:10.1016/j.est.2026.123421
LLM Integrated Hybrid Framework for Multi-Objective Residential Energy Scheduling with EV and Battery Flexibility under Real-Time PricingPreprint
Kharkwal, V. S., & Kharkwal, V. S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7199676
Online EV Fast Charging Scheduling With Energy Storage System Using a Profit-Predictable MILP
Yang, C.-A., Lin, T.-C., Chen, S.-C., Chen, Y.-W., Kuo, J.-J., & Tsai, M.-J. (2026) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2026.3714449
“It is challenging to determine an optimal charging schedule due to inherent randomness in tra ﬃ c conditions, user commuting behaviors, and ﬂuctuating utility pricing [8].”
Influential
Systematic review of reward function design in reinforcement learning for EV charging and battery energy storage
Rehman, A. U., Lu, J., Du, B., Bai, F., Hossain, M. A., & Sanjari, M. J. (2026) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2026.117366
MASAC: A Multi-Agent Soft Actor-Critic Framework for Decentralized EV Charging Networks
Shen, L., Jin, J., Wang, Q., Hao, L., Huang, Q., & Xu, Y. (2026) · IEEE International Conference on Power Science and Technology (ICPST) · doi:10.1109/icpst69917.2026.11636350
“Electric vehicle (EV) charging coordination in distribution networks has become a critical operational challenge as rapid EV adoption leads to concentrated load [1] increases and imposes significant stress on existing distribution infrastructure [2].”
Improving grid scale battery storage operation through thermal and economic optimization in hot dry climates
Guanghua, L., Cheng, L., & Shah, S. H. H. (2026) · Journal of Energy Storage · doi:10.1016/j.est.2026.124300
Explainable AI for Electric Vehicle Charging Optimization: Challenges, Opportunities, and Future Directions: A Review
Rahul, R., & Sasikala, K. (2026) · International Conference on Circuit, Power and Computing Technologies · doi:10.1109/iccpct70290.2026.11654688
Degradation‐aware energy management for photovoltaic‐storage‐charging microgrids under extreme operating environments: A comprehensive review
Liu, Q., Wang, J., Duan, J., Liu, Y., & Chen, Y. (2026) · Energy Conversion and Economics · doi:10.1049/enc2.70046
ChargeUp: Trust-Aware EV Charging Scheduling via Fuzzy-Stackelberg Game Theory and Credibility-Shaped Reinforcement Learning
Sasikumar, J., Raj, K. S. B., Sreerajan, M., Pandi, V. R., Radhu Krishna, R., & Sathyan, S. (2026) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2026.3732603
“Wan et al. [22] showed model-free scheduling, which can adjust to non-stationary demand without having an explicit model of the environment.”
Physics-Informed Reinforcement Learning for Real-Time Energy Management of Heavy-Duty Electric Vehicle Charging StationsPreprint
Ashkezari, L. S., Yorke‐Smith, N., Bauer, P., & Mouli, G. R. C. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7450619
AI-Driven Sustainable EV Infrastructure Planning Using Geopolymer Concrete: A Review Framework for Smart Cities
Vikas, G., Jalaluddin, M., Poloju, K. K., Rizwan, M., Latheef, M. A., Uddin, M. Z., & Rahman, A. (2026) · MEC International Conference on Big Data and Smart City (ICBDSC) · doi:10.1109/icadaai64945.2026.11679184
Artificial neural network modeling, particle swarm optimization and Deep Q-Network energy management of a novel solar tetra-generation system with hot water storage tank
Sanaye, S., Borzuei, D., & Boyaghchi, M. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.113029
Multi-objective Evolutionary Charging Scheduling for Electric Vehicles Integrating Regional Meteorology and Spatiotemporal Bimodal Characteristics
Wang, F., Li, W., & Lu, J. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.08.149
Reinforcement-Learning-Guided Two-Stage Multi-Energy Optimization for Carbon-Aware Charging Infrastructure
Aykut, T., & Güner, S. (2026) · Electronics · doi:10.3390/electronics15184294
Configuration and Scheduling of Hybrid Energy Storage Systems via Variational Mode Decomposition and Deep Reinforcement Learning
He, Y., Li, Z., Wu, X., Wang, Z., Liu, H., Di, X., Sun, K., & Xiangyu, L. (2026) · Energy Technology · doi:10.1002/ente.70625
Trajectory-Based Residual Deep Reinforcement Learning for Electric Vehicle Charging Scheduling With Grid Stability Constraints
Han, Y., Zhang, R., Fan, Z., Zhao, Q., Guo, G., & Si, F. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3738822
“Model-free deep reinforcement learning has been applied to real-time EV charging without repeatedly solving a complete mathematical program [17].”
A Hybrid Model‐Data‐Driven Scheduling Strategy for Vehicle‐to‐Grid Interaction Based on Virtual Power Plant Coordination
Chen, L., Lv, X., Fang, J., Zhou, Y., & Wang, K. (2026) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.70443
AI-Driven Multi-Agent Energy Management for Hybrid AC–DC Microgrids with EV Integration Using Transformer-Based Forecasting and Reinforcement Learning
Khan, S., Khan, I., Yadav, V. K., Hasan, S., & Kumar, P. (2026) · International Power Electronics and Motion Control Conference (EPE-PEMC) · doi:10.1109/pemc69244.2026.11726113
Transportation-energy nexus for renewable-rich electrified mobility: A review of modeling, planning, and control
Wu, X., Jiang, S., Shi, R., Bai, W., Fang, F., Teng, J., Xu, Y., & Liu, Z. (2026) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2026.105315
Intelligent EV Charging Infrastructure and Smart Charging Strategies: A Deep Reinforcement Learning-Based Optimization Framework
Palta, C., Jain, A., & Lohchab, Y. (2026) · IEEE International Conference on Renewable Energy and Sustainable E-Mobility (RESEM) · doi:10.1109/resem70152.2026.11721620
“Wan et al. [6] showed that even without explicitly programming for price responsiveness and solar-aware charging patterns, reinforcement learning (RL) can learn to develop these responses independently, surpassing traditional rule-based methods for managing EV charging in home energy systems.”
Machine Learning for Real‐Time Energy Optimization
Pindoo, I. A. (2026) · Wiley eBooks · doi:10.1002/9781394404346.ch1
A GRP hybrid optimization framework with pricing mechanisms for multi-objective EV charging station operation
Jia, G., Qian, J., Pang, Q., Liu, W., & Niu, F. (2026) · International Journal of Emerging Electric Power Systems · doi:10.1515/ijeeps-2026-0097
“Reinforcement learning (RL) provides a complementary mechanism for adaptive decision making, and Deep Deterministic Policy Gradient (DDPG) and Q -learning algorithms have been applied to real-time EV charging scheduling [26, 27].”

Authors: Wan, Z., Li, H., He, H., & Prokhorov, D.
Affiliations: University of Rhode Island (US)
Shenyang Institute of Automation (CN)
Toyota Motor North America Research & Development (United States) (US)
ORCID: Wan, Zhiqiang
Li, Hepeng
He, Haibo
Prokhorov, Danil
Venue: IEEE Transactions on Smart Grid
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019-09
Volume: 10
Issue: 5
Pages: 5246-5257
ISSN: 1949-3053 (ISSN-L)
1949-3061
DOI: 10.1109/tsg.2018.2879572
OpenAlex: W2899639849
Cited by: 548
References: 46
Citations by year: 2026: 61 · 2025: 71 · 2024: 83 · 2023: 95 · 2022: 87 · 2021: 74 · 2020: 57 · 2019: 20
FWCI: 14.43
Type: journal article
Language: en
Open access: green
digitalcommons.uri.edu
Licence: cc-by
ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Transactions on Smart Grid (publishedVersion)
Journal of Media Literacy Education (submittedVersion, cc-by)
SIA OpenIR (Chinese Academy of Sciences) (submittedVersion)
Funders: Office of Naval Research (N00014-18-1-2396)
Topics: Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Power System Optimization
Keywords: electric vehicle charging scheduling, real-time pricing, demand response, deep reinforcement learning, model-free learning, unknown transition probabilities, Markov decision process, Q-network, electricity price forecasting, smart grid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks