The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning
Open accessAbstract
Driven by the recent advances in electric vehicle (EV) technologies, EVs have become important for smart grid economy. When EVs participate in demand response program which has real-time pricing signals, the charging cost can be greatly reduced by taking full advantage of these pricing signals. However, it is challenging to determine an optimal charging strategy due to the existence of randomness in traffic conditions, user's commuting behavior, and the pricing process of the utility. Conventional model-based approaches require a model of forecast on the uncertainty and optimization for the scheduling process. In this paper, we formulate this scheduling problem as a Markov Decision Process (MDP) with unknown transition probability. A model-free approach based on deep reinforcement learning is proposed to determine the optimal strategy for this problem. The proposed approach can adaptively learn the transition probability and does not require any system model information. The architecture of the proposed approach contains two networks: a representation network to extract discriminative features from the electricity prices and a Q network to approximate the optimal action-value function. Numerous experimental results demonstrate the effectiveness of the proposed approach.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For load monitoring and electric vehicle charging scheduling, transfer learning [200] and deep reinforcement learning [215] methods are proposed.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [215]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
A Critical Review of Recurrent Neural Networks for Sequence LearningPreprint
Lipton, Z. C., Berkowitz, J. A., & Elkan, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00019 · Cited by 3,159
Short-Term Residential Load Forecasting Based on LSTM Recurrent Neural Network
Kong, W., Dong, Z. Y., Jia, Y., Hill, D. J., Xu, Y. H., & Zhang, Y. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2753802 · Cited by 2,806
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Flagged first, then by citations
Research works printed but not linked: 5
- bellman, 1958, Dynamic Programming
- 2018, Grid-Connected Renewable Energy Systems
- goodfellow, 2016, Deep Learning, 1
- 2017, The Egallon How Much Cheaper is it to Drive on Electricity?
- 2017, California Independent System Operator
Cited by
Sorted by citations
Items 101–200 of 547
Distributed Energy Trading and Scheduling Among Microgrids via Multiagent Reinforcement Learning
Gao, G., Wen, Y., & Tao, D. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3170070 · Cited by 60
“Therefore, it has been widely adopted for energy scheduling, such as electric vehicle (EV) charging [32], power management [33], and energy scheduling in smart buildings and microgrids [9], [34], [35].”
BackgroundTan, M., Dai, Z., Su, Y., Chen, C., Wang, L., & Chen, J. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105557 · Cited by 58
Zhang, Y., Rao, X., Liu, C., Zhang, X., & Zhou, Y. F. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105642 · Cited by 58
Li, F., Liu, L., & Yu, Y. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125558 · Cited by 58
An Advanced Satisfaction-Based Home Energy Management System Using Deep Reinforcement Learning
Forootani, A., Rastegar, M., & Jooshaki, M. (2022) · IEEE Access · doi:10.1109/access.2022.3172327 · Cited by 57
“In [17]-[18], DQN is adopted for the optimal EV charging and navigation, respectively.”
BackgroundMethodInfluentialWang, K., Wang, H., Wang, H., Feng, J., Li, Y., Yang, J., & Chen, Z. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121186 · Cited by 57
Gan, L., Chen, X., Yu, K., Zheng, J., & Du, W. (2020) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2020.2989690 · Cited by 57
“on historical data [12], the other is the simulation methods based on probabilistic distribution analysis of the influencing factors [13].”
MethodWang, C., Zhang, J., Wang, A., Wang, Z., Yang, N., Zhao, Z., Lai, C. S., & Lai, L. L. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.123471 · Cited by 56
Li, S., Hu, W., Cao, D., Zhang, Z., Huang, Q., Chen, Z., & Blaabjerg, F. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3139650 · Cited by 60
“Normally, different EV owners have different commuting behaviors due to the difference of individual habits, traffic [27], [28], etc.”
BackgroundChu, Y., Wei, Z., Fang, X., Chen, S., & Zhou, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3206020 · Cited by 58
“[15] 69 developed a model-free DRL framework to adaptively learn 70 the dynamics of the changing environment and determine 71 the optimal scheduling strategy.”
MethodEV Charging Behavior Analysis Using Hybrid Intelligence for 5G Smart Grid
Shen, Y., Fang, W., Ye, F., & Kadoch, M. (2020) · Electronics · doi:10.3390/electronics9010080 · Cited by 55
“For example, some papers have proposed a real-time EV charging strategy based on reinforcement learning [16].”
BackgroundLai, C. S., Chen, D., Zhang, J., Zhang, X., Xu, X., Taylor, G. A., & Lai, L. L. (2022) · Energy · doi:10.1016/j.energy.2022.124852 · Cited by 54
An Imbalance Fault Detection Algorithm for Variable-Speed Wind Turbines: A Deep Learning Approach
Chen, J., Hu, W., Cao, D., Zhang, B., Huang, Q., Chen, Z., & Blaabjerg, F. (2019) · Energies · doi:10.3390/en12142764 · Cited by 52
“The most important part to the LSTM network is the cell state Lt [37].”
BackgroundYe, Y., Qiu, D., Wang, H., Tang, Y. P., & Štrbac, G. (2021) · Energies · doi:10.3390/en14030531 · Cited by 53
“(1) State: The state st at step t received by the HEMS agent entails the influence of its action on the status of the environment. st is identified as an 11-dimensional vector st = [t, λ+t , λ − t , H out t , H in t , P d t , P pv t , E ev t , E es t , A ev t , A wa t ] ∈ S , which comprises the following sensory info…”
MethodInfluentialLiu, J., Lin, G., Rehtanz, C., Huang, S., Zhou, Y., & Li, Y. (2021) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108218 · Cited by 52
Moniruzzaman, M., Yassine, A., & Benlamri, R. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3262597 · Cited by 51
“The concept of charging behavior is significantly inspired by the approach taken in [44].”
MethodDiffCharge: Generating EV Charging Scenarios via a Denoising Diffusion Model
Li, S., Xiong, H., & Chen, Y. (2024) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2024.3360874 · Cited by 58
“In this regard, grid operators are required to properly manage the increasing EV integration using coordinated charging strategies to both fulfill grid operation goals and meet user charging needs [3].”
BackgroundZhang, H., Zhang, G., Zhao, M., & Liu, Y. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3371568 · Cited by 52
“We assume that the ESM system can provide discrete charging/discharging energy, i.e., a t ∈ { F 1 , . . . , F L } as suggested by [41].”
BackgroundReal-time feedback based online aggregate EV power flexibility characterization
Yan, D., Huang, S., & Chen, Y. (2024) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2023.3324705 · Cited by 51
“A deep Q-network (DQN) was used to approximate the optimal action-value function to determine the EV charging strategy [27].”
MethodRani, G. S. A., Priya, P. S. L., Jayan, J., Satheesh, R., & Kolhe, M. L. (2024) · IEEE Access · doi:10.1109/access.2024.3398059 · Cited by 51
“In [22], Wan et al. implemented a model-free methodology utilizing DRL to establish the most effective strategy for EV charging.”
MethodChen, G., Yang, L., & Cao, X. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124706 · Cited by 52
Qiu, D., Wang, Y., Sun, M., & Štrbac, G. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.118790 · Cited by 53
Sharifi, P., Banerjee, A., & Feizollahi, M. J. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106762 · Cited by 48
“The EV arrival and departure time are dynamic and time-varying since the driving behavior of EV owners is uncertain as it depends on several factors, such as traffic conditions, and randomness in commuting behavior of drivers [15].”
BackgroundYang, F., Gao, F., Liu, B., & Ci, S. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3142406 · Cited by 52
Li, S., Hu, W., Cao, D., Zhang, Z., Huang, Q., Chen, Z., & Blaabjerg, F. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3167021 · Cited by 52
Dual Alignment for Partial Domain Adaptation
Li, L., Wan, Z., & He, H. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2983337 · Cited by 48
“Reinforcement learning performs a certain goal by interacting with a dynamic environment [26], [27].”
BackgroundSelf-citationHo, G. T. S., Tang, Y. M., Leung, E. K. H., & Tong, P. H. (2025) · Transportation Research Part E Logistics and Transportation Review · doi:10.1016/j.tre.2025.104008 · Cited by 47
Xu, X., Xu, Y. H., Wang, M., Li, J., Xu, Z., Chai, S., & He, Y. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3016336 · Cited by 47
“By comparison, as a modelfree, adaptive, and concise machine learning technique [13], reinforcement learning exhibits excellent performance on the decision-making process.”
BackgroundPower output optimization of electric vehicles smart charging hubs using deep reinforcement learning
Bertolini, A., Martins, M. S. E., Vieira, S. M., & Sousa, J. M. C. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116995 · Cited by 46
Constrained large-scale real-time EV scheduling based on recurrent deep reinforcement learning
Li, H., Li, G., Lie, T. T., Li, X., Wang, K., Han, B., & Xu, J. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108603 · Cited by 46
Self-citationMulti-agent reinforcement learning for electric vehicle decarbonized routing and scheduling
Wang, Y., Qiu, D., He, Y., Zhou, Q., & Štrbac, G. (2023) · Energy · doi:10.1016/j.energy.2023.129335 · Cited by 48
Jawad, S., & Liu, J. (2020) · Energies · doi:10.3390/en13133371 · Cited by 46
Zhang, J., Guan, Y., Che, L., & Shahidehpour, M. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3281782 · Cited by 46
“Compared with [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], this overcomes the limitation that existing DRL methods cannot strictly guarantee the allocation accuracy and charging security under the DSO’s frequently changing commands.”
BackgroundMethodResultInfluentialMaeng, J., Min, D., & Kang, Y. (2023) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2023.101224 · Cited by 49
Jin, J., & Xu, Y. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3181613 · Cited by 46
“A variety of DRL methods, including deep Q -learning [17], safe DRL [18], and soft-actor–critic [19], has been adopted to deal with the unknown dynamics of system uncertainties in renewable generation and electricity prices.”
BackgroundZhou, H., Fan, S., Wu, Q. H., Dong, L., Li, Z., & He, G. (2021) · Applied Energy · doi:10.1016/j.apenergy.2020.116424 · Cited by 44
Jiang, Y., Ye, Q., Sun, B., Wu, Y., & Tsang, D. H. K. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3135977 · Cited by 43
“Looking at the economic aspect, Li et al. [15], Yan et al. [16], Zhang et al. [18], and Wan et al. [24] envisioned a future smart grid with V2G transmission enabled.”
BackgroundShadow-Price DRL: A Framework for Online Scheduling of Shared Autonomous EVs Fleets
Qian, T., Shao, C., Wang, X., Zhou, Q., & Shahidehpour, M. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3155455 · Cited by 44
Zou, L., Munir, M. S., Tun, Y. K., Kang, S., & Hong, C. S. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3160210 · Cited by 44
“Therefore, for simplicity, when determining this price, we assume that battery degradation cost [44] and other base costs (e.”
BackgroundPaudel, D., & Das, T. K. (2023) · Energy · doi:10.1016/j.energy.2023.129097 · Cited by 43
Reinforcement Learning for Electric Vehicle Charging Management: Theory and Applications
Michailidis, P., Michailidis, I., & Kosmatopoulos, E. B. (2025) · Energies · doi:10.3390/en18195225 · Cited by 56
Reka, S. S., Venugopal, P., Alhelou, H. H., Siano, P., & Golshan, M. E. H. (2021) · IEEE Access · doi:10.1109/access.2021.3071993 · Cited by 43
“…strategy is implemented for achieving optimal charging strategy of an EV by considering many constraints involved during EV charging process [23] An online based optimal energy optimization technique is proposed for scheduling time-scaling and time-shifting load using DQN and deterministic…”
MethodLiu, L., & Li, Z. (2022) · IEEE Access · doi:10.1109/access.2022.3176444 · Cited by 43
He, X., Ge, S., Liu, H., Xu, Z., Yang, M., & Wang, C. (2022) · Electric Power Systems Research · doi:10.1016/j.epsr.2022.108962 · Cited by 42
Zheng, X., Liang, C., Wang, Y., Shi, J., & Lim, G. J. (2022) · Mathematics · doi:10.3390/math10234575 · Cited by 42
“In some cases, the state space becomes so large that the learning problem becomes intractable [23].”
BackgroundGharibi, M. A., Nafisi, H., Abyaneh, H. A., & Hajizadeh, A. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121614 · Cited by 42
Reinforcement learning for vehicle-to-grid: A review
Xie, H., Song, G., Shi, Z., Zhang, J., Lin, Z., Yu, Q., Fu, H., Song, X., & Zhang, H. (2025) · Advances in Applied Energy · doi:10.1016/j.adapen.2025.100214 · Cited by 53
A Blockchain-Enabled Demand Management and Control Framework Driven by Deep Reinforcement Learning
Ma, R., Yi, Z., Xiang, Y., Shi, D., Xu, C., & Wu, H. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3146631 · Cited by 41
“DQN is also used in [16] and [17] to help optimally manage the charging and discharging of EVs.”
MethodDolatabadi, A., Abdeltawab, H. H., & Mohamed, Y. A. I. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3177704 · Cited by 43
“In this paper, the experience replay mechanism is used to break the temporally correlated transitions and alleviate this limitation [14], [25].”
MethodQuantum Reinforcement Learning for real-time optimization in Electric Vehicle charging systems
Xu, H., Zhang, A., Wang, Q., Hu, Y., Fang, F., & Cheng, L. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.125279 · Cited by 41
Autonomous PEV Charging Scheduling Using Dyna-Q Reinforcement Learning
Wang, F., Gao, J., Li, M., & Zhao, L. Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3026004 · Cited by 41
“The proposed scheme is compared to five other charging schemes: (cid:2) Model-free DQN proposed in [33].”
MethodInfluentialGuo, G., & Gong, Y. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108061 · Cited by 41
Chen, X., Qu, G., Tang, Y., Low, S. H., & Li, N. (2021) · CaltechAUTHORS (California Institute of Technology) · Cited by 42
Zhang, H., Liao, K., Yang, J., Yin, Z., & He, Z. Y. (2025) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2025.3529215 · Cited by 40
“DRL can avoid complex models with excellent performance in dealing with diverse scenarios and multiple uncertainties [19], [20].”
Karmaker, A. K., Behrens, S., Hossain, M. J., & Pota, H. R. (2023) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.139145 · Cited by 39
Real-Time Scheduling for Electric Vehicles Charging/Discharging Using Reinforcement Learning
Mhaisen, N., Fetais, N., & Massoud, A. M. (2020) · IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT) · doi:10.1109/iciot48696.2020.9089471 · Cited by 39
“Such charging power discretization has been assumed before in scheduling problems [6], [12], [14].”
BackgroundMulti-agent DRL-based data-driven approach for PEVs charging/discharging scheduling in smart grid
Wan, Y., Qin, J., Ma, Q., Fu, W., & Wang, S. (2022) · Journal of the Franklin Institute · doi:10.1016/j.jfranklin.2022.01.016 · Cited by 38
Neural combinatorial optimization with reinforcement learning in industrial engineering: a survey
Chung, K. T., Lee, C. K. M., & Tsang, Y. P. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11045-1 · Cited by 43
“There is a common focus among the reviewed papers, namely, James et al. (2019) and Wan et al. (2018), on intelligent transportation systems with electric vehicles and power systems.”
Deep Inverse Reinforcement Learning for Objective Function Identification in Bidding Models
Guo, H., Chen, Q., Xia, Q. L., & Kang, C. (2021) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2021.3076296 · Cited by 38
Wang, R., Chen, Z., Xing, Q., Zhang, Z., & Zhang, T. (2022) · Sustainability · doi:10.3390/su14031884 · Cited by 37
Optimal Operation of a Microgrid with Hydrogen Storage Based on Deep Reinforcement Learning
Zhu, Z., Weng, Z., & Zheng, H. (2022) · Electronics · doi:10.3390/electronics11020196 · Cited by 36
“Currently, deep reinforcement learning has been used to achieve certain results in several areas, such as reactive power optimization [13,14], electric vehicles [15,16], and power markets [17,18].”
BackgroundParaskevas, A., Aletras, D., Chrysopoulos, A. C., Marinopoulos, A. G., & Doukas, D. I. (2022) · Energies · doi:10.3390/en15072323 · Cited by 36
Integrating Incentive Factors in the Optimization for Bidirectional Charging of Electric Vehicles
Li, S., Zhao, P., Gu, C., Bu, S., Li, J., & Cheng, S. (2023) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2023.3308233 · Cited by 38
“Recentlydevelopeddeepreinforcementlearning(DRL)tech- nologiesbringabrightprospecttoresolvingtheaboveprob-lems[25],[26].”
BackgroundSV2G-ET: A Secure Vehicle-to-Grid Energy Trading Scheme Using Deep Reinforcement Learning
Kumari, A., Trivedi, M., Tanwar, S., Sharma, G., & Sharma, R. (2022) · International Transactions on Electrical Energy Systems · doi:10.1155/2022/9761157 · Cited by 34
Computational Challenges and Approaches for Electric Vehicles
Kalakanti, A. K., & Rao, S. (2023) · ACM Computing Surveys · doi:10.1145/3582076 · Cited by 38
“The authors of [172, 185] proposed a dynamic pricing technique based on multiagent RL for charge scheduling.”
MethodCui, D., Wang, Z., Liu, P., Wang, S., Zhao, Y., & Zhan, W. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.120798 · Cited by 34
Wu, Z. S., Li, Y., Gu, W., Dong, Z., Zhao, J., Liu, W., Zhang, X., Liu, P., & Sun, Q. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108830 · Cited by 34
Multi-Agent DDPG Based Electric Vehicles Charging Station Recommendation
Bachiri, K., Yahyaouy, A., Gualous, H., Malek, M., Bennani, Y., Makany, P., & Rogovschi, N. (2023) · Energies · doi:10.3390/en16166067 · Cited by 33
“[33] formulated the electric vehicle charging scheduling problem using a Markov decision process (MDP) based on deep reinforcement learning (DRL), engaging with the environment to teach the ideal policy and handling the randomness in an EV’s arrival and start charge time.”
MethodAzzouz, I., & Hassen, W. F. (2023) · Energies · doi:10.3390/en16248102 · Cited by 33
A Deep Reinforcement Learning Based Approach for Home Energy Management System
Li, H., Wan, Z., & He, H. (2020) · IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT) · doi:10.1109/isgt45199.2020.9087647 · Cited by 34
“4) Rewards: The reward is formulated as where the first term is the electricity cost at time step t ; the second term is the EV range anxiety, which is the square of the unfulfilled energy to fully charge the EV at the departure time [12].”
BackgroundSelf-citationSuspension Regulation of Medium-Low-Speed Maglev Trains Via Deep Reinforcement Learning
Zhao, F., You, K., Song, S., Zhang, W., & Tong, L. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3097313 · Cited by 34
DiffPLF: A conditional diffusion model for probabilistic forecasting of EV charging load
Li, S., Xiong, H., & Chen, Y. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.110723 · Cited by 39
Li, S., Zhao, P., Gu, C., Bu, S., Chung, E., Tian, Z., Li, J., & Cheng, S. (2024) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2024.3425843 · Cited by 31
“Similarly, battery degradation is simplified to a function of total energy output and C-rate in [24] and [25] to boost DRL model trainingbysacrificingV2Ganti-agingperformance.”
Hao, L., Jin, J., & Xu, Y. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3159189 · Cited by 32
TRUST-REGION BASED POLICY OPTIMIZATION FOR EFFICIENT REINFORCEMENT LEARNING
Li, H. (2023) · Digital Commons - URI (University of Rhode Island) · doi:10.23860/diss-li-hepfeng-2023 · Cited by 32
Self-citationGu, N., Cui, J., & Wu, C. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3265398 · Cited by 31
“DRL solves tasks that either the cost (reward, in [29]) or transition function (dynamics, in [18], [30]) is unknown and needs to be explored by sampling, which is different in our problem because both kinds of information are well-defined.”
BackgroundJiao, F., Zou, Y., Zhou, Y. F., Zhang, Y., & Zhang, X. (2023) · Energy · doi:10.1016/j.energy.2023.128410 · Cited by 31
Panda, D. K., Turner, O., Das, S., & Abusara, M. A. (2023) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.139947 · Cited by 31
Enabling Optimal Control Under Demand Elasticity for Electric Vehicle Charging Systems
Fan, G., Yang, Z., Jin, H., Gan, X., & Wang, X. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3015121 · Cited by 31
Dong, J., Wang, H., Yang, J., Lu, X., Liu, G., & Zhou, X. (2021) · IEEE Access · doi:10.1109/access.2021.3114335 · Cited by 31
“The parameters of DQN are selected from [18].”
BackgroundZhu, J., Hu, W., Xu, X., Liu, H., Li, P., Fan, H., Zhang, Z., & Chen, Z. (2022) · Renewable Energy · doi:10.1016/j.renene.2022.10.094 · Cited by 31
A novel forecasting approach to schedule aggregated electric vehicle charging
Brinkel, N., Visser, L., van Sark, W. G. J. H. M., & AlSkaif, T. A. (2023) · Energy and AI · doi:10.1016/j.egyai.2023.100297 · Cited by 31
Deep Q-Learning-Based Smart Scheduling of EVs for Demand Response in Smart Grids
Chifu, V. R., Cioara, T., Pop, C. B., Rusu, H. G., & Anghel, I. M. (2024) · Applied Sciences · doi:10.3390/app14041421 · Cited by 30
Yin, B., Weng, H., Hu, Y., Xi, J., Ding, P., & Liu, J. (2024) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2024.3404472 · Cited by 30
A hybrid policy gradient and rule-based control framework for electric vehicle charging
Mbuwir, B. V., Vanmunster, L., Thoelen, K., & Deconinck, G. (2021) · Energy and AI · doi:10.1016/j.egyai.2021.100059 · Cited by 32
Chang, X., Jia, X., Fu, S., Hu, H., & Liu, K. (2022) · Proceedings of the Institution of Mechanical Engineers Part B Journal of Engineering Manufacture · doi:10.1177/09544054221121934 · Cited by 29
“The deep Q network (DQN), as a classical DRL method, has widely studied over the last several years.(39) For example, Waschneck et al.”
MethodLi, S., Hu, W., Cao, D., Chen, Z., Huang, Q., Blaabjerg, F., & Liao, K. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.121359 · Cited by 29
Pan, W., Yu, X., Guo, Z., Qian, T., & Li, Y. (2024) · Energies · doi:10.3390/en17112491 · Cited by 28
Salehpour, M. J., & Hossain, M. J. (2024) · Journal of Energy Storage · doi:10.1016/j.est.2024.112151 · Cited by 32
Tian, J., Jia, H., Wang, G., Huang, Q., Wu, R., Gao, H., & Liu, C. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129343 · Cited by 28
Alfaverh, F., Denai, M. A., & Sun, Y. (2021) · IET Electrical Systems in Transportation · doi:10.1049/els2.12030 · Cited by 28
“Other studies have focussed on using Markov Decision Process (MDP) algorithms in HEMS to determine the optimal strategy for the scheduling of EV charging and discharging [16].”
MethodSmart Scheduling of Electric Vehicles Based on Reinforcement Learning
Viziteu, A., Furtuna, D., Robu, A., Senocico, S., Cioată, P., Marian, B. R., Raboacă, M. S., & Constantin, F. (2022) · Sensors · doi:10.3390/s22103718 · Cited by 28
Chu, Y., Wei, Z., Sun, G., Zang, H., Chen, S., & Zhou, Y. (2022) · Electric Power Systems Research · doi:10.1016/j.epsr.2022.108617 · Cited by 28
Raja, G., Saravanan, G., Prathiba, S. B., Akhtar, Z., Khowaja, S. A., & Dev, K. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3244854 · Cited by 28
“Wan et al. [15] and Xing et al. [16] performed a detailed analysis of several EV charge scheduling strategies and illustrated the benefits of the proposed solution.”
BackgroundModel-data-event based community integrated energy system low-carbon economic scheduling
Lin, X., Wang, J., Zhang, Y., Yong, W., Qi, J., & Li, H. (2023) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2023.113379 · Cited by 27
Self-citationJeon, S., Nguyen, T. T., & Choi, D. (2023) · IEEE Access · doi:10.1109/access.2023.3264687 · Cited by 26
“A model-free DRL approach for real-time scheduling of EV charging and discharging was developed in [24].”
MethodSmart Electric Vehicle Charging Management Using Reinforcement Learning on FPGA Platforms
Damodarin, U. M., Cardarilli, G. C., Nunzio, L. D., Re, M., & Spanò, S. (2025) · Sensors · doi:10.3390/s25082585 · Cited by 32
“In this context, Reinforcement Learning (RL) [11] offers a promising approach to intelligently manage EV charging [12,13].”
Jin, J., & Xu, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3111414 · Cited by 28
Multiadvisor Reinforcement Learning for Multiagent Multiobjective Smart Home Energy Control
Tittaferrante, A., & Yassine, A. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3125918 · Cited by 26
Hou, H., Ge, X., Chen, Y., Tang, J., Hou, T., & Fang, R. (2022) · Energy and Buildings · doi:10.1016/j.enbuild.2022.112594 · Cited by 26
Metadata
- Authors
- Wan, Z., Li, H., He, H., & Prokhorov, D.
- Affiliations
- University of Rhode Island (US)
- Shenyang Institute of Automation (CN)
- Toyota Motor North America Research & Development (United States) (US)
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2019-09
- Volume
- 10
- Issue
- 5
- Pages
- 5246-5257
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W2899639849
- Cited by
- 548
- References
- 46
- Citations by year
- 2026: 61 · 2025: 71 · 2024: 83 · 2023: 95 · 2022: 87 · 2021: 74 · 2020: 57 · 2019: 20
- FWCI
- 14.43
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- IEEE Transactions on Smart Grid (publishedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- SIA OpenIR (Chinese Academy of Sciences) (submittedVersion)
- Funders
- Office of Naval Research (N00014-18-1-2396)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Vehicles and Infrastructure
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Electric Power System Optimization
- Keywords
- electric vehicle charging scheduling, real-time pricing, demand response, deep reinforcement learning, model-free learning, unknown transition probabilities, Markov decision process, Q-network, electricity price forecasting, smart grid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
