On-Line Building Energy Optimization Using Deep Reinforcement Learning
Open accessAbstract
Unprecedented high volumes of data are becoming available with the growth of the advanced metering infrastructure. These are expected to benefit planning and operation of the future power systems and to help customers transition from a passive to an active role. In this paper, we explore for the first time in the smart grid context the benefits of using deep reinforcement learning, a hybrid type of methods that combines reinforcement learning with deep learning, to perform on-line optimization of schedules for building energy management systems. The learning procedure was explored using two methods, Deep Q-learning and deep policy gradient, both of which have been extended to perform multiple actions simultaneously. The proposed approach was validated on the large-scale Pecan Street Inc. database. This highly dimensional database includes information about photovoltaic power generation, electric vehicles and buildings appliances. Moreover, these on-line energy scheduling strategies could be used to provide realtime feedback to consumers to encourage more efficient use of electricity.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
(2011) · Cited by 6,993 · see citing works
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855 · see citing works
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103 · see citing works
Optimal Residential Load Control With Price Prediction in Real-Time Electricity Pricing Environments
Mohsenian-Rad, A.-H., & Leon‐Garcia, A. (2010) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2010.2055903 · Cited by 1,851 · see citing works
Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN
Shi, H., Xu, M., & Li, R. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2686012 · Cited by 1,106 · see citing works
A Survey on Demand Response Programs in Smart Grids: Pricing Methods and Optimization Algorithms
Vardakas, J. S., Zorba, N., & Verikoukis, C. V. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2341586 · Cited by 951 · see citing works
He, Y., Mendis, G. J., & Wei, J. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2703842 · Cited by 844 · see citing works
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831 · see citing works
Deep learning based ensemble approach for probabilistic wind power forecasting
Wang, H., Li, G., Wang, G., Peng, J., Jiang, H. Q., & Liu, Y. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.11.111 · Cited by 754 · see citing works
Building energy load forecasting using Deep Neural Networks
Marino, D., Amarasinghe, K., & Manic, M. (2016) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2016.7793413 · Cited by 681 · see citing works
Deep learning for estimating building energy consumption
Mocanu, E., Nguyen, P. H., Gibescu, M., & Kling, W. L. (2016) · Sustainable Energy Grids and Networks · doi:10.1016/j.segan.2016.02.005 · Cited by 627 · see citing works
Self-citationOn rectified linear units for speech processing
Zeiler, M., Ranzato, M., Monga, R., Mao, M. Z., Yang, K., Le, Q. V., Nguyen, P., Senior, A., Vanhoucke, V., Dean, J., et al. (2013) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.2013.6638312 · Cited by 552 · see citing works
Self-citationDeep Neural Network Based Demand Side Short Term Load Forecasting
Ryu, S., Noh, J.-K., & Kim, H. (2016) · Energies · doi:10.3390/en10010003 · Cited by 414 · see citing works
Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning
Kim, B., Zhang, Y., van der Schaar, M., & Lee, J.-W. (2015) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2495145 · Cited by 299 · see citing works
Residential Demand Response Using Reinforcement Learning
O’Neill, D. C., Levorato, M., Goldsmith, A. J., et al. (2010) · IEEE International Conference on Smart Grid Communications (SmartGridComm) · doi:10.1109/smartgrid.2010.5622078 · Cited by 290 · see citing works
Dynamic Network Energy Management via Proximal Message Passing
Kraning, M. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601987150 · Cited by 260 · see citing works
Dynamic Network Energy Management via Proximal Message Passing
Kraning, M., Chu, E., Lavaei, J., & Boyd, S. (2013) · Foundations and Trends® in Optimization · doi:10.1561/2400000002 · Cited by 237 · see citing works
Paterakis, N. G., Erdinç, O., Pappi, I. N., Bakirtzis, A. G., & Catalào, J. P. S. (2016) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2015.2512501 · Cited by 224 · see citing works
Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System Problem
Ernst, D., Glavić, M., Capitanescu, F., & Wehenkel, L. A. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007630 · Cited by 214 · see citing works
Optimization Models and Methods for Demand-Side Management of Residential Users: A Survey
Barbato, A., & Capone, A. (2014) · Energies · doi:10.3390/en7095787 · Cited by 205 · see citing works
Mocanu, E., Nguyen, P. H., Kling, W. L., & Gibescu, M. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.01.030 · Cited by 171 · see citing works
Self-citationOnline Multi-Task Learning for Policy Gradient Methods
Ammar, H. B., Eaton, E., Ruvolo, P., & Taylor, M. E. (2014) · Cited by 151 · see citing works
Deep Reinforcement Learning Solutions for Energy Microgrids Management
François-Lavet, V., Taralla, D., Ernst, D., & Raphael Fonteneau, D. (2016) · ORBi (University of Liège) · Cited by 128 · see citing works
Deep learning versus traditional machine learning methods for aggregated energy demand prediction
Paterakis, N. G., Mocanu, E., Gibescu, M., Stappers, B., & van Alst, W. (2017) · 2017 IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe) · doi:10.1109/isgteurope.2017.8260289 · Cited by 118 · see citing works
Self-citationIntelligent Buildings of the Future: Cyberaware, Deep Learning Powered, and Human Interacting
Manic, M., Amarasinghe, K., Rodriguez-Andina, J. J., et al. (2016) · IEEE Industrial Electronics Magazine · doi:10.1109/mie.2016.2615575 · Cited by 102 · see citing works
Ruelens, F., Claessens, B., Vandael, S., et al. (2014) · Power Systems Computation Conference · doi:10.1109/pscc.2014.7038106 · Cited by 81 · see citing works
Decentralized Multi-Period Economic Dispatch for Real-Time Flexible Demand Management
Loukarakis, E., Dent, C. J., & Bialek, J. W. (2015) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2015.2402518 · Cited by 72 · see citing works
Comparison of machine learning methods for estimating energy consumption in buildings
Mocanu, E., Nguyen, P. H., Gibescu, M., et al. (2014) · International Conference on Probabilistic Methods Applied to Power Systems (PMAPS) · doi:10.1109/pmaps.2014.6960635 · Cited by 55 · see citing works
Self-citationComputational Methods for Residential Energy Cost Optimization in Smart Grids
Alam, M. R., St‐Hilaire, M., & Kunz, T. (2016) · ACM Computing Surveys · doi:10.1145/2897165 · Cited by 38 · see citing works
Demand forecasting at low aggregation levels using Factored Conditional Restricted Boltzmann Machine
Mocanu, E., Nguyen, P. H., Gibescu, M., Larsen, E. M., & Pinson, P. (2016) · Power Systems Computation Conference · doi:10.1109/pscc.2016.7540994 · Cited by 27 · see citing works
Self-citationHurtado, L. A., Mocanu, E., Nguyen, P. H., et al. (2015) · International Conference on Smart Cities and Green ICT Systems · doi:10.5220/0005444101570166 · Cited by 16 · see citing works
Self-citationMeasuring Short-term Air Conditioner Demand Reductions for Operations and Settlement
Bode, J. (2013) · eScholarship (California Digital Library) · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774 · see citing works
Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review
Cao, D., Hu, W., Zhao, J., Zhang, G., Zhang, B., Liu, Z., Chen, Z., & Blaabjerg, F. (2020) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2020.000552 · Cited by 460 · see citing works
A Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home Energy Management
Xu, X., Jia, Y., Xu, Y., Xu, Z., Chai, S., & Lai, C. S. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2971427 · Cited by 420 · see citing works
Fusing TensorFlow with building energy simulation for intelligent energy management in smart cities
Vázquez-Canteli, J. R., Ulyanin, S., Kämpf, J., & Nagy, Z. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2018.11.021 · Cited by 195 · see citing works
Tackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336 · see citing works
Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids
Chung, H.-M., Maharjan, S., Zhang, Y., & Eliassen, F. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3007167 · Cited by 120 · see citing works
Álvarez-López, C., González-Briones, A., & Li, T. (2026) · Electronics · doi:10.3390/electronics15020385 · Cited by 23 · see citing works
A New Anti-Jamming Strategy Based on Deep Reinforcement Learning for MANET
Xu, Y., Lei, M., Li, M., Zhao, M., & Hu, B. (2019) · 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring) · doi:10.1109/vtcspring.2019.8746494 · Cited by 14 · see citing works
- Authors
- Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G.
- Affiliations
- Eindhoven University of Technology (NL)
- University of Derby (GB)
- The University of Texas at Austin (US)
- ORCID
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2019-07
- Volume
- 10
- Issue
- 4
- Pages
- 3698-3708
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W2963317745
- Cited by
- 649
- References
- 36
- Citations by year
- 2026: 60 · 2025: 74 · 2024: 77 · 2023: 77 · 2022: 108 · 2021: 100 · 2020: 97 · 2019: 43 · 2018: 13
- FWCI
- 24.20
- Type
- journal article
- Language
- en
- Open access
- green
- research.tue.nl
- Licence
- Locations
- IEEE Transactions on Smart Grid (publishedVersion)
- TU/e Research Portal (publishedVersion, other-oa)
- Edinburgh Napier Research Repository (Edinburgh Napier University) (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion)
- TU/e Research Portal (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Research Output (Edinburgh Napier University) (submittedVersion, other-oa)
- Open Research (University of Surrey) (submittedVersion)
- TU/e Research Portal (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- NL Enterprise Agency through the TKI SG-BEMS project of Dutch Top Sector
- Horizon 2020 Framework Programme (687283)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Physical Sciences › Engineering › Building and Construction › Building Energy and Comfort Optimization
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- building energy management systems, deep reinforcement learning, deep Q-learning, smart grid, advanced metering infrastructure, photovoltaic generation, electric vehicles, appliance scheduling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
