Unsupervised energy prediction in a Smart Grid context using reinforcement cross-building transfer learning
Open accessAbstract
• We propose a new prediction method to model building energy consumption in Smart Grid. • It incorporates deep learning into reinforcement learning for continuous states estimation. • The learned model is transferred to a new building to predict energy consumption. • The method is validated on a real database recorded over seven years, with hourly resolution. In a future Smart Grid context, increasing challenges in managing the stochastic local energy supply and demand are expected. This increased the need of more accurate energy prediction methods in order to support further complex decision-making processes. Although many methods aiming to predict the energy consumption exist, all these require labelled data, such as historical or simulated data. Still, such datasets are not always available under the emerging Smart Grid transition and complex people behaviour. Our approach goes beyond the state-of-the-art energy prediction methods in that it does not require labelled data. Firstly, two reinforcement learning algorithms are investigated in order to model the building energy consumption. Secondly, as a main theoretical contribution, a Deep Belief Network (DBN) is incorporated into each of these algorithms, making them suitable for continuous states. Thirdly, the proposed methods yield a cross-building transfer that can target new behaviour of existing buildings (due to changes in their structure or installations), as well as completely new types of buildings. The methods are developed in the MATLAB ® environment and tested on a real database recorded over seven years, with hourly resolution. Experimental results demonstrate that the energy prediction accuracy in terms of RMSE has been significantly improved in 91.42% of the cases after using a DBN for automatically extracting high-level features from the unlabelled data, compared to the equivalent methods without the DBN pre-processing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Mocanu et al. modeled building load profiles with RL and deep belief networks and used transfer learning to predict new buildings.
“For instance, Mocanu et al. [544] modeled building load profiles with RL and deep belief networks using data on commercial and residential buildings; they then used approximate RL and transfer learning to make predictions about new buildings, enabling the transfer of knowledge from commercial to residential buildings, and from gas-to power-heated buildings.”
From Tackling climate change with machine learning (2023) · cites this paper as [544]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Watkins, C., & Dayan, P. (1992) · Machine-mediated learning · doi:10.1023/a:1022676722315 · Cited by 14,277 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037 · see citing works
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500769 · Cited by 7,210 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 7,048 · see citing works
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,865 · see citing works
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · Cited by 6,831 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 5,037 · see citing works
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · see citing works
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.2307/2280095 · Cited by 3,690 · see citing works
A Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852 · see citing works
Dimensionality Reduction: A Comparative Review
van der Maaten, L., Postma, E. O., & van den Herik, H. J. (2009) · Cited by 2,066 · see citing works
A review on the prediction of building energy consumption
Zhao, H., & Magoulès, F. (2012) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2012.02.049 · Cited by 1,961 · see citing works
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A., & Puterman, M. L. (1995) · Technometrics · doi:10.2307/1269932 · Cited by 1,889 · see citing works
Thermal comfort and building energy consumption implications—A review
Yang, L., Yan, H., & Lam, J. C. (2014) · Applied Energy · doi:10.1016/j.apenergy.2013.10.062 · Cited by 1,448 · see citing works
Energy models for demand forecasting—A review
Suganthi, L., & Samuel, A. A. (2011) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2011.08.014 · Cited by 1,198 · see citing works
State of the art in building modelling and energy performances prediction: A review
Foucquier, A., Robert, S., Suard, F., Stéphan, L., & Jay, A. (2013) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2013.03.004 · Cited by 835 · see citing works
Improving Elevator Performance Using Reinforcement Learning
Crites, R. H., & Barto, A. G. (1995) · Cited by 499 · see citing works
Building operation and energy performance: Monitoring, analysis and optimisation toolkit
Costa, A., Keane, M. M., Torrens, J. I., & Corry, E. (2011) · Applied Energy · doi:10.1016/j.apenergy.2011.10.037 · Cited by 396 · see citing works
Robust estimation and outlier detection with correlation coefficients
Devlin, S. J., Gnanadesikan, R., & Kettenring, J. R. (1975) · Biometrika · doi:10.1093/biomet/62.3.531 · Cited by 385 · see citing works
Artificial neural networks for energy analysis of office buildings with daylighting
Wong, S. L., Wan, K. K. W., & Lam, T. N. T. (2009) · Applied Energy · doi:10.1016/j.apenergy.2009.06.028 · Cited by 323 · see citing works
Electric load forecasting by support vector model
Hong, W. (2008) · Applied Mathematical Modelling · doi:10.1016/j.apm.2008.07.010 · Cited by 314 · see citing works
Köksal, M. A., & Ugursal, V. I. (2007) · Applied Energy · doi:10.1016/j.apenergy.2006.09.012 · Cited by 274 · see citing works
Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System Problem
Ernst, D., Glavić, M., Capitanescu, F., & Wehenkel, L. A. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007630 · Cited by 214 · see citing works
Reinforcement Learning: State-of-the-Art
Nowé, A., Vrancx, P., & Hauwere, Y.-M. D. (2012) · Cited by 189 · see citing works
Reinforcement Learning with Factored States and Actions
Sallans, B., & Hinton, G. E. (2004) · doi:10.5555/1005332.1016794 · Cited by 139 · see citing works
Artificial neural networks in energy applications in buildings
Kalogirou, S. A. (2006) · International Journal of Low-Carbon Technologies · doi:10.1093/ijlct/1.3.201 · Cited by 124 · see citing works
A Comparison of Smart Grid Technologies and Progresses in Europe and the U.S.
Simões, M. G., Roche, R., Kyriakides, E., et al. (2012) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2012.2199730 · Cited by 117 · see citing works
Two Distributed-State Models For Generating High-Dimensional Time Series
Taylor, G. W., Hinton, G. E., & Roweis, S. T. (2011) · doi:10.5555/1953048.2021035 · Cited by 116 · see citing works
Artificial intelligence for energy conservation in buildings
Dounis, A. I. (2010) · Advances in Building Energy Research · doi:10.3763/aber.2009.0408 · Cited by 103 · see citing works
Learning Deep Boltzmann Machines using Adaptive MCMC
Salakhutdinov, R. (2010) · Cited by 73 · see citing works
Approximate reinforcement learning: An overview
Buşoniu, L., Ernst, D., Schutter, B. D., et al. (2011) · IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning · doi:10.1109/adprl.2011.5967353 · Cited by 65 · see citing works
Comparison of machine learning methods for estimating energy consumption in buildings
Mocanu, E., Nguyen, P. H., Gibescu, M., et al. (2014) · International Conference on Probabilistic Methods Applied to Power Systems (PMAPS) · doi:10.1109/pmaps.2014.6960635 · Cited by 55 · see citing works
Self-citationA Novel Hybrid Approach of KPCA and SVM for Building Cooling Load Prediction
Xuemei, L., Ding, L., Jinhu, L., et al. (2010) · doi:10.1109/wkdd.2010.137 · Cited by 53 · see citing works
Actor-Critic Reinforcement Learning with Energy-Based Policies
Heess, N., Silver, D., & Teh, Y. W. (2012) · Cited by 46 · see citing works
Wytock, M., & Kolter, J. Z. (2013) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.2013.6760016 · Cited by 39 · see citing works
Learning Exploration/Exploitation Strategies for Single Trajectory Reinforcement Learning
Castronovo, M., Maes, F., Fonteneau, R., et al. (2012) · Open Repository and Bibliography (University of Liège) · Cited by 32 · see citing works
Mestekemper, T., Kauermann, G., & Smith, M. S. (2012) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2012.03.003 · Cited by 30 · see citing works
Next Day Load Forecasting Using SVM
Li, X., Gong, D., Li, L., et al. (2005) · Lecture notes in computer science · doi:10.1007/11427469_101 · Cited by 25 · see citing works
Ammar, H. B., Mocanu, D. C., Taylor, M. E., et al. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40991-2_29 · Cited by 25 · see citing works
Comparison of advanced power system operations models for large-scale renewable integration
Bakirtzis, E. A., Simoglou, C. K., Biskas, P. N., et al. (2015) · Electric Power Systems Research · doi:10.1016/j.epsr.2015.06.025 · Cited by 24 · see citing works
Reinforcement Learning:State of the Art
Wiering, M. (2012) · Adaptation, learning, and optimization · doi:10.1007/978-3-642-27645-3 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- M. Krarti, Energy Audit of Building Systems: An Engineering Approach, Mechanical and Aerospace Engineering Series, 2nd ed., Taylor & Francis, 2012.
Cited by
Sorted by citations
Antonopoulos, I., Robu, V., Couraud, B., Kirli, D., Norbu, S., Kiprakis, A., Flynn, D., Elizondo-Gonzalez, S., & Wattam, S. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109899 · Cited by 656 · see citing works
On-Line Building Energy Optimization Using Deep Reinforcement Learning
Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2834219 · Cited by 649 · see citing works
Self-citationO’Dwyer, E., Pan, I., Acha, S., & Shah, N. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.01.024 · Cited by 410 · see citing works
Tackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336 · see citing works
Artificial Intelligence Enabled Demand Response: Prospects and Challenges in Smart Grid Environment
Khan, M. A., Saleh, A. M., Waseem, M., & Sajjad, I. A. (2023) · IEEE Access · doi:10.1109/access.2022.3231444 · Cited by 179 · see citing works
Deep learning versus traditional machine learning methods for aggregated energy demand prediction
Paterakis, N. G., Mocanu, E., Gibescu, M., Stappers, B., & van Alst, W. (2017) · 2017 IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe) · doi:10.1109/isgteurope.2017.8260289 · Cited by 118 · see citing works
Self-citationProcedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning
Muller-Brockhausen, M., Preuss, M., & Plaat, A. (2021) · 2021 IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619000 · Cited by 11 · see citing works
- Authors
- Mocanu, E., Nguyen, P. H., Kling, W. L., & Gibescu, M.
- Affiliations
- Eindhoven University of Technology (NL)
- Corresponding author
- Mocanu, Elena
- Venue
- Energy and Buildings
- Publisher
- Elsevier BV
- Published
- 2016-03
- Volume
- 116
- Pages
- 646-655
- ISSN
- 0378-7788 (ISSN-L)
- OpenAlex
- W2281071090
- Cited by
- 171
- References
- 45
- Citations by year
- 2026: 8 · 2025: 13 · 2024: 18 · 2023: 20 · 2022: 25 · 2021: 27 · 2020: 29 · 2019: 19 · 2018: 5 · 2017: 4 · 2016: 3
- FWCI
- 11.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Energy and Buildings (publishedVersion, cc-by)
- TU/e Research Portal (submittedVersion, other-oa)
- TU/e Research Portal (submittedVersion)
- Funders
- NL Enterprise Agency RVO.nl
- Topics
- Physical Sciences › Engineering › Building and Construction › Building Energy and Comfort Optimization
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Energy Load and Power Forecasting
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Smart Grid Energy Management
- Keywords
- building energy consumption prediction, unsupervised learning, reinforcement learning, deep belief network, smart grid, load forecasting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
