The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning
Open accessAbstract
Heating, ventilation, and air conditioning (HVAC) are extremely energy consuming, accounting for 40% of total building energy consumption. It is crucial to design some energy-efficient building thermal comfort control strategy which can reduce the energy consumption of the HVAC while maintaining the comfort of the occupants. However, implementing such a strategy is challenging, because the changes of the thermal states in a building environment are influenced by various factors. The relationships among these influencing factors are hard to model and are always different in different building environments. To address this challenge, we propose a deep-reinforcement-learning-based framework, DeepComfort, for thermal comfort control in buildings. We formulate the thermal comfort control as a cost-minimization problem by jointly considering the energy consumption of the HVAC and the occupants' thermal comfort. We first design a deep feedforward neural network (FNN)-based approach for predicting the occupants' thermal comfort and then propose a deep deterministic policy gradients (DDPGs)-based approach for learning the optimal thermal comfort control policy. We implement a building thermal comfort control simulation environment and evaluate the performance under various settings. The experimental results show that our approaches can improve the performance of thermal comfort prediction by 14.5% and reduce the energy consumption of HVAC by 4.31% while improving the occupants' thermal comfort by 13.6%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Training feedforward networks with the Marquardt algorithm
Hagan, M. T., & Menhaj, M. B. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.329697 · Cited by 7,701
A review on buildings energy consumption information
Pérez-Lombard, L., Ortiz, J., & Pout, C. (2008) · Energy and Buildings · doi:10.1016/j.enbuild.2007.03.007 · Cited by 6,649 · cited 1×
“One main concern of the HVAC system is the high energy consumption, which makes building energy consumption account for 20%–40% of the total energy consumption [1].”
BackgroundOn the Theory of the Brownian Motion
Uhlenbeck, G. E., & Ornstein, L. S. (1930) · Physical Review · doi:10.1103/physrev.36.823 · Cited by 4,482
Thermal comfort: Analysis and applications in environmental engineering
(1972) · Applied Ergonomics · doi:10.1016/s0003-6870(72)80074-7 · Cited by 4,451
Deterministic Policy Gradient Algorithms
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014) · HAL · Cited by 3,275
Gauss-Newton approximation to Bayesian learning
Dan Foresee, F., & Hagan, M. T. (2002) · Proceedings of International Conference on Neural Networks (ICNN'97) · doi:10.1109/icnn.1997.614194 · Cited by 1,390
Minoli, D., Sohraby, K., & Occhiogrosso, B. (2017) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2647881 · Cited by 874
Dounis, A. I., & Caraiscos, C. (2009) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2008.09.015 · Cited by 851
Heating, Ventilating, and Air Conditioning: Analysis and Design
McQuiston, F. C., & Parker, J. D. (1977) · Cited by 742
Reinforcement Learning and Markov Decision Processes
van Otterlo, M., & Wiering, M. (2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3_1 · Cited by 628
Model Predictive Control for the Operation of Building Cooling Systems
Ma, Y., Borrelli, F., Hencey, B., Coffey, B., Bengea, S., & Haves, P. (2012) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2011.2124461 · Cited by 527
Deep Reinforcement Learning for Building HVAC Control
Wei, T., Wang, Y., & Zhu, Q. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062224 · Cited by 478
Deep Reinforcement Learning for Smart Home Energy Management
Yu, L., Xie, W., Xie, D., Zou, Y., Zhang, D., Sun, Z., Zhang, L., Zhang, Y., & Jiang, T. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2957289 · Cited by 471
Oldewurtel, F., Parisio, A., Jones, C. N., Morari, M., Gyalistras, D., Gwerder, M., Stauch, V, Lehmann, B., & Wirth, K. (2010) · Proceedings of the 2010 American Control Conference · doi:10.1109/acc.2010.5530680 · Cited by 435
An Internet of Things Framework for Smart Energy in Buildings: Designs, Prototype, and Experiments
Pan, J., Jain, R., Paul, S., Vu, T., Saifullah, A., & Sha, M. (2015) · IEEE Internet of Things Journal · doi:10.1109/jiot.2015.2413397 · Cited by 367
Global database of thermal comfort field experiments
de Dear, R. (1998) · ASHRAE winter conference papers · Cited by 320
Model-based optimal control of VAV air-conditioning system using genetic algorithm
Wang, S., & Jin, X. (2000) · Building and Environment · doi:10.1016/s0360-1323(99)00032-3 · Cited by 282
Corgnati, S. P., Filippi, M., & Viazzo, S. (2005) · Building and Environment · doi:10.1016/j.buildenv.2005.10.027 · Cited by 262
Thermal comfort models: A review and numerical investigation
Cheng, Y., Niu, J., & Gao, N. (2012) · Building and Environment · doi:10.1016/j.buildenv.2011.05.011 · Cited by 258
The control of indoor thermal comfort conditions: introducing a fuzzy adaptive controller
Calvino, F., Gennusa, M. L., Rizzo, G., & Scaccianoce, G. (2003) · Energy and Buildings · doi:10.1016/j.enbuild.2003.10.004 · Cited by 221
Reinforcement learning for energy conservation and comfort in buildings
Dalamagkidis, K., Kolokotsa, D. D., Kalaitzakis, K. C., & Stavrakakis, G. S. (2006) · Building and Environment · doi:10.1016/j.buildenv.2006.07.010 · Cited by 203
Wang, Y., Velswamy, K., & Huang, B. (2017) · Processes · doi:10.3390/pr5030046 · Cited by 157
Autonomous HVAC Control, A Reinforcement Learning Approach
Barrett, E., & Linder, S. P. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23461-8_1 · Cited by 135
Model-Based Hierarchical Optimal Control Design for HVAC Systems
Maasoumy, M., Pinto, A., & Sangiovanni-Vincentelli, A. L. (2011) · doi:10.1115/dscc2011-6078 · Cited by 106
DeepQoE: A Multimodal Learning Framework for Video Quality of Experience (QoE) Prediction
Zhang, H., Dong, L., Gao, G., Hu, H., Wen, Y., & Guan, K. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2973828 · Cited by 94
Self-citationUsing reinforcement learning to optimize occupant comfort and energy usage in HVAC systems
Fazenda, P., Veeramachaneni, K. K., Lima, P. U., & O’Reilly, U.-M. (2014) · Journal of Ambient Intelligence and Smart Environments · doi:10.3233/ais-140288 · Cited by 92
Optimal heating control in a passive solar commercial building
Kummert, M., André, P., & Nicolas, J. (2001) · Solar Energy · doi:10.1016/s0038-092x(01)00038-x · Cited by 84
A multi-grid reinforcement learning method for energy conservation and comfort of HVAC in buildings
Li, B., & Xia, L. (2015) · 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2015.7294119 · Cited by 74
iTCM: Toward Learning-Based Thermal Comfort Modeling via Pervasive Sensing for Smart Buildings
Hu, W., Wen, Y., Guan, K., Jin, G., & Tseng, K. J. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2861831 · Cited by 72
Self-citationComparison of Conventional and Fuzzy Control of Indoor Air Quality in Buildings
Dounis, A. I., Bruant, M., Santamouris, M., et al. (1996) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/ifs-1996-4205 · Cited by 63
Shepherd, A., & Batty, W. J. (2003) · Building Services Engineering Research and Technology · doi:10.1191/0143624403bt059oa · Cited by 52
Co-scheduling of HVAC control, EV charging and battery usage for building energy efficiency
Wei, T., Zhu, Q., & Maasoumy, M. (2014) · International Conference on Computer Aided Design · doi:10.1109/iccad.2014.7001351 · Cited by 49
Building energy management systems : application to heating and control
Levermore, G. J. (1992) · E. & F.N. Spon eBooks · Cited by 40
Content-Aware Personalised Rate Adaptation for Adaptive Streaming via Deep Video Analysis
Gao, G., Dong, L., Zhang, H., Wen, Y., & Zeng, W. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761156 · Cited by 34
Self-citationRobust Reinforcement Learning for Heating, Ventilation, and Air Conditioning Control of Buildings
Si, J., Barto, A. G., Powell, W. B., et al. (2009) · Institute of Electrical and Electronics Engineers eBooks · doi:10.1109/9780470544785.ch20 · Cited by 12
Research works printed but not linked: 10
- 10.1109/IOT.2010.5678444 · doi:10.1109/iot.2010.5678444
- nikovski, 2013, A method for computing optimal set-point schedules for HVAC systems, Proc 11th REHVA World Congr Clima, 1
- 10.1007/978-3-642-40942-4_30 · doi:10.1007/978-3-642-40942-4_30
- 1992, ASHRAE STANDARD An American Standard Thermal Environmental Conditions for Human Occupancy
- 2018, MySQL The World s Most Popular Open Source Database
- 2019, Type 155 TRNSYS-MATLAB Link
- dong, 2019, Baconian A unified opensource framework for model-based reinforcement learning
- 10.1109/TMM.2016.2635019 · doi:10.1109/tmm.2016.2635019
- 2018, PyTorch An open source deep learning platform
- 2011, TRNSYS Transient System Simulation Tool
Cited by
Himeur, Y., Elnour, M., Fadli, F., Meskin, N., Petri, I., Rezgui, Y., Bensaali, F., & Amira, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10286-2 · Cited by 541
A Review of Deep Reinforcement Learning for Smart Building Energy Management
Yu, L., Qin, S., Zhang, M., Shen, C., Jiang, T., & Guan, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078462 · Cited by 476
Chen, X., Qu, G., Tang, Y., Low, S., & Li, N. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3154718 · Cited by 415
Wu, Y., Dai, H.-N., Wang, H., Xiong, Z., & Guo, S. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3158270 · Cited by 323
Deep Reinforcement Learning for Internet of Things: A Comprehensive Survey
Chen, W., Qiu, X., Cai, T., Dai, H.-N., Zheng, Z., & Zhang, Y. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3073036 · Cited by 311
Metadata
- Authors
- Gao, G., Li, J., & Wen, Y.
- Venue
- IEEE Internet of Things Journal
- Published
- 2020-09
- Volume
- 7
- Issue
- 9
- Pages
- 8472-8484
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W3023669592
- Cited by
- 283
- References
- 37
- FWCI
- 17.96
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- Project Fund from DSAIR@NTU, and a BSEWWT Project Fund from National Research Foundation Singapore, administrated through the BSEWWT Program Office (BSEWWT2017_2_06)
- Green Buildings Innovation Cluster (NRF2015ENC-GBICRD001-012)
- Building and Construction Authority Singapore
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.