The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Optimal control of HVAC and window systems for natural ventilation through reinforcement learning
Abstract
Abstract Natural ventilation is a green building strategy that improves building energy efficiency, indoor thermal environment, and air quality. However, in practice, it is not always clear when and how to utilize the natural ventilation and coordinate its operation with the HVAC system. This paper introduces a reinforcement learning control strategy, specifically through model-free Q-learning, that makes optimal control decisions for HVAC and window systems to minimize both energy consumption and thermal discomfort. This control system evaluates the outdoor and indoor environments (temperature, humidity, solar radiation, and wind speed) at each time step, and responds with the best control decision that targets both immediate and long-term goals. The reinforcement learning control is evaluated through numerical simulation on a building thermal model and compared with a rule-based heuristic control strategy. Case studies in hot-and-humid Miami and warm-and-mild Los Angeles demonstrated the superior performance of reinforcement learning control, which led to 13% and 23% lower HVAC system energy consumption, 62% and 80% lower discomfort degree hours, and 63% and 77% fewer high humidity hours compared to heuristic control. Unlike heuristic control that requires specific knowledge of individual buildings and the creation of exhaustive decision-making scenarios to improve performance, reinforcement learning control guarantees optimality through self-advancement on given goals and cost functions and is able to adapt to stochastic occupancy and occupant behaviors, which is difficult to accommodate by heuristic control.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Afram, A., Janabi-Sharifi, F., Fung, A. S., & Raahemifar, K. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.02.012 · Cited by 655
Review of building energy modeling for control and operation
Li, X., & Wen, J. (2014) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2014.05.056 · Cited by 629
van Hooff, T., & Blocken, B. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2009.07.008 · Cited by 427
Predictive controllers for thermal comfort optimization and energy savings
Freire, R. Z., Oliveira, G. H. C., & Mendes, N. (2008) · Energy and Buildings · doi:10.1016/j.enbuild.2007.12.007 · Cited by 362
Design analysis of single-sided natural ventilation
Allocca, C., Chen, Q., & Glicksman, L. R. (2003) · Energy and Buildings · doi:10.1016/s0378-7788(02)00239-6 · Cited by 292
Energy saving potential of natural ventilation in China: The impact of ambient air pollution
Tong, Z., Chen, Y., Malkawi, A., Liu, Z., & Freeman, R. B. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.07.019 · Cited by 285
Self-citationRoadside vegetation barrier designs to mitigate near-road air pollution impacts
Tong, Z., Baldauf, R. W., Isakov, V., Deshmukh, P., & Max Zhang, K. (2016) · Science of The Total Environment · doi:10.1016/j.scitotenv.2015.09.067 · Cited by 280
Investigating natural ventilation potentials across the globe: Regional and climatic variations
Chen, Y., Tong, Z., & Malkawi, A. (2017) · Building and Environment · doi:10.1016/j.buildenv.2017.06.026 · Cited by 225
Self-citationTong, Z., Chen, Y., Malkawi, A., Adamkiewicz, G., & Spengler, J. D. (2016) · Environment International · doi:10.1016/j.envint.2016.01.016 · Cited by 222
Self-citationPredictive control techniques for energy and indoor environmental quality management in buildings
Kolokotsa, D. D., Pouliezos, A. D., Stavrakakis, G. M., & Lazos, C. (2008) · Building and Environment · doi:10.1016/j.buildenv.2008.12.007 · Cited by 217
Reinforcement Learning Versus Model Predictive Control: A Comparison on a Power System Problem
Ernst, D., Glavic, M., Capitanescu, F., & Wehenkel, L. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007630 · Cited by 214
Reinforcement learning for energy conservation and comfort in buildings
Dalamagkidis, K., Kolokotsa, D. D., Kalaitzakis, K. C., & Stavrakakis, G. S. (2006) · Building and Environment · doi:10.1016/j.buildenv.2006.07.010 · Cited by 203
04/00989 Design analysis of single-sided natural ventilation
(2004) · Fuel and Energy Abstracts · doi:10.1016/s0140-6701(04)93371-1 · Cited by 199
Review of the impact of urban block form on thermal performance, solar access and ventilation
Sanaieian, H., Tenpierik, M., Linden, K. V. D., Mehdizadeh Seraj, F., & Mofidi Shemrani, S. M. (2014) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2014.06.007 · Cited by 187
Reinforcement learning for optimal control of low exergy buildings
Yang, L., Nagy, Z., Goffin, P., & Schlueter, A. (2015) · Applied Energy · doi:10.1016/j.apenergy.2015.07.050 · Cited by 182
Quantifying the effect of vegetation on near-road air quality using brief campaigns
Tong, Z., Whitlow, T. H., MacRae, P. F., Landers, A. J., & Harada, Y. (2015) · Environmental Pollution · doi:10.1016/j.envpol.2015.02.026 · Cited by 181
Parametric energy simulation in early design: High-rise residential buildings in urban contexts
Samuelson, H., Claussnitzer, S., Goyal, A., Chen, Y., & Romo-Castillo, A. (2016) · Building and Environment · doi:10.1016/j.buildenv.2016.02.018 · Cited by 172
Self-citationGao, C. F., & Lee, W. L. (2010) · Building and Environment · doi:10.1016/j.buildenv.2010.10.029 · Cited by 155
Natural ventilation design for houses in Thailand
Tantasavasdi, C., Srebric, J., & Chen, Q. (2001) · Energy and Buildings · doi:10.1016/s0378-7788(01)00073-1 · Cited by 154
Oropeza-Perez, I., & Østergaard, P. A. (2014) · Applied Energy · doi:10.1016/j.apenergy.2014.05.035 · Cited by 148
Tong, Z., Chen, Y., & Malkawi, A. M. (2017) · Applied Energy · doi:10.1016/j.apenergy.2017.02.041 · Cited by 135
Self-citationHirst, N. A. C. (2013) · Springer eBooks · doi:10.1007/978-1-4471-4781-7_2 · Cited by 133
Satisfaction based Q-learning for integrated lighting and blind control
Cheng, Z., Zhao, Q., Wang, F., Jiang, Y., Xia, L., & Ding, J. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.05.067 · Cited by 132
Liu, S., & Henze, G. P. (2005) · Energy and Buildings · doi:10.1016/j.enbuild.2005.06.001 · Cited by 124
Liu, S., & Henze, G. P. (2005) · Energy and Buildings · doi:10.1016/j.enbuild.2005.06.002 · Cited by 120
Passive cooling in a low-energy office building
Breesch, H., Bossaer, A., & Janssens, A. (2005) · Solar Energy · doi:10.1016/j.solener.2004.12.002 · Cited by 117
Yuan, S., & Perez, R. A. (2006) · Energy and Buildings · doi:10.1016/j.enbuild.2006.03.007 · Cited by 111
Short-term building energy model recommendation system: A meta-learning approach
Cui, C., Wu, T., Hu, M., Weir, J. D., & Li, X. (2016) · Applied Energy · doi:10.1016/j.apenergy.2016.03.112 · Cited by 110
Integrated design workflow and a new tool for urban rainwater management
Chen, Y., Samuelson, H. W., & Tong, Z. (2016) · Journal of Environmental Management · doi:10.1016/j.jenvman.2016.04.059 · Cited by 107
Self-citationBuilding energy consumption on-line forecasting using physics based system identification
Li, X., & Wen, J. (2014) · Energy and Buildings · doi:10.1016/j.enbuild.2014.07.021 · Cited by 105
Natural ventilation in high-rise buildings with double facades, saving or waste of energy
Pasquay, T. (2004) · Energy and Buildings · doi:10.1016/j.enbuild.2004.01.018 · Cited by 100
A recommendation system for meta-modeling: A meta-learning based approach
Cui, C., Hu, M., Weir, J. D., & Wu, T. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.10.021 · Cited by 76
Homod, R. Z., Sahari, K. S. M., & Almurib, H. A. F. (2014) · Renewable Energy · doi:10.1016/j.renene.2014.06.015 · Cited by 75
A case study of air quality above an urban roof top vegetable farm
Tong, Z., Whitlow, T. H., Landers, A., et al. (2015) · Environmental Pollution · doi:10.1016/j.envpol.2015.07.006 · Cited by 71
Laurent, J. G. C., Samuelson, H. W., & Chen, Y. (2017) · Building Simulation · doi:10.1007/s12273-017-0399-3 · Cited by 43
Self-citationKoinakis, C. J. (2004) · Energy and Buildings · doi:10.1016/j.enbuild.2004.06.022 · Cited by 29
Li, X., & Wen, J. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.08.014 · Cited by 22
Cited by
Reinforcement learning for demand response: A review of algorithms and modeling techniques
Vázquez-Canteli, J. R., & Nagy, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.11.002 · Cited by 883
Reinforcement learning for building controls: The opportunities and challenges
Wang, Z., & Hong, T. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115036 · Cited by 577
Himeur, Y., Elnour, M., Fadli, F., Meskin, N., Petri, I., Rezgui, Y., Bensaali, F., & Amira, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10286-2 · Cited by 541
Applications of reinforcement learning in energy systems
Perera, A. T. D., & Kamalaruban, P. (2021) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.110618 · Cited by 420
Zhang, Z., Chong, A., Pan, Y., Zhang, C., & Lam, K. P. (2019) · Energy and Buildings · doi:10.1016/j.enbuild.2019.07.029 · Cited by 339
Metadata
- Authors
- Chen, Y., Norford, L. K., Samuelson, H. W., & Malkawi, A.
- Venue
- Energy and Buildings
- Publisher
- Elsevier BV
- Published
- 2018-06
- Volume
- 169
- Pages
- 195-205
- ISSN
- 0378-7788 (ISSN-L)
- OpenAlex
- W2795237036
- Cited by
- 289
- References
- 37
- FWCI
- 22.50
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.