The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings
Open accessAbstract
In commercial buildings, about 40%-50% of the total electricity consumption is attributed to Heating, Ventilation, and Air Conditioning (HVAC) systems, which places an economic burden on building operators. In this paper, we intend to minimize the energy cost of an HVAC system in a multi-zone commercial building with the consideration of random zone occupancy, thermal comfort, and indoor air quality comfort. Due to the existence of unknown thermal dynamics models, parameter uncertainties (e.g., outdoor temperature, electricity price, and number of occupants), spatially and temporally coupled constraints associated with indoor temperature and CO2 concentration, a large discrete solution space, and a non-convex and non-separable objective function, it is very challenging to achieve the above aim. To this end, the above energy cost minimization problem is reformulated as a Markov game. Then, an HVAC control algorithm is proposed to solve the Markov game based on multi-agent deep reinforcement learning with attention mechanism. The proposed algorithm does not require any prior knowledge of uncertain parameters and can operate without knowing building thermal dynamics models. Simulation results based on real-world traces show the effectiveness, robustness and scalability of the proposed algorithm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
Markov games as a framework for multi-agent reinforcement learning
Littman, M. L. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50027-1 · Cited by 2,614
Reinforcement learning for demand response: A review of algorithms and modeling techniques
Vázquez-Canteli, J. R., & Nagy, Z. (2019) · Applied Energy · doi:10.1016/j.apenergy.2018.11.002 · Cited by 883
An Optimized EV Charging Model Considering TOU Price and SOC Curve
Cao, Y., Tang, S., Li, C., Zhang, P., Tan, Y., Zhang, Z., & Li, J. (2012) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2011.2159630 · Cited by 719
On-Line Building Energy Optimization Using Deep Reinforcement Learning
Mocanu, E., Mocanu, D. C., Nguyen, P. H., Liotta, A., Webber, M. E., Gibescu, M., & Slootweg, J. G. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2834219 · Cited by 649
Demand Response Optimization for Smart Home Scheduling Under Real-Time Pricing
Tsui, K. M., & Chan, S. C. (2012) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2012.2218835 · Cited by 526
Deep Reinforcement Learning for Building HVAC Control
Wei, T., Wang, Y., & Zhu, Q. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062224 · Cited by 478
Deep Reinforcement Learning for Smart Home Energy Management
Yu, L., Xie, W., Xie, D., Zou, Y., Zhang, D., Sun, Z., Zhang, L., Zhang, Y., & Jiang, T. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2957289 · Cited by 471
Self-citationLu, R., Hong, S. H., & Yu, M. (2019) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2019.2909266 · Cited by 389
Zhang, Z., Chong, A., Pan, Y., Zhang, C., & Lam, K. P. (2019) · Energy and Buildings · doi:10.1016/j.enbuild.2019.07.029 · Cited by 339
Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning
Ruelens, F., Claessens, B. J., Vandael, S., De Schutter, B., Babuska, R., & Belmans, R. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2016.2517211 · Cited by 333
A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning
Wang, S., Duan, J., Shi, D., Xu, C., Li, H., Diao, R., & Wang, Z. (2020) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2020.2990179 · Cited by 315
Transactive Control of Commercial Buildings for Demand Response
Hao, H., Corbin, C. D., Kalsi, K., & Pratt, R. G. (2017) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2016.2559485 · Cited by 285
Actor-Attention-Critic for Multi-Agent Reinforcement Learning
Iqbal, S., & Sha, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02912 · Cited by 280
Wang, L., Wang, Z., & Yang, R. (2012) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2011.2178044 · Cited by 235
Stochastic Model Predictive Control for Building HVAC Systems: Complexity and Conservatism
Ma, Y., Matusko, J., & Borrelli, F. (2015) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2014.2313736 · Cited by 233
Real-Time Residential Demand Response
Li, H., Wan, Z., & He, H. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2978061 · Cited by 205
Zou, Z., Yu, X., & Ergan, S. (2020) · Building and Environment · doi:10.1016/j.buildenv.2019.106535 · Cited by 188
A review of reinforcement learning methodologies for controlling occupant comfort in buildings
Han, M., May, R., Zhang, X., Wang, X., Pan, S., Yan, D., Jin, Y., & Xu, L. (2019) · Sustainable Cities and Society · doi:10.1016/j.scs.2019.101748 · Cited by 162
Online Energy Management for a Sustainable Smart Home With an HVAC Load and Random Occupancy
Yu, L., Jiang, T., & Zou, Y. (2017) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2017.2775209 · Cited by 153
Self-citationChen, B., Cai, Z., & Berges, M. E. (2019) · ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys) · doi:10.1145/3360322.3360849 · Cited by 147
Temperature Control of a Commercial Building With Model Predictive Control Techniques
Mantovani, G., & Ferrarini, L. (2014) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2014.2387095 · Cited by 141
Kim, Y. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2790429 · Cited by 100
Yu, L., Xie, D., Huang, C., Jiang, T., & Zou, Y. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2875727 · Cited by 92
Self-citationAI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCs
Lan, T., Duan, J., Zhang, B., Shi, D., Wang, Z., Diao, R., & Zhang, X. (2020) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm41954.2020.9281518 · Cited by 74
Optimal Control of Multiroom HVAC System: An Event-Based Approach
Wu, Z., Jia, Q., & Guan, X. (2015) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2015.2446955 · Cited by 73
Self-citationEnergy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement LearningPreprint
Gao, G., Li, J., & Wen, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.04693 · Cited by 71
Xu, Z., Liu, S., Hu, G., & Spanos, C. J. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.02.051 · Cited by 47
Self-citationYoon, A.-Y., Kim, Y., & Moon, S. (2018) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2018.2883701 · Cited by 34
Cited by
Javed, A. R., Shahzad, F., Rehman, S. U., Zikria, Y. B., Razzak, I., Jalil, Z., & Xu, G. (2022) · Cities · doi:10.1016/j.cities.2022.103794 · Cited by 606
A Review of Deep Reinforcement Learning for Smart Building Energy Management
Yu, L., Qin, S., Zhang, M., Shen, C., Jiang, T., & Guan, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078462 · Cited by 476
Self-citationReinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review
Cao, D., Hu, W., Zhao, J., Zhang, G., Zhang, B., Liu, Z., Chen, Z., & Blaabjerg, F. (2020) · Journal of Modern Power Systems and Clean Energy · doi:10.35833/mpce.2020.000552 · Cited by 460
Chen, X., Qu, G., Tang, Y., Low, S., & Li, N. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3154718 · Cited by 415
Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning
Du, Y., Zandi, H., Kotevska, O., Kurte, K., Munk, J., Amasyali, K., Mckee, E., & Li, F. (2021) · Applied Energy · doi:10.1016/j.apenergy.2020.116117 · Cited by 268
Metadata
- Authors
- Yu, L., Sun, Y., Xu, Z., Shen, C., Yue, D., Jiang, T., & Guan, X.
- Venue
- IEEE Transactions on Smart Grid
- Published
- 2021-01
- Volume
- 12
- Issue
- 1
- Pages
- 407-419
- ISSN
- 1949-3053 (ISSN-L)
- 1949-3061
- OpenAlex
- W3037631072
- Cited by
- 353
- References
- 31
- FWCI
- 22.65
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.