What You Know Does Not Show: Domain Knowledge and Occupant-AI Interaction in Building Energy Management
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
AI-powered assistants reveal a potential to be deployed in buildings to support energy management and occupant engagement through natural language interactions. The success of these assistants in real-world building contexts hinges not only on their technical capabilities but also on how occupants actually interact with them. Prior research in human-AI collaboration has shown that domain knowledge can influence how users interact with AI systems, allowing them to challenge and evaluate AI outputs more effectively. However, it remains unclear how this dynamic plays out in the context of building energy management, where system-of-systems complexity and occupant behavior stochasticity may pose unique challenges. This study presents a content analysis of 289 user turns from 30 participants who interacted with a large language model (LLM) assistant to identify behavioral energy-saving strategies for a real household under time-of-use pricing. Participants were stratified by self-assessed domain knowledge into high and low groups. Each turn was coded on three theory-grounded dimensions – problem representation depth, reliance calibration, and interaction initiative – using a dual-LLM coding pipeline (GPT-4.1 and Gemini 2.5 Pro) with self-consistency verification, validated against a human coder. Despite meaningful differences in self-assessed knowledge, we find no significant group differences on problem representation depth or reliance calibration (both p >.29). The only signific…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other
Mann, H. B., & Whitney, D. R. (1947) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730491 · Cited by 14,320
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“To investigate this question, we conducted a content analysis of 289 user turns from 30 participants interacting with an LLM energy advisor, coding each turn on three theory-grounded dimensions – problem representation depth [3], reliance calibration [20], and interaction initiative [11].”
Categorization and Representation of Physics Problems by Experts and Novices*
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) · Cognitive Science · doi:10.1207/s15516709cog0502_2 · Cited by 5,248
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 2×
“Automation bias has been shown to persist even among domain experts [8], and the relationship between domain knowledge and AI reliance may be non-linear rather than simply protective [23].”
Risko, E. F., & Gilbert, S. J. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.07.002 · Cited by 1,502 · cited 1×
“When the AI suggests appropriate appliances and strategies, both knowledgeable and non-knowledgeable participants may accept – i.e., “cognitive offloading” [26], rather than challenge or elaborate, thus suppressing observable differences in prompting behavior.”
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 2×
“To investigate this question, we conducted a content analysis of 289 user turns from 30 participants interacting with an LLM energy advisor, coding each turn on three theory-grounded dimensions – problem representation depth [3], reliance calibration [20], and interaction initiative [11].”
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 1,274
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · cited 2×
“Automation bias has been shown to persist even among domain experts [8], and the relationship between domain knowledge and AI reliance may be non-linear rather than simply protective [23].”
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · cited 1×
“Across multiple decision-support contexts, users with domain knowledge have been shown to be more discerning – rejecting incorrect AI recommendations that non-experts accept [7] and reducing their reliance when AI quality is low [4].”
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490
Inert knowledge: Analyses and remedies
Renkl, A., Mandl, H., & Gruber, H. (1996) · Educational Psychologist · doi:10.1207/s15326985ep3102_3 · Cited by 289
Jung, W., & Jazizadeh, F. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.01.070 · Cited by 276
Self-citationWang, D., Churchill, E., Maes, P., Fan, X., Shneiderman, B., Shi, Y., & Wang, Q. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3381069 · Cited by 271
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning
Gajos, K. Z., & Mamykina, L. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511138 · Cited by 127
I can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119
Dikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · cited 1×
“Across multiple decision-support contexts, users with domain knowledge have been shown to be more discerning – rejecting incorrect AI recommendations that non-experts accept [7] and reducing their reliance when AI quality is low [4].”
The role of domain expertise in trusting and following explainable AI decision support systems
Bayer, S., Gimpel, H., & Markgraf, M. (2022) · Journal of Decision Systems · doi:10.1080/12460125.2021.1958505 · Cited by 103
Large Language Model Interface for Home Energy Management Systems
Michelon, F., Zhou, Y., & Morstyn, T. (2025) · ACM International Conference on Future Energy Systems (e-Energy) · doi:10.1145/3679240.3734586 · Cited by 38
Makroum, R. E., Zwickl-Bernhard, S., & Kranzl, L. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.109857 · Cited by 29
Multi-agent home energy management assistant (HEMA)
Jung, W. (2026) · SoftwareX · doi:10.1016/j.softx.2026.102633 · Cited by 15
Self-citationJung, W., Babon-Ayeng, P., & Jeon, K. (2026) · E3S Web of Conferences · doi:10.1051/e3sconf/202671606001 · Cited by 5
Self-citationWhat LLM-Simulated Users Can and Cannot Tell Us About Conversational Energy Management Systems
Jung, W. (2026) · doi:10.1145/3765611.3815484 · Cited by 1
Self-citation
Research works printed but not linked: 3
- Tianzhi He and Farrokh Jazizadeh. 2024. LLM-Based Building Energy Management Assistants. In ASCE International Conference on Computing in Civil Engineering.
- Tianzhi He and Farrokh Jazizadeh. 2025. Context-aware LLM-based AI Agents for Human-centered Energy Management Systems in Smart Buildings. arXiv preprint (2025). arXiv:https://arXiv.org/abs/2512.25055. · arXiv
- Jordan Rey-Jouanchicot André Bottaro Eric Campo Jean-Léon Bouraoui Nadine Vigouroux and Frédéric Vella. 2024. Leveraging Large Language Models for Enhanced Personalised User Experience in Smart Homes. arXiv preprint (2024). arXiv:https://arXiv.org/abs/2407.12024. · arXiv
Cited by
Still being checked.
Metadata
- Authors
- Jung, W., Jeon, K., & Babon-Ayeng, P.
- Affiliations
- University of Arizona (US)
- Illinois Institute of Technology (US)
- Venue
- Proceedings of the 2026 ACM Sustainability Week
- Publisher
- ACM
- Published
- 2026-06-22
- Pages
- 501-508
- OpenAlex
- W7165379553
- Cited by
- 0
- References
- 26
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Sustainability Week (publishedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Engineering › Building and Construction › Building Energy and Comfort Optimization
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- domain knowledge, building energy management, AI assistants, natural language interfaces, human-AI collaboration, occupant engagement, occupant behavior, building energy systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.