A review of LLMs and their applications in the architecture, engineering and construction industry
Open accessThe DOI matches its Crossref record; 165 of the 165 references it cites were found in a registry or an index; 61 of the 115 sources its claims cite were checked for support.
Abstract
During the past decade, there has been rapid emergence, continuous development and advancements in the field of Artificial Intelligence (AI), and a broad adaptation ofLarge Language Models (LLMs) in a wide variety of application domains transforming and streamlining industry practices. However, the construction industry has yet to fully incorporate these technologies, delaying their wide-scale adaptation. Only a limited number of recent studies have explored the opportunities, capabilities and potential of current LLM implementations in the broad domain of Architecture Engineering and Construction (AEC) industry, leaving a significant gap in this field of research. This study aims to address this gap and provide an extensive review of already established state-of-the-art applications and use case scenarios of LLMs in the AEC industry. Apart from that, by exploring the key contributions and limitations of these applications, and by considering relative reviews on this subject, it was possible to categorize them, to extract the emerging challenges and future directions of the field and propose actionable recommendations for industry stakeholders. This study also includes an introduction to important concepts and recent advancements of LLM technologies, focusing on transformer-based architectures and providing an extensive list of LLM families.
Overview
CoChat Summary
TL;DR
This review surveys LLM applications in the architecture, engineering, and construction (AEC) industry, categorizing 28 established use cases from 2021–2024 literature and outlining challenges and recommendations. It finds adoption is still limited by domain-specific knowledge gaps, prompt sensitivity, computational costs, interoperability, and ethical/regulatory concerns.
Key findings
- From 927 Scopus and 437 Web of Science records, after filtering, merging, and screening, 28 studies presenting AEC LLM applications were reviewed; use cases span seven categories: training/education/literature analysis; planning/scheduling; safety/hazard recognition; document generation/compliance; virtual assistants/QA; code/data generation/interpretation/visualization; and BIM/BEM.
- Case studies report both promise and reliability issues: Uddin et al. found over 25% increase in hazards detected with ChatGPT but almost 40% remained undetected; Hassan et al. used 5,845 OSHA injury cases; Moon et al. used 2,807 clauses from 56 documents; Jang et al. reported 83% accuracy for exterior wall detailing; Ouyang and Srivastava’s LLMSense achieved over 80% on occupancy tracking.
- Recurring limitations across applications include lack of construction-specific knowledge, prompt sensitivity, computational/resource demands, data interoperability, need for human oversight, and ethical/privacy/security concerns.
- Recommendations for stakeholders include selecting appropriate LLM variants, domain-specific fine-tuning, integrating with BIM/digital tools via standardized APIs, improving computational efficiency, and implementing validation frameworks with human oversight.
Methods
- Literature review using Scopus and Web of Science; search combined LLM-related and AEC-related keyword sets, limited to English research articles and conference papers published 2021–2024.
- Identification yielded 927 Scopus and 437 WoS records; after filtering: 510 Scopus and 266 WoS; after duplicate removal: 776; title screening eliminated 494 and abstract screening 113, leaving 95; full-text eligibility selected 28 studies with use cases/applications.
- Relevant review articles were also considered to establish challenges, limitations, future directions, recommendations, and categorization of use cases.
Limitations
- Stated: the review focuses on transformer-based architectures mainly for text generation, not audio, image, or video generation; because the focus is AEC, the analysis of LLM state-of-the-art implementations and techniques is not extensive and is limited to basic concepts and trends.
- Stated/evident: the final review set is only 28 studies, and the search is restricted to Scopus/WoS, English, and 2021–2024, so it may miss other databases, languages, or pre-2021 work.
- Evident: as a literature review, it does not generate or analyze new datasets (the paper states “No datasets were generated or analysed during the current study”) and does not quantitatively meta-analyze the reviewed applications.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
55 Valid · 4 Not valid · 48 Not checked · 14 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While their work is comprehensive and a significant contribution to the field, they present only six already established applications of LLMs in construction, not addressing the recent adoption of this technology.”
What the sources say
- Partly supportedcites Taiwo et al. 2024 sourceExperimental — accuracy not yet measured
Section 4.1 (page unavailable) says the review identified six peer-reviewed articles on LLM uses in construction, not that there are only six established applications; it describes the applications as emerging. It also discusses adoption, stating that it remains in the very early stages, while Section 4.3 (page unavailable) says adoption is growing. Thus the claim's numerical basis is present, but its characterization of the applications and assertion that the paper does not address adoption are not supported. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Saka et al. ( 2023a ) did a holistic review of GPT models in the construction industry, focusing on opportunities and limitations in all stages of construction.”
What the sources say
- Supportedcites Saka et al. 2023a source
The Introduction states that the study critically reviews opportunities and limitations of GPT models in the AEC industry. The Abstract says it finds opportunities throughout the project lifecycle, and Section 4.2 organizes them by lifecycle phase. This supports the citing sentence’s description of a broad review covering construction stages. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“the lack of construction-specific knowledge of the models often resulted in inaccuracies or incomplete domain specifications.”
What the sources say
- Partly supportedcites Guan et al. (2023) sourceExperimental — accuracy not yet measured
Section 5.1 reports factual errors in generated PDDL domain models, including missing preconditions and effects, and suggests GPT-4 may have an inaccurate understanding of actions. It does not identify a lack of construction-specific knowledge as the cause or report findings about the construction industry; here, “construction” refers to constructing PDDL models. Page numbers are unavailable in our copy. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The results demonstrated an over 25% increase in the detected hazards when students utilized ChatGPT.”
What the sources say
- Supportedcites Uddin et al. (2023) source
The Conclusions (p. 17) state that using ChatGPT as an intervention yielded hazard-recognition gains exceeding 25%. The same section describes a pre-intervention assessment and a subsequent assessment of construction students, matching the claimed comparison. (LLM, conf=0.94)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Prompt sensitivity is another limitation”
What the sources say
- Partly supportedcites Prieto et al. (2023) sourceExperimental — accuracy not yet measured
The Methodology (Section 3) says the study evaluated consistency under slight prompt modifications. The Results and discussion (Section 5) show that ChatGPT changed its schedule when the prompt added electrical work, but present this as adaptability, not as a limitation caused by prompt sensitivity. (LLM, conf=0.78)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
25 more claims · 8 with their cited sources checked
Create a free account to see all 121 claims and every source verdict.
- Probably not a substantial claim. 43
- It may state two separate claims as one. 70
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.3 Current LLM families
“117M in GPT-2”
§ 3.3 Current LLM families
“175B in GPT-2 and GPT-3.”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751 · cited 1×
“Another promising architecture that is related to transformers is the State Space Models (SSM), a configuration that can be interpreted as a combination of RNNs and CNNs that drew inspiration from classical state space models (Kalman 1960 ).”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 9×
“Their framework was tested in three scenarios: a household robot, Tyreworld and Logistics, and exhibited better performance with fewer errors than GPT-3.”
InfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453 · cited 1×
“…involves decomposing weight matrices into smaller, lower dimension ones, requiring again the adjustment of a small number of parameters for tuning (Hu et al. 2021 ), and Retrieval-Augmented Generation (RAG), a post-training approach that involves requesting additional data related to the input…”
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“…exponentially more computational resources with performance improving at a gradually decreasing rate when dataset and model size remain fixed (Kaplan et al. 2020 ), with Sorscher et al. ( 2022 ) highlighting that for a reduction of 2% in error, orders of magni - tude more energy consumption…”
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1×
“As a result, the outputs may contain social and cultural preju - dices making the model behavior biased and less neutral (Bender et al. 2021 ).”
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 1×
“…(Wei et al. 2021 ), Fine-tuning, a post-training process that involves using a smaller and more focused dataset targeting a specific domain or task (Howard and Ruder 2018), and Prompt-tuning, a post-training method that focuses on providing opti - mized and specifically structured prompts to make…”
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · cited 1×
“…of a small number of parameters for tuning (Hu et al. 2021 ), and Retrieval-Augmented Generation (RAG), a post-training approach that involves requesting additional data related to the input from specified data - bases during inference (Lewis et al. 2020; Arora et al. 2023 ; Borgeaud et al. 2021 ).”
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · cited 1×
“…dataset targeting a specific domain or task (Howard and Ruder 2018), and Prompt-tuning, a post-training method that focuses on providing opti - mized and specifically structured prompts to make the model produce more accurate outputs without altering its parameter values (Lester et al. 2021 ).”
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 2×
“…based on their position in a sentence, an important factor for translation, text generation in GPT-3 (Brown et al. 2020 ), Relative bias used in T5 (Raffel et al. 2019 ), Rotary used in PALM (Chowdhery et al. 2022 ) and LLaMA (Touvron et al. 2023a ) and ALiBi used in BLOOM (Scao et al. 2022 ).”
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Gu, A., & Dao, T. (2023) · arXiv · doi:10.48550/arxiv.2312.00752 · Cited by 1,579
Multilingual Denoising Pre-training for Neural Machine Translation
Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., & Zettlemoyer, L. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00343 · Cited by 1,527
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436
Big Bird: Transformers for Longer Sequences
Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., Yang, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2007.14062 · Cited by 1,349
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2023) · arXiv · doi:10.48550/arxiv.2307.06435 · Cited by 1,259
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257 · cited 1×
“…Raaian et al. (2024) who provided an overview of transformer-based LLMs—exam - ining their history, architecture, and performance across various domains—and Naveed et al. ( 2023 ) who focused on technical aspects, performance differences, and fine-tuning strat - egies across multiple LLM families.”
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · cited 1×
“…technique that involves training LLMs to follow human instructions by exposing it to specific prompts that provide task-specific guidance (Wei et al. 2021 ), Fine-tuning, a post-training process that involves using a smaller and more focused dataset targeting a specific domain or task…”
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · cited 1×
“LLM agents demonstrated promising per - formance on a variety of tasks, including code generation (Shinn et al. 2023 ), autopilot systems (Jin et al. 2023 ), complex tasks in gaming environment (Wang et al. 2023 ) and more.”
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · cited 1×
“Reinforcement Learning from Human Feedback (RLHF) is also a post-training method that involves rewarding or penal - izing desirable or unwanted outputs respectively by employing human input on the model’s responses (Christiano et al. 2017 ).”
The construction industry as a loosely coupled system: implications for productivity and innovation
Dubois, A., & Gadde, L.-E. (2002) · Construction Management and Economics · doi:10.1080/01446190210163543 · Cited by 1,041 · cited 1×
“This can be attributed to the relatively recent development and growth of LLMs leaving not much time for research - ers to fully adopt them in all fields, but also on the fact that the construction industry is characterized by slow adaptation to new technologies (Dubois and Gadde 2002 ) (Table 1 ).”
Abioye, S. O., Oyedele, L. O., Akanbi, L., Ajayi, A., Davila Delgado, J. M., Bilal, M., Akinade, O. O., & Ahmed, A. (2021) · Journal of Building Engineering · doi:10.1016/j.jobe.2021.103299 · Cited by 1,015
Baduge, S. K., Thilakarathna, S., Perera, J. S., Arashpour, M., Sharafi, P., Teodosio, B., Shringi, A., & Mendis, P. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104440 · Cited by 1,007 · cited 1×
“…AI and its subdomains are successfully employed in a variety of industry related tasks, including cost prediction (Wil - liams and Gong 2014 ; Baduge et al. 2022 ), safety management (Poh et al. 2018; Liu et al. 2022 ), schedule optimization (Zhang et al. 2019; Hatami et al. 2022 ), progress…”
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Fedus, W., Zoph, B., & Shazeer, N. (2021) · arXiv · doi:10.48550/arxiv.2101.03961 · Cited by 1,005
(2024) · doi:10.1109/access.2024.3365742 · Cited by 934
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · cited 1×
“The basic concept is that different experts specialize in different tasks or data types and a gating network dynamically selects which experts to activate given a particular input (Shazeer et al. 2017 ).”
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021) · arXiv · doi:10.48550/arxiv.2104.08786 · Cited by 615 · cited 1×
“A key issue is prompt sensitivity or brittleness, where minor variations in the prompt syntax, often unintuitive to humans, can produce significantly different outputs (Lu et al. 2022 ; Webson and Pavlick 2022 ) reducing the reliability and reproducibility of LLMs.”
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.556 · Cited by 610 · cited 1×
“A key issue is prompt sensitivity or brittleness, where minor variations in the prompt syntax, often unintuitive to humans, can produce significantly different outputs (Lu et al. 2022 ; Webson and Pavlick 2022 ) reducing the reliability and reproducibility of LLMs.”
Multilingual Denoising Pre-training for Neural Machine TranslationPreprint
Liu, Y., Gu, J., Goyal, N., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.08210 · Cited by 606
mT5: A massively multilingual pre-trained text-to-text transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al‐Rfou, R., Siddhant, A., Barua, A., & Raffel, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11934 · Cited by 594
Torres H., J. P. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17754506 · Cited by 580
Code Llama: Open Foundation Models for Code
Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12950 · Cited by 531
Do Prompt-Based Models Really Understand the Meaning of Their Prompts?Preprint
Webson, A., & Pavlick, E. (2021) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2109.01247 · Cited by 527
Artificial intelligence applications in supply chain management
Pournader, M., Ghaderi, H., Hassanzadegan, A., & Fahimnia, B. (2021) · International Journal of Production Economics · doi:10.1016/j.ijpe.2021.108250 · Cited by 519
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505
Language Model Tokenizers Introduce Unfairness Between Languages
Petrov, A., Malfa, E. L., Torr, P., & Bibi, A. (2023) · neural information processing systems · doi:10.52202/075280-1608 · Cited by 492
Security and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445 · cited 1×
“Researchers have identified such exploits, like backdoor attacks, jailbreaking, data poisoning, prompt injections that can jeopardize the security of LLMs (Das et al. 2024 ).”
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
He, P., Liu, X., Gao, J., & Chen, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03654 · Cited by 420
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., & Chen, Z. (2020) · arXiv · doi:10.48550/arxiv.2006.16668 · Cited by 412
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink
Patterson, D., Gonzalez, J., Holzle, U., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D. R., Texier, M., & Dean, J. (2022) · Computer · doi:10.1109/mc.2022.3148714 · Cited by 400
Improving language models by retrieving from trillions of tokens
Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Driessche, G. V. D., Lespiau, J.-B., Damoc, B., Clark, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04426 · Cited by 382 · cited 1×
“…of a small number of parameters for tuning (Hu et al. 2021 ), and Retrieval-Augmented Generation (RAG), a post-training approach that involves requesting additional data related to the input from specified data - bases during inference (Lewis et al. 2020; Arora et al. 2023 ; Borgeaud et al. 2021 ).”
Self-Attention ConvLSTM for Spatiotemporal Prediction
Lin, Z., Li, M., Zheng, Z., Cheng, Y., & Yuan, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6819 · Cited by 378
How can artificial intelligence impact sustainability: A systematic literature review
Kar, A. K., Choudhary, S. K., & Singh, V. K. (2022) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2022.134120 · Cited by 366
Bengesi, S., El-Sayed, H., Sarker, M. K., Houkpati, Y., Irungu, J., & Oladunni, T. (2024) · IEEE Access · doi:10.1109/access.2024.3397775 · Cited by 362
Data-efficient Fine-tuning for LLM-based Recommendation
Lin, X., Wang, W., Li, Y., Yang, S., Feng, F., Wei, Y., & Chua, T. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657807 · Cited by 361
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340
Towards Expert-Level Medical Question Answering with Large Language Models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.09617 · Cited by 338
Large Language Models: A Survey
Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., & Gao, J. (2024) · arXiv · doi:10.48550/arxiv.2402.06196 · Cited by 325
Machine learning applications in urban building energy performance forecasting: A systematic review
Fathi, S., Srinivasan, R., Fenner, A., & Fathi, S. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.110287 · Cited by 313
Language Model Tokenizers Introduce Unfairness Between LanguagesPreprint
Bibi, A., Malfa, E. L., Petrov, A., et al. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2305.15425 · Cited by 303
Liu, Y., Yang, Z., Yu, Z., Liu, Z., Liu, D., Lin, H., Li, M., Ma, S., Avdeev, M., & Shi, S. (2023) · Journal of Materiomics · doi:10.1016/j.jmat.2023.05.001 · Cited by 294
Safety leading indicators for construction sites: A machine learning approach
Poh, C. Q. X., Ubeynarayana, C. U., & Goh, Y. M. (2018) · Automation in Construction · doi:10.1016/j.autcon.2018.03.022 · Cited by 290
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · arXiv · doi:10.48550/arxiv.2101.00190 · Cited by 289
Mehmood, M. U., Chun, D., Zeeshan, Han, H., Jeon, G., & Chen, K. (2019) · Energy and Buildings · doi:10.1016/j.enbuild.2019.109383 · Cited by 286
Investigating the Use of ChatGPT for the Scheduling of Construction Projects
Prieto, S. A., Mengiste, E. T., & García de Soto, B. (2023) · Buildings · doi:10.3390/buildings13040857 · Cited by 229 · cited 1×
“Prieto et al. ( 2023 ) also evaluated the GPT-3.”
Voyager: An Open-Ended Embodied Agent with Large Language Models
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023) · arXiv · doi:10.48550/arxiv.2305.16291 · Cited by 222 · cited 2×
“Xie et al. ( 2023 ) focused on evaluating LLMs, particularly GPT-3.”
EPlus-LLM: A large language model-based computing platform for automated building energy modeling
Jiang, G., Ma, Z., Zhang, L., & Chen, J. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.123431 · Cited by 204
Do Prompt-Based Models Really Understand the Meaning of Their Prompts?
Webson, A., & Pavlick, E. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.167 · Cited by 203
GPT models in construction industry: Opportunities, limitations, and a use case validation
Saka, A., Taiwo, R., Saka, N., Salami, B. A., Ajayi, S., Akande, K., & Kazemi, H. (2024) · Developments in the Built Environment · doi:10.1016/j.dibe.2023.100300 · Cited by 202 · cited 3×
“During the design phase, they perform code compliance verification, cost estimation, energy efficiency modeling, and automated design generation (Saka et al. 2023a, b ).”
Challenges and Applications of Large Language Models
Kaddour, J., Harris, J., Mozes, M., Bradley, H., Raileanu, R., & McHardy, R. (2023) · arXiv · doi:10.48550/arxiv.2307.10169 · Cited by 196 · cited 1×
“A more thorough analysis of the challenges and current approaches, solutions and directions for future research on LLMs was con - ducted by Kaddour et al. ( 2023 ).”
Environmental planning based on reduce, reuse, recycle and recover using artificial intelligence
Yu, K. H., Zhang, Y., Li, D., Montenegro-Marin, C. E., & Kumar, P. M. (2021) · Environmental Impact Assessment Review · doi:10.1016/j.eiar.2020.106492 · Cited by 193
Applications of machine learning to BIM: A systematic literature review
Zabin, A., González, V. A., Zou, Y., & Amor, R. (2021) · Advanced Engineering Informatics · doi:10.1016/j.aei.2021.101474 · Cited by 180
OpenAI, :, Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., et al. (2024) · arXiv · Cited by 180
Estimating construction waste generation in the Greater Bay Area, China using machine learning
Lu, W., Lou, J., Webster, C. J., Xue, F., Bao, Z., & Chi, B. (2021) · Waste Management · doi:10.1016/j.wasman.2021.08.012 · Cited by 176
Ghimire, P., Kim, K., & Acharya, M. (2024) · Buildings · doi:10.3390/buildings14010220 · Cited by 175 · cited 2×
“In the pre-design phase, LLMs can streamline feasibility analysis, generate project briefing documents, and assist in conceptual design selection through rapid data retrieval and generative modeling (Ghimire et al. 2024 ) .”
Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel
Tsai, Y.-H. H., Bai, S., Yamada, M., Morency, L.-P., & Salakhutdinov, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1443 · Cited by 173 · cited 1×
“PE is crucial for most transformer-based architec - tures, but certain decoder-only models can perform better even without PE (Tsai et al. 2019 ).”
Afzal, F., Yunfei, S., Nazir, M., & Bhatti, S. M. (2021) · International Journal of Managing Projects in Business · doi:10.1108/ijmpb-02-2019-0047 · Cited by 172 · cited 1×
“…2020 ), supply chain manage - ment (Pournader et al. 2021 ), logistics management (Fang and Ng 2019 ), risk management (Sanni-Anibire et al. 2020 ; Afzal et al. 2019 ), dispute resolution (Chou and Lin 2012 ), waste management (Lu et al. 2021; Yu et al. 2020 ), sustainability assessment (Kar et…”
Deep learning-based data analytics for safety in construction
Liu, J., Luo, H., & Liu, J. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104302 · Cited by 169
Data-efficient Fine-tuning for LLM-based Recommendation
Lin, X., Wang, W., Li, Y., Yang, S., Feng, F., Wei, Y., & Chua, T.-S. (2024) · arXiv · doi:10.48550/arxiv.2401.17197 · Cited by 165
Predicting construction cost overruns using text mining, numerical data and ensemble classifiers
Williams, T. P., & Gong, J. (2014) · Automation in Construction · doi:10.1016/j.autcon.2014.02.014 · Cited by 164
Applications of natural language processing in construction
Ding, Y., Ma, J., & Luo, X. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104169 · Cited by 161
Saka, A. B., Oyedele, L. O., Akanbi, L. A., Ganiyu, S. A., Chan, D. W. M., & Bello, S. A. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101869 · Cited by 161 · cited 4×
“During the design phase, they perform code compliance verification, cost estimation, energy efficiency modeling, and automated design generation (Saka et al. 2023a, b ).”
InfluentialLeveraging ChatGPT to Aid Construction Hazard Recognition and Support Safety Education and Training
Uddin, S. M. J., Albert, A., Ovid, A., & Alsharef, A. (2023) · Sustainability · doi:10.3390/su15097121 · Cited by 158 · cited 1×
“Uddin et al. ( 2023 ) conducted a study involving construction students to determine if ChatGPT can assist them in identifying construction hazards.”
Dynamic prompt-based virtual assistant framework for BIM information search
Zheng, J., & Fischer, M. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105067 · Cited by 154 · cited 1×
“Zheng and Fischer ( 2023 ) developed a virtual assistant framework for BIM incorporating LLMs (GPT) to search and extract information providing responses with relevant 3D rep - resentations.”
Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximation
Zhang, C., Kuppannagari, S. R., Kannan, R., & Prasanna, V. K. (2019) · ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys) · doi:10.1145/3360322.3360861 · Cited by 149
Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. D. L., Hanna, E. B., Bressand, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.04088 · Cited by 143
Beyond neural scaling laws: beating power law scaling via data pruning
Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S., & Morcos, A. S. (2022) · arXiv · doi:10.48550/arxiv.2206.14486 · Cited by 142 · cited 1×
“…resources with performance improving at a gradually decreasing rate when dataset and model size remain fixed (Kaplan et al. 2020 ), with Sorscher et al. ( 2022 ) highlighting that for a reduction of 2% in error, orders of magni - tude more energy consumption is required, proving…”
Beyond Neural Scaling Laws: Beating Power Law Scaling Via Data Pruning
Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S., & Morcos, A. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1419 · Cited by 142
Gambatese, J. A., & Hallowell, M. (2011) · Construction Management and Economics · doi:10.1080/01446193.2011.570355 · Cited by 138
Moon, S., Chi, S., & Im, S.-B. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104465 · Cited by 134 · cited 1×
“Moon et al. ( 2022 ) also presented a method utilizing BERT for identifying and classify - ing risk-related clauses from construction specifications.”
AutoRepo: A general framework for multimodal LLM-based automated construction reporting
Pu, H., Yang, X., Li, J., & Guo, R. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124601 · Cited by 126
Jin, Y., Yang, R., Yi, Z., Shen, X., Peng, H., Liu, X., Qin, J., Li, J., Xie, J., Gao, P., et al. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802229 · Cited by 118
Large language model-based interpretable machine learning control in building energy systems
Zhang, L., & Chen, Z. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.114278 · Cited by 116
The Falcon Series of Open Language Models
Almazrouei, E., Alobeidli, H., Alshamsi, A., Cappelli, A., Cojocaru, R., Debbah, M. A., Goffinet, É., Hesslow, D., Launay, J., Malartic, Q., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.16867 · Cited by 113
Multi-Level Domain Adaptive Learning for Cross-Domain Detection
Xie, R., Yu, F., Wang, J., Wang, Y., & Zhang, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00401 · Cited by 109
Chen, N., Lin, X., Jiang, H., & An, Y. (2024) · Buildings · doi:10.3390/buildings14071983 · Cited by 106 · cited 1×
“Chen et al. ( 2024 ) proposed a compliance checking framework that integrates LLMs, DL and ontology knowledge models.”
Advancing building energy modeling with large language models: Exploration and case studies
Zhang, L., Chen, Z., & Ford, V. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.114788 · Cited by 102
Robot-Enabled Construction Assembly with Automated Sequence Planning based on ChatGPT: RoboGPT
You, H., Ye, Y., Zhou, T., Zhu, Q., & Du, J. (2023) · arXiv · doi:10.48550/arxiv.2304.11018 · Cited by 100 · cited 1×
“You et al. ( 2023 ) integrated ChatGPT’s planning capabilities into robotic systems to automate construction assembly, with the model used for optimizing sequence planning and for adapting to dynamic environments.”
Robot-Enabled Construction Assembly with Automated Sequence Planning Based on ChatGPT: RoboGPT
You, H., Ye, Y., Zhou, T., Zhu, Q., & Du, J. (2023) · Buildings · doi:10.3390/buildings13071772 · Cited by 99
Predicting Disputes in Public-Private Partnership Projects: Classification and Ensemble Models
Chou, J.-S., & Lin, C. (2013) · Journal of Computing in Civil Engineering · doi:10.1061/(asce)cp.1943-5487.0000197 · Cited by 94
Highway Construction Safety Analysis Using Large Language Models
Smetana, M., Salles de Salles, L., Sukharev, I., & Khazanovich, L. (2024) · Applied Sciences · doi:10.3390/app14041352 · Cited by 87
LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Traces
Ouyang, X., & Srivastava, M. B. (2024) · IEEE Workshop on Machine Learning on Edge in Sensor Systems (SenSys-ML) · doi:10.1109/sensys-ml62579.2024.00007 · Cited by 83
Domain Adaptive Transfer Learning with Specialist Models
Ngiam, J., Peng, D., Vasudevan, V., Kornblith, S., Le, Q. V., & Pang, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07056 · Cited by 80
British Library Books genre detection model
van Strien, D. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.5245174 · Cited by 80
Jang, S., Lee, G., Oh, J., Lee, J., & Koo, B. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102532 · Cited by 77 · cited 1×
“Jang et al. ( 2024 ) developed a framework utilizing LLMs to generate detailed architec - tural designs of exterior walls by processing text input in natural language and integrating them into the BIM models.”
Application of artificial intelligence and machine learning for BIM: review
Bassir, D., Lodge, H., Chang, H., Majak, J., & Chen, G. (2023) · International Journal for Simulation and Multidisciplinary Design Optimization · doi:10.1051/smdo/2023005 · Cited by 77
Płoszaj-Mazurek, M., & Ryńska, E. D. (2024) · Energies · doi:10.3390/en17122997 · Cited by 73
Parameter Optimization for Automated Concrete Detection in Image Data
Zhu, Z., & Brilakis, I. (2010) · Automation in Construction · doi:10.1016/j.autcon.2010.06.008 · Cited by 71 · cited 1×
“…), safety management (Poh et al. 2018; Liu et al. 2022 ), schedule optimization (Zhang et al. 2019; Hatami et al. 2022 ), progress monitor - ing (Zhu and Brilakis 2010 ), quality control (Andenæs et al. 2020 ), supply chain manage - ment (Pournader et al. 2021 ), logistics management (Fang and…”
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71
An LLM-Based Digital Twin for Optimizing Human-in-the Loop Systems
Yang, H., Siew, M., & Joe‐Wong, C. (2024) · 2024 IEEE International Workshop on Foundation Models for Cyber-Physical Systems & Internet of Things (FMSys) · doi:10.1109/fmsys62467.2024.00009 · Cited by 71
Genetic algorithm for determining the construction logistics of precast components
Fang, Y., & Ng, S. T. (2019) · Engineering Construction & Architectural Management · doi:10.1108/ecam-09-2018-0386 · Cited by 70
Machine learning model for delay risk assessment in tall building projects
Sanni-Anibire, M. O., Zin, R. M., & Olatunji, S. O. (2022) · International Journal of Construction Management · doi:10.1080/15623599.2020.1768326 · Cited by 69
Amer, F., Jung, Y., & Golparvar-Fard, M. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103929 · Cited by 68 · cited 1×
“One of the first attempts at exploring the use of LLMs in construction management is the work of Amer et al. ( 2021 ) that employed an earlier version (GPT-2) of a transformer-based model to automate the alignment between long-term master schedules with short-term look-ahead plans.”
Domain-specific language models pre-trained on construction management systems corpora
Zhong, Y., & Goodfellow, S. D. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105316 · Cited by 65 · cited 1×
“Zhong and Goodfellow ( 2024 ) realized the need for a domain-specific LLM on the field of Construction Management Systems (CMS) and produced a CMS domain corpus with aca - demic papers that they employed to fine-tune pre-trained BERT and RoBERTa models.”
Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
Talebirad, Y., & Nadiri, A. (2023) · arXiv · doi:10.48550/arxiv.2306.03314 · Cited by 65 · cited 1×
“Systems utilizing multiple agents or multi-agent systems is a recent development with researchers exploring and developing frameworks for multi-agent collaborations (Liu et al. 2023b ; Talebirad and Nadiri 2023 ), demonstrating advanced problem-solving skills, autonomy and versatility.”
Guan, L., Valmeekam, K., Sreedharan, S., & Kambhampati, S. (2023) · arXiv · doi:10.48550/arxiv.2305.14909 · Cited by 64
Lü, J., Tian, X., Zhang, C., Zhao, Y., Zhang, J., Zhang, W., Feng, C., He, J., Wang, J., & He, F. (2024) · Energy and Built Environment · doi:10.1016/j.enbenv.2024.03.010 · Cited by 56
Falcon2-11B Technical ReportPreprint
Malartic, Q., Chowdhury, N. R., Cojocaru, R., Farooq, M., Campesan, G., Djilali, Y. A. D., Narayan, S., Singh, A., Velikanov, M., Boussaha, B. E. A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.14885 · Cited by 53
Guan, L., Valmeekam, K., Sreedharan, S., & Subbarao Kambhampati, B. (2023) · neural information processing systems · doi:10.52202/075280-3459 · Cited by 52
Translating Natural Language to Planning Goals with Large-Language Models
Xie, Y., Yu, C., Zhu, T., Bai, J., Gong, Z., & Soh, H. (2023) · arXiv · doi:10.48550/arxiv.2302.05128 · Cited by 50 · cited 1×
“Xie et al. ( 2023 ) focused on evaluating LLMs, particularly GPT-3.”
Rane, N. (2023) · doi:10.2139/ssrn.4603221 · Cited by 43 · cited 1×
“Another contribution comes from Rane ( 2023 ), who demonstrated the role of generative AI, particularly ChatGPT, in transforming the construction industry in three major aspects: Industry 4.0, Industry 5.0, and Society 5.0.”
InfluentialMeng, F., Lu, Z., Li, X., Han, W., Peng, J., Liu, X., & Niu, Z. (2024) · Energy · doi:10.1016/j.energy.2024.130303 · Cited by 43 · cited 1×
“Meng et al. ( 2024 ) focused on the domain of Demand-Size Management (DSM) and utilized LLMs to perform extensive bibliometric analysis in the field.”
Jang, S., & Lee, G. (2024) · ASCE International Conference on Computing in Civil Engineering (i3CE) · doi:10.1061/9780784485231.035 · Cited by 43
The Framework of Combining Artificial Intelligence and Construction 3D Printing in Civil Engineering
Tan, K. H. (2018) · MATEC Web of Conferences · doi:10.1051/matecconf/201820601008 · Cited by 42
Designing Home Automation Routines Using an LLM-Based Chatbot
Giudici, M., Padalino, L., Paolino, G., Paratici, I., Pascu, A. I., & Garzotto, F. (2024) · Designs · doi:10.3390/designs8030043 · Cited by 40 · cited 1×
“Giudici et al. ( 2024 ) proposed an LLM-based home automation agent framework, named GreenIFTTT, that interacts with users to suggest energy consumption reduction and cost optimization routines.”
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models
Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D., Smith, N., & Tsvetkov, Y. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.614 · Cited by 40
An LLM-Based Digital Twin for Optimizing Human-in-the Loop Systems
Yang, H., Siew, M., & Joe-Wong, C. (2024) · arXiv · doi:10.48550/arxiv.2403.16809 · Cited by 38
LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Traces
Ouyang, X., & Srivastava, M. (2024) · arXiv · doi:10.48550/arxiv.2403.19857 · Cited by 37
Applying Fine-Tuned LLMs for Reducing Data Needs in Load Profile AnalysisPreprint
Hu, Y., Kim, H., Ye, K., et al. (2024) · Applied Energy · doi:10.48550/arxiv.2406.02479 · Cited by 36
Security and Privacy Challenges of Large Language Models: A SurveyPreprint
Das, B. C., Amini, M. H., & Wu, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.00888 · Cited by 33
Jin, Y., Yang, R., Yi, Z., Shen, X., Peng, H., Liu, X., Qin, J., Li, J., Xie, J., Gao, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.13193 · Cited by 29 · cited 1×
“LLM agents demonstrated promising per - formance on a variety of tasks, including code generation (Shinn et al. 2023 ), autopilot systems (Jin et al. 2023 ), complex tasks in gaming environment (Wang et al. 2023 ) and more.”
Inference of Drawing Elements and Space Usage on Architectural Drawings Using Semantic Segmentation
Seo, J., Park, H., & Choo, S. (2020) · Applied Sciences · doi:10.3390/app10207347 · Cited by 26
A BERT-Based Model for Question Answering on Construction Incident Reports
Mohamed, H. A., Marengo, E., & Nutt, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08473-7_20 · Cited by 25
Generative AI in the Construction Industry: A State-of-the-art Analysis
Taiwo, R., Bello, I. T., Abdulai, S. F., Yussif, A.-M., Salami, B. A., Saka, A., & Zayed, T. (2024) · arXiv · doi:10.48550/arxiv.2402.09939 · Cited by 21 · cited 1×
“Recent reviews on the field of GAI in construction include major contributions like Taiwo et al. ( 2024 ) who conducted a review of potential GAI applications in the construction industry, presenting the current architectures, models being currently developed for processing and generating text,…”
Wang, J., Jiang, H., Liu, Y., Ma, C., Zhang, X., Pan, Y., Liu, M., Gu, P., Xia, S., Chen, L., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.01319 · Cited by 21
A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration
Liu, Z., Zhang, Y., Li, P., Liu, Y., & Yang, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.02170 · Cited by 20
Han, D., Zhao, W., Yin, H., Qu, M., Zhu, J., Ma, F., Ying, Y., & Pan, A. (2025) · Journal of Asian Architecture and Building Engineering · doi:10.1080/13467581.2024.2329351 · Cited by 20
Hatami, M., Franz, B. W., Paneru, S., & Flood, I. (2022) · ASCE International Conference on Computing in Civil Engineering (i3CE) · doi:10.1061/9780784483893.143 · Cited by 19
Zong, M., & Krishnamachari, B. (2022) · arXiv · doi:10.48550/arxiv.2212.00857 · Cited by 13
Multi-level Domain Adaptive learning for Cross-Domain DetectionPreprint
Xie, R., Yu, F. R., Wang, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.11484 · Cited by 10
You Are What You Write: Preserving Privacy in the Era of Large Language Models
Plant, R., Giuffrida, V., & Gkatzia, D. (2022) · arXiv · doi:10.48550/arxiv.2204.09391 · Cited by 10
Perspectives on Quality Risk in the Building Process of Blue-Green Roofs in Norway
Andenæs, E., Engebø, A., Time, B., Lohne, J., Torp, O., & Kvande, T. (2020) · Buildings · doi:10.3390/buildings10100189 · Cited by 9
GAR-meets-RAG Paradigm for Zero-Shot Information Retrieval
Arora, D., Kini, A., Chowdhury, S. R., Natarajan, N., Sinha, G., & Sharma, A. (2023) · arXiv · doi:10.48550/arxiv.2310.20158 · Cited by 8
LLM-based System for Technical Writing Real-time Review in Urban Construction and Technology
Cruz Castro, L. M., Castelblanco, G., & Antonenko, P. D. (2024) · EPiC Series in Built Environment · doi:10.29007/d9j3 · Cited by 8 · cited 1×
“Castro et al. ( 2024 ) targeted the challenge of writing technical reports in the field of urban construction and explored the capabilities of GPT-4 in providing feedback in a timely and scalable manner on student’s technical reports.”
Jang, S., & Lee, G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14165 · Cited by 7
Transformer Dissection: A Unified Understanding of Transformer's\n Attention via the Lens of KernelPreprint
Tsai, Y.-H. H., Bai, S., Yamada, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11775 · Cited by 6
Jang, S., Lee, G., Oh, J., et al. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4674577 · Cited by 5
Advancing Building Energy Modeling with Large Language Models: Exploration and Case Studies
Zhang, L., Chen, Z., Ford, V., & Xu, P. (2024) · IBPSA-USA Building Simulation Conference (SimBuild) · doi:10.26868/30680611.2024.2104 · Cited by 4
Do All Languages Cost the Same? Tokenization in the Era of Commercial Language ModelsPreprint
Ahia, O., Kumar, S., Gonen, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.13707 · Cited by 3
A Simplified Query-Only Attention for Encoder-Based Transformer Models
Yeom, H. G., & An, K. (2024) · Applied Sciences · doi:10.3390/app14198646 · Cited by 2
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Workshop, B., :, Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.05100
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · arXiv · doi:10.48550/arxiv.1801.06146
Eplus-Llm: A Large Language Model-Based Computing Platform for Automated Building Energy Modeling
Jiang, G., Ma, Z., Zhang, L., et al. (2024) · doi:10.2139/ssrn.4743437
Research works printed but not linked: 1
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. https://api.semanticscholar.org/CorpusID:160025533
Cited by
Hallucination to truth: a review of fact-checking and factuality evaluation in large language models
Rahman, S. S., Islam, M. A., Alam, M. M., Zeba, M., Rahman, M. A., Chowa, S. S., Raiaan, M. A. K., & Azam, S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11454-w · Cited by 39
Yu, G., Wang, Y., Wang, Z., Chen, L., Zheng, Y., & Liu, Y. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104655 · Cited by 22
Ten questions concerning Large Language Models (LLMs) for building applications
Ma, N., Labib, R., Amor, R., Chong, A., Fan, C., Forth, K., Fu, X., Fuchs, S., Hong, T., Klimenkova, N., et al. (2026) · Building and Environment · doi:10.1016/j.buildenv.2026.114260 · Cited by 30
Xu, Y., Li, A., Guo, X., Chan, J. C. F., Gong, X., Kwok, H. L., & Cheng, J. C. P. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106611 · Cited by 16
Automatic classification of construction accident reports using BERTopic-GLDA approach
Wang, J., Qu, Z., Wu, S., Skitmore, M., & Ma, L. (2025) · Engineering Construction & Architectural Management · doi:10.1108/ecam-04-2025-0692 · Cited by 11
Metadata
- Authors
- Kampelopoulos, D., Tsanousa, A., Vrochidis, S., & Kompatsiaris, I.
- Affiliations
- Centre for Research and Technology Hellas (GR)
- Corresponding author
- Kampelopoulos, Dimitrios
- Tsanousa, Athina
- Vrochidis, Stefanos
- Kompatsiaris, Ioannis
- Venue
- Artificial Intelligence Review
- Published
- 2025-05-16
- Volume
- 58
- Issue
- 8
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W4410453424
- Cited by
- 69
- References
- 134
- Citations by year
- 2026: 60 · 2025: 9
- FWCI
- 28.59
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Artificial Intelligence Review (publishedVersion, cc-by)
- Funders
- Centre for Research & Technology Hellas
- Topics
- Physical Sciences › Engineering › Building and Construction › BIM and Construction Integration
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, architecture, engineering and construction, construction industry, transformer models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.