Empirical study on fine-tuning pre-trained large language models for fault diagnosis of complex systems

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

Survey of Hallucination in Natural Language Generation

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025

Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505

Learning without Forgetting

Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202

Large language models encode clinical knowledge

Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083

Large Language Models Encode Clinical Knowledge

Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083

A plant-wide industrial process control problem

Downs, J. J., & Vogel, E. F. (1993) · Computers & Chemical Engineering · doi:10.1016/0098-1354(93)80018-i · Cited by 3,348

Training Language Models to Follow Instructions with Human Feedback

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101

Llama 2: Open Foundation and Fine-Tuned Chat Models

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604

Film: Visual reasoning with a general conditioning layer

Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11671 · Cited by 2,490

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473

ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope

Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463

Investigating the impact of data normalization on classification performance

Singh, D., & Singh, B. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105524 · Cited by 1,820

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

(2021) · doi:10.18653/v1/2021.emnlp-main · Cited by 1,685

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

(2020) · doi:10.18653/v1/2020.emnlp-main · Cited by 1,296

Self-Instruct: Aligning Language Models with Self-Generated Instructions

Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.754 · Cited by 1,243

Parameter-efficient fine-tuning of large-scale pre-trained language models

Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00626-4 · Cited by 1,191

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · UCL Discovery (University College London) · Cited by 1,187

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082

Deep learning modelling techniques: Current progress, applications, advantages, and challenges

Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079

Prognostics and Health Management (PHM): Where are we and where do we (need to) go in theory and practice

Zio, E. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.108119 · Cited by 789

A review of the application of deep learning in intelligent fault diagnosis of rotating machinery

Zhu, Z., Lei, Y., Qi, G., Chai, Y., Mazur, N., An, Y., & Huang, X. (2022) · Measurement · doi:10.1016/j.measurement.2022.112346 · Cited by 779

Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning

Aghajanyan, A., Gupta, S., & Zettlemoyer, L. (2021) · doi:10.18653/v1/2021.acl-long.568 · Cited by 723

Proceedings of the 31st annual meeting on Association for Computational Linguistics -

(1993) · doi:10.3115/981574 · Cited by 558

Graph of Thoughts: Solving Elaborate Problems with Large Language Models

Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556

Graph of Thoughts: Solving Elaborate Problems with Large Language Models

Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.09687 · Cited by 556

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

(2020) · doi:10.18653/v1/2020.acl-main · Cited by 551

Pre-Trained Language Models and Their Applications

Wang, H., Li, J., Wu, H. R., Hovy, E. H., & Sun, Y. (2022) · Engineering · doi:10.1016/j.eng.2022.04.024 · Cited by 435

Generative adversarial network in mechanical fault diagnosis under small sample: A systematic review on applications and future perspectives

Pan, T., Chen, J., Zhang, T., Liu, S., He, S., & Lv, H. (2021) · ISA Transactions · doi:10.1016/j.isatra.2021.11.040 · Cited by 287

Towards trustworthy machine fault diagnosis: A probabilistic Bayesian deep learning framework

Zhou, T., Han, T., & Droguett, E. L. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.108525 · Cited by 268

Long short-term memory network with Bayesian optimization for health prognostics of lithium-ion batteries based on partial incremental capacity analysis

Meng, H., Geng, M., & Han, T. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109288 · Cited by 264

Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting

Chen, S., Hou, Y., Cui, Y., Che, W., Liu, T., & Yu, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.634 · Cited by 226

A rotating machinery fault diagnosis method based on multi-scale dimensionless indicators and random forests

Hu, Q., Si, X., Zhang, Q.-H., & Qin, A. (2020) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.106609 · Cited by 177

Rolling Bearing Fault Diagnosis Under Data Imbalance and Variable Speed Based on Adaptive Clustering Weighted Oversampling

Li, S., Peng, Y., Shen, Y., Zhao, S., Shao, H., Bin, G., Guo, Y. Z., Yang, X., & Fan, C. (2024) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2024.109938 · Cited by 154

ChatGPT-like large-scale foundation models for prognostics and health management: A survey and roadmaps

Li, Y., Wang, H., & Sun, M. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109850 · Cited by 149

Explainability-driven model improvement for SOH estimation of lithium-ion battery

Wang, F., Zhao, Z., Zhai, Z., Shang, Z., Yan, R., & Chen, X. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.109046 · Cited by 147

From task structures to world models: what do LLMs know?

Yildirim, I., & Paul, L. A. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.02.008 · Cited by 138

TRA-ACGAN: A motor bearing fault diagnosis model based on an auxiliary classifier generative adversarial network and transformer network

Fu, Z., Liu, Z., Shuangrui, P. I. N. G., Li, W., & Liu, J. (2024) · ISA Transactions · doi:10.1016/j.isatra.2024.03.033 · Cited by 120

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Luo, Z., Xu, C., Zhao, P., Sun, Q., Geng, X., Hu, W., Tao, C., Ma, J., Lin, Q., & Jiang, D. (2023) · arXiv · doi:10.48550/arxiv.2306.08568 · Cited by 120

Frozen Pretrained Transformers as Universal Computation Engines

Lu, K., Grover, A., Abbeel, P., & Mordatch, I. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20729 · Cited by 113

LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks

Dinh, T., Zeng, Y., Zhang, R., Lin, Z., Gira, M., Rajput, S., Sohn, J.-Y., Papailiopoulos, D., & Lee, K. (2022) · neural information processing systems · doi:10.52202/068431-0855 · Cited by 93

Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

Pezeshkpour, P., & Hruschka, E. (2023) · arXiv · doi:10.48550/arxiv.2308.11483 · Cited by 86

Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions

Pezeshkpour, P., & Hruschka, E. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.130 · Cited by 85

PMSM Stator Winding Fault Detection and Classification Based on Bispectrum Analysis and Convolutional Neural Network

Pietrzak, P., Wolkiewicz, M., & Orłowska-Kowalska, T. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2022.3189076 · Cited by 83

Challenges and opportunities for battery health estimation: Bridging laboratory research and real-world applications

Han, T., Tian, J., Chung, C. Y., & Wei, Y. (2023) · Journal of Energy Chemistry · doi:10.1016/j.jechem.2023.10.032 · Cited by 81

A method for fault detection in multi-component systems based on sparse autoencoder-based deep neural networks

Yang, Z., Baraldi, P., & Zio, E. (2021) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2021.108278 · Cited by 80

XFDDC: eXplainable Fault Detection Diagnosis and Correction framework for chemical process systems

Harinarayan, R. R. A., & Shalinie, S. M. (2022) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2022.07.019 · Cited by 75

A Local Adaptive Minority Selection and Oversampling Method for Class-Imbalanced Fault Diagnostics in Industrial Systems

Wu, Z., Lin, W., Fu, B., Guo, J., Ji, Y., & PECHT, M. G. (2019) · IEEE Transactions on Reliability · doi:10.1109/tr.2019.2942049 · Cited by 73

Companion Proceedings of the ACM Web Conference 2023

(2023) · doi:10.1145/3543873 · Cited by 69

Water management fault diagnosis for proton-exchange membrane fuel cells based on deep learning methods

Xiao, F., Chen, T., Zhang, J., & Zhang, S. (2023) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2023.03.097 · Cited by 63

Causal Graph Attention Network with Disentangled Representations for Complex Systems Fault Detection

Liu, J., Zheng, S., & Wang, C. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109232 · Cited by 62

Fault detection in complex mechatronic systems by a hierarchical graph convolution attention network based on causal paths

Zheng, S., Wang, C., Zio, E., & Liu, J. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109872 · Cited by 58

A SVM-based framework for fault detection in high-speed trains

Liu, J., Hu, Y., & Yang, S. (2020) · Measurement · doi:10.1016/j.measurement.2020.108779 · Cited by 57

Weighted K-NN Classification Method of Bearings Fault Diagnosis With Multi-Dimensional Sensitive Features

Wang, Q., Wang, S., Wei, B., Chen, W., & Zhang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3066489 · Cited by 56

Mutual Information Alleviates Hallucinations in Abstractive Summarization

van der Poel, L., Cotterell, R., & Meister, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.399 · Cited by 54

LLMs to the Moon? Reddit Market Sentiment Analysis with Large Language Models

Deng, X., Bashlovkina, V., Han, F., Baumgartner, S., & Bendersky, M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587605 · Cited by 51

A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3Preprint

Nguyen, H.-T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.05729 · Cited by 44

Multi-Class Support Vector Machine with Maximizing Minimum Margin

Nie, F., Hao, Z., & Wang, R. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i13.29361 · Cited by 33

Physics-guided, data-refined fault root cause tracing framework for complex electromechanical system

Xu, J., Wang, R., Liang, Z., Liu, P., Gao, J., & Wang, Z. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109293 · Cited by 25

A fault diagnosis method for building electrical systems based on the combination of variational modal decomposition and new mutual dimensionless

Xiong, J., Qian, W., Cen, J., Li, J., Liu, J., & Tang, L. (2023) · Scientific Reports · doi:10.1038/s41598-022-27031-y · Cited by 10

Fault Detection in Industrial Systems Using Maximized Divergence Analysis Approach

Jiang, B., & Lu, Q. (2022) · IEEE Access · doi:10.1109/access.2022.3181360 · Cited by 5

Mutual Information Alleviates Hallucinations in Abstractive SummarizationPreprint

van der Poel, L., Cotterell, R., & Meister, C. (2022) · arXiv (Cornell University) · doi:10.3929/ethz-b-000588609 · Cited by 4

FD-LLM: Large language model for fault diagnosis of complex equipment

Lin, L., Zhang, S., Fu, S., & Liu, Y. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103208 · Cited by 153

Enhancing diagnostic capability with multi-agents conversational large language models

Chen, X., Yi, H., You, M., Liu, W., Wang, L., Li, H., Zhang, X., Guo, Y., Fan, L., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01550-0 · Cited by 147

IIoT-enabled digital twin for legacy and smart factory machines with LLM integration

Gautam, A., Aryal, M. R., Deshpande, S., Padalkar, S., Nikolaenko, M., Tang, M., & Anand, S. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.03.022 · Cited by 64

A Survey on Applications of Large Language Model-Driven Digital Twins for Intelligent Network Optimization

Guo, Z., Tang, F., Luo, L., Zhao, M., & Kato, N. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3568637 · Cited by 52

A Review of Large Language Models for Energy Systems: Applications, Challenges, and Future Prospects

Mirshekali, H., Shadi, M. R., Ladani, F. G., & Shaker, H. R. (2025) · IEEE Access · doi:10.1109/access.2025.3610994 · Cited by 64

Exploring LLM-based Frameworks for Fault DiagnosisPreprint

Lee, X. Y., Vidyaratne, L., Farahat, A., & Gupta, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.23113

The Interpretable Reasoning and Intelligent Decision-Making Based on Event Knowledge Graph With LLMs in Fault Diagnosis Scenarios

Men, C., Han, Y., Wang, P., Tao, J., & Huang, C. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3550999 · Cited by 37

Domain-specific large language models for fault diagnosis of heating, ventilation, and air conditioning systems by labeled-data-supervised fine-tuning

Zhang, J., Zhang, C., Lu, J., & Zhao, Y. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124378 · Cited by 79

Using SLMs and LLMs for Diagnosis

Plank, S., & Wotawa, F. (2026) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.dx.2026.14

A Multi-stage Post-Training Framework for Domain-Specific Language Models in Fault DiagnosisPreprint

Zhang, W., Fang, H., Wu, T., Wang, C., Xu, L., Bu, J., Yu, Y. S., & Zhong, Q. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5953702