FD-LLM: Large language model for fault diagnosis of complex equipment

In complex equipment fault diagnosis, traditional deep learning-based fault diagnosis methods usually require special design and training of “one model for one scenario,” the features of different fault categories overlap seriously, making misdiagnosis easy to occur. The Multimodal Large Language Model (MM-LLM) demonstrates strong multimodal understanding and logical reasoning abilities. This article attempts to use MM-LLM for complex equipment fault diagnosis to yield higher diagnostic accuracy. However, existing MM-LLMs lack domain-specific knowledge and modal alignment training for engineering time-series data, limiting their effectiveness in industrial fault diagnosis tasks. This article proposes a new fault diagnosis method based on MM-LLM in response to the above issues. First, by conducting modal alignment training on the description text of engineering data and equipment operation status in the feature space, the ability of the LLM to understand time-series data modalities is activated. Second, a fuzzy semantic embedding method is proposed to address the difficulty of identifying engineering data with pattern aliasing in the feature space. In addition, supplementary fault diagnosis background knowledge is introduced into MM-LLM by a learnable prompt embedding. Finally, the LORA method is used to fine-tune the LLM. Experimental results show that the proposed method can achieve higher fault classification accuracy.