FSAMLM: A few-shot adaptation multimodal large model for cross-domain fault diagnosis

Recent advances in deep learning have demonstrated remarkable performance in mechanical fault diagnosis. However, most existing approaches are designed for a single data modality or a single task, limiting their flexibility and generalization ability. Moreover, common knowledge across different tasks remains largely unexploited. Inspired by the success of large-scale models in natural language processing and computer vision, integrating multimodal data with multi-task learning strategies may significantly improve the performance of fault diagnosis models. Nonetheless, a significant gap exists between general pre-trained knowledge and domain-specific expertise, posing considerable challenges for effective integration. To address these issues, we propose a novel few-shot adaptation multimodal large model (FSAMLM) that efficiently captures shared representations through a parameter-efficient fine-tuning strategy, structured in two stages. Specifically, we first design a low-rank adaptation meta-learning (LoRAML) framework, which employs low-rank decomposition on pre-trained parameters to reduce computational complexity and improve robustness. This approach not only accelerates adaptation to new few-shot tasks but also preserves pre-training knowledge effectively. During the second fine-tuning stage, we implement target-domain training with limited samples and introduce a training-free inference option for real-world deployment. Experimental validation using six public datasets demonstrates FSAMLM’s superior domain generalization capability in few-shot fault diagnosis tasks compared to existing methods. The code repository is publicly available at: https://github.com/ohhyeeaah/FSAMLM-for-fault-diagnosis.