GFIA: Generative Fault Image Analysis via vision–language model its application to train bogie transmission system

Abstract

Multimedia fault analytics plays a critical role in industrial applications, ensuring safety and reliability. Previous studies have explored fault classification using either one-dimensional signals or two-dimensional images, while understanding fault types and providing appropriate responses remains challenging, especially for complex system failures. To step further in this field, we leverage the powerful reasoning and generative capabilities of Large Multimodal Models (LMMs) for the fault analysis, then transform multi-channel sensor signals from the system into structured grayscale images suitable for visual–language models. Additionally, a domain-specific, strongly supervised dataset is constructed, that is, the Bogie Transmission Unified Fault Dataset (BTU), which contains expert-curated fault types, causes, and solutions. By integrating both image and language modalities, we fine-tune a visual–language model, Generative Fault Image Analysis (GFIA), to enhance fault reasoning and interpretation. Extensive experiments on our BTU dataset demonstrate that GFIA achieves an average diagnostic accuracy exceeding 99.9% for motor faults, reaching 100% for gearbox faults, and exceeding 99.8% for leftaxlebox faults. The proposed GFIA model outperforms traditional deep-learning methods and state-of-the-art large language models, highlighting the effectiveness of vision–language integration for fault analysis.