Visual question answering for intelligent maintenance of maglev railway systems

Abstract

Traditional methodologies effectively assess the railway components by categorizing them as either normal or abnormal. However, these methods provide limited insight into underlying conditions and the reasons for abnormalities, often lacking a comprehensive explanation. With advancements in multimodal feature learning, multimodal data are potentially integrated for various downstream tasks, such as visual question answering (VQA). This paper proposes a three-phase procedure for VQA-guided maintenance of maglev conditions, aiming to automatically recognize evidence of damage and failure using accumulated multimodal knowledge. As one of the early VQA datasets designed for railway condition monitoring, it is organized as image-question-answer tuples, where images are generated from time-frequency spectrograms, questions and answers are formulated based on maglev structural dynamic characteristics. The results indicate that the proposed model is reliable in answer accuracy and expression quality. This advancement contributes to forming intelligent decision-making processes in railway infrastructure, ultimately promoting safer and more efficient railway operations.