A multimodal multi-scale fusion network for leak detection in marine piping systems

Marine system monitoring data inherently exhibit multimodal characteristics, making artificial intelligence-driven correlation and fusion essential for improving fault feature recognition. However, existing intelligent diagnosis methods mostly focus on feature fusion within homogeneous data types, such as fusing multiple time-series signals or multiple image sets, while systematic exploration of joint representation learning across heterogeneous dimensions remains under-explored. This limitation constrains the recognition capability for complex failure modes. Meanwhile, the inherent differences in physical meanings and representations of multimodal data pose significant challenges in constructing effective correlations, often limiting the performance of mainstream machine learning based fault diagnosis approaches. The proposed method enhances the fault diagnosis capability of mainstream approaches through the fusion of multi-sensor data and visual data, with its core innovation residing in a multimodal fusion framework leveraging attention mechanisms to effectively integrate cross-dimensional representations of multivariate time-series data and imaging data. Compared to existing multimodal transformer techniques, this dual-strategy architecture enables the model to simultaneously capture shared systemic behaviors and modality-unique signatures, substantially elevating diagnosis precision. Experimental validation on real-world leak detection datasets demonstrates that the proposed model achieves F1-scores consistently surpassing 90 % across diverse marine monitoring scenarios, with quantitative evaluations further confirming its superior performance over conventional multivariate time-series diagnosis methods in establishing multimodal correlations, conclusively validating both technical excellence and engineering practicability.