Multi-modal digital twins for industrial anomaly detection: Framework, method, and application

Abstract

Anomaly detection plays a key role in maintaining the reliable and stable operation of industrial systems, especially in high-reliability fields. Conventional single-modal data cannot provide comprehensive information about the detected object, resulting in false or missed detection. To address the challenges of complex anomaly patterns and heterogeneous data in industrial scenarios, we propose MMDT-IAD, a multi-modal digital twin (DT)-based anomaly detection framework that integrates edge–cloud collaboration. By lever- aging physical, geometric, visual, and semantic modalities, MMDT-IAD constructs a comprehensive virtual representation of monitored objects and enables real-time, scalable detection across distributed industrial environments. Next, to enable efficient fusion of heterogeneous DT modalities, we propose a One-Primary- Three-Auxiliary (1P3A) cross-modal decision fusion strategy. Finally, we apply the MMDT-IAD frame-work to the anomaly detection of aviation electrical connector pins, and present a detailed application process. The experimental results prove the effectiveness of the MMDT-IAD framework in detecting abnormal pins. Moreover, we discuss the generality of MMDT-IAD framework considering several common industrial anomalies. These results highlight the potential of MMDT-IAD framework and 1P3A method to significantly improve anomaly detection in other complex industrial scenarios.