CCF-former: A transformer with cross-channel feature aggregation and frozen backbone for fault prediction

Abstract

Unexpected system faults may cause significant economic losses, service disruption, and safety risks in failure-prone interconnected systems, including industrial and distributed computing infrastructures. Therefore, accurate and timely fault prediction is essential for ensuring system reliability and maintaining continuous service availability. In this paper, we propose CCF-Former, a Transformer-based fault prediction framework that combines Cross-Channel feature aggregation and a Frozen pretrained backbone to predict failures in such interconnected systems. The proposed framework exhibits excellent fault prediction performance, maintaining both high precision and robustness. The framework combines three main components: (1) a Cross-Channel Feature Aggregation Module (CCFAM) that captures long-range dependencies and subtle fault patterns by aggregating and redistributing informative representations across input features; (2) a Frozen Pre-trained Transformer Module (FPTM) that captures temporal patterns using rich pre-trained representations, significantly reducing resource consumption and avoiding repeated fine-tuning; and (3) a Failure Inference Module (FIM) that produces reliable fault judgements through reconstruction-based scoring and adaptive thresholding. Extensive experiments on multiple public benchmarks, including server monitoring and spacecraft telemetry datasets, demonstrate that CCF-Former consistently outperforms state-of-the-art baselines, achieving a top F1-score of 87.94 %. The proposed framework offers a robust and effective solution for fault prediction in complex interconnected systems. Our code is publicly available at https://github.com/Yolandalt/CCF-Former.