A Failure Prediction Approach Supporting Multi Granularity Data Fusion for Large-scale Cloud Storage Systems

With the development of cloud computing and cloud storage technology, the data scale has grown rapidly. In order to store and process large-scale data, there are thousands of nodes and devices in the cloud storage center, resulting in a surge in the frequency of failures. In various types of failure events, storage device failure is the most important one. However, most cloud storage systems lack disk failure prediction mechanisms and could only replace disks after disk failures. It is particularly important to predict the potential risks in the system operation environment. In this paper, we propose a disk failure prediction approach that supports multi granularity data fusion, which solves problems of unbalanced samples, single data source, cross scenario model migration and insufficient generalization ability of prediction models in disk failure prediction. Through our proposed approach, the cloud storage system could accurately predict disk failures and actively push prediction results to users, so as to improve the pertinence and planning of the operation and maintenance work. The approach presented in this paper has been validated to be valid through a series of qualitative and quantitative experiments.