Failure Characterization and Prediction of Scheduling Jobs in Google Cluster Traces

Abstract

In cloud computing, all services including infrastructure, platform, and software experience failures due to their large scale and heterogeneity nature. These failures can lead to job failure execution that may cause performance deterioration and resource waste. Most studies have focused mainly on failure analysis and characterization while there is limited research has been done on failure prediction. In this paper, the overall aim is to develop a failure prediction framework that can early detect failed jobs, and the real advantages of this framework are to decrease the resources waste and to increase the performance of cloud applications. Our failure analysis and prediction are based on Google cluster traces. We have developed a failure prediction model for job failure execution based on applying different machine learning algorithms and selecting the best accurate model. Moreover, we evaluate the model performance using different types of evaluation metrics to ensure that the proposed prediction model provides the highest accuracy of predicted values. Finally, we apply different feature selection techniques to improve the accuracy of our proposed model. Our evaluation results show that our model has achieved a high rate of precision, recall, and f1-score.