Data Augmentation to Improve the Performance of Ensemble Learning for System Failure Prediction with Limited Observations

Abstract

Ensemble learning has been widely used to improve the performance and robustness of machine learning algorithms on time series data. However, in real operational processes where the observed data is limited, it hinders the capability of ensemble learning algorithms. To address the challenge of limited observed data, this paper proposes a novel three-layer ensemble learning framework by use of data augmentation. Firstly, multiple classical time series augmentation methods are applied to increase the size of the data set. Subsequently, after pre-processing, these augmented data is trained by multiple basic learners with K-fold cross-validation as the first layer of the developed ensemble learning framework. The outputs of the first layer are integrated via LASSO to further improve the prediction performance, which serves as the second layer of the developed framework. Finally, the third-layer output is generated by averaging the prediction of the second layer and the output from an improved Long-Short Term Memory model that provides prediction based on the augmented data. A case study on a real wastewater treatment plant is used to illustrate the effectiveness of the proposed method.