From ensemble learning to deep ensemble learning: A case study on multi-indicator prediction of pavement performance

Abstract

Recently, Big data analytics approaches were combined with emerging machine learning techniques, which can provide more sophisticated insights into information-intensive activity. Compared to traditional shallow-architecture machine learning algorithms, Deep learning could excavate more potential information from the raw features. However, its powerful representational capacity relies on the support of enormous samples. The ensemble trees system performs superior on small sample problems due to better generalization capacity. To merge both benefits of deep learning and ensemble tree system, this paper developed a deep ensemble algorithm applied to multiple indicators prediction of pavement performance, including International Roughness Index and pavement 3-layer modulus. The deep ensemble algorithm is developed by merging a deep neural network (DNN) with the decision manifold property of the decision trees (TabNet) into a cascade ensemble system, combined with a sliding window algorithm to extract dependency information from raw data. During the training stage, the Bayesian Optimization Algorithm (BOA) is used to search for the optimal combination of sub-decision makers in the cascade ensemble. And equipped with GPU, it can speed up by 2.6–4.0 times. In the case study of pavement engineering, with sufficient training samples, it can achieve an average accuracy of 98.74 %, higher than DNN (97.49 %) and XGBoost (96.12 %) in predicting pavement indicators. With insufficient training samples, it can achieve an accuracy improvement of 12 % than XGBoost (75 %) and 24.5 % than DNN (62.5 %).