Bagging of Complementary Neural Networks with Double Dynamic Weight Averaging

Abstract

Ensemble technique has been widely applied in regression problems. This paper proposes a novel approach of the ensemble of Complementary Neural Network (CMTNN) using double dynamic weight averaging. In order to enhance the diversity in the ensemble, different training datasets created based on bagging technique are applied to an ensemble of pairs of feed-forward back-propagation neural networks created to predict the level of truth and falsity values. In order to obtain more accuracy, uncertainties in the prediction of truth and falsity values are used to weight the prediction results in two steps. In the first step, the weight is used to average the truth and the falsity values whereas the weight in the second step is used to calculate the final regression output. The proposed approach has been tested with benchmarking UCI data sets. The results derived from our technique improve the prediction performance while compared to the traditional ensemble of neural networks which is predicted based on only the truth values. Furthermore, the obtained results from our novel approach outperform the results from the existing ensemble of complementary neural network.