Make decision boundary smoother by transition learning

Abstract

Transition learning means the short learning period after switching from one learning method to another in this paper. The idea of transition learning is to apply balanced ensemble learning for a certain time, and then to switch to negative correlation learning. Because of the different learning functions between the two methods, the learning behaviors are expected to have a sudden changes in transition learning. Experimental studies had been conducted to examine such learning behaviors in the transition process. It was found that the training error rates jumped immediately in the transition while the testing error rates often appeared to fall slightly. Such large changes in error rates suggested that the decision boundary formed by balanced ensemble learning had been greatly altered in transition learning. This paper presents the explanations of the transition learning from both the ensemble and individual neural network levels.