Building a Diverse Ensemble for Classification

Abstract

Pattern recognition systems are widely used in a host of different fields. Due to some reasons such as lack of knowledge about a method based on which the best classifier is detected for any arbitrary problem, and thanks to significant improvement in accuracy, researchers turn to ensemble methods in almost every task of pattern recognition. Classification as a major task in pattern recognition, have been subject to this transition. The classifier ensemble which uses a number of base classifiers is considered as meta-classifier to learn any classification problem in pattern recognition. Although some researchers think they are better than single classifiers, they will not be better if some conditions are not met. The most important condition among them is diversity of base classifiers. Generally in design of multiple classifier systems, the more diverse the results of the classifiers, the more appropriate the aggregated result. It has been shown that the necessary diversity for the ensemble can be achieved by manipulation of dataset features, manipulation of data points in dataset, different sub-samplings of dataset, and usage of different classification algorithms. We also propose a new method of creating this diversity. We use Linear Discriminante Analysis to manipulate the data points in dataset. Although the classifier ensemble produced by proposed method may not always outperform all of its base classifiers, it always possesses the diversity needed for creation of an ensemble, and consequently it always outperforms all of its base classifiers on average.