Pattern classification using support vector machine ensemble

While the support vector machine (SVM) can provide a good generalization performance, the classification result of the SVM is often far from the theoretically expected level in practical implementation because they are based on approximated algorithms due to the high complexity of time and space. To improve the limited classification performance of the real SVM, we propose to use an SVM ensemble with bagging (bootstrap aggregating) or boosting. In bagging, each individual SVM is trained independently, using randomly chosen training samples via a bootstrap technique. In boosting, each individual SVM is trained using training samples chosen according to the sample's probability distribution, which is updated in proportion to the degree of error of the sample. In both bagging and boosting, the trained individual SVMs are aggregated to make a collective decision in several ways, such as majority voting, least squares estimation based weighting, and double-layer hierarchical combination. Various simulation results for handwritten digit recognition and fraud detection show that the proposed SVM ensemble with bagging or boosting greatly outperforms a single SVM in terms of classification accuracy.