A Feature Selection Based on Minimum Upper Bound of Bayes Error

Abstract

This paper presents a novel feature selection scheme based on the upper bound of Bayes error under normal distribution for the multi-class dimension reduction problem. The upper bound of Bayes error in the multi-class problem is represented by the sum of the upper bound of Bayes error of every two-class pair. In order to obtain an accurate solution of the feature selection transform matrix in term of the minimum upper bound of Bayes error, a recursive algorithm based on gradient method is developed. The principal component analysis (PCA) is used as a pre-processing to reduce the intractably heavy computation burden of the recursive algorithm. The superior experimental results on the handwritten digit recognition with the MNIST database demonstrate the effectiveness of our proposed method