Feature selection for pattern recognition by LASSO and thresholding methods - a comparison

Abstract

For high-dimensional data processing, like pattern recognition, it seems desirable to precede with a reduction of the number of describing features. Our aim is a comparison of various feature selection methods for pattern recognition. We consider two-class supervised classification problem for signals decomposed in wavelet bases. We test kNN classification rule with soft and hard thresholding, performed in two stages: (1) wavelet detail coefficient thresholding (noise reduction) and (2) searching for the most differentiating coefficients between classes (selection of discriminating coefficients). We present a new classification rule based on LARS/LASSO. We compare criteria for L1-norm regularization of wavelet coefficients: AIC, BIC and the thresh derived for kNN rule. There were performed simulations for noisy signals with SNR in the range from 0 to 22 [dB], approximated for all possible wavelet resolutions. The quality of pattern recognition for the presented algorithms was measured by the estimated recognition risk and the size of reduced model.