A New Approach to Classification with the Least Number of Features

Recently, the so-called Support Feature Machine (SFM) was proposed as a novel approach to feature selection for classification, based on minimisation of the zero norm of a separating hyper plane. We propose an extension for linearly non-separable datasets that allows a direct trade-off between the number of misclassified data points and the number of dimensions. Results on toy examples as well as real-world datasets demonstrate that this method is able to identify relevant features very effectively.