Maximizing margins of multilayer neural networks

Abstract

According to the CARVE algorithm, any pattern classification problem can be synthesized in three layers without misclassification. In this paper, we propose to train multilayer neural network classifiers based on the CARVE algorithm. In hidden layer training, we find a hyperplane that separates a set of data belonging to one class from the remaining data. Then, we remove the separated data from the training data, and repeat this procedure until only the data belonging to one class remain. In determining the hyperplane, we maximize margins heuristically so that data of one class are on one side of the hyperplane. In output layer training, we determine the hyperplane by a quadratic optimization technique. The performance of this new algorithm is evaluated by some benchmark data sets.