A Simple Gaussian Kernel Classifier with Automated Hyperparameter Tuning

This paper establishes a fitting method for a kernel logistic regression model that uses generalized Gaussian kernel and its parameter optimization method. Kernel logistic regression is a classification model that uses kernel methods effectively. This is one of the methods to construct an effective nonlinear system with a reproducing kernel Hilbert space (RKHS) induced from positive semi-definite kernels. Most classifiers that are combined with Gaussian kernel functions generally assume uncorrelatedness within the feature vectors. Thus, the Gaussian kernel consists of only two parameters (namely, mean and precision). In this paper, we propose a model using a generalized Gaussian kernel represented flexibly in each dimension of feature vector. In addition, the parameters of kernel are fully data-driven. For the fitting of proposed model, an ℓ1-regularization is introduced to supress the number of support vectors. A numerical experiment showed that the classification performance of the proposed model is almost the same as RBF-SVM even though the proposed model has a small number of support vectors.