Automatic Decision Making for Parameters in Kernel Method

Abstract

We propose to use the relationship between the parameter of kernel function and its decisional angle or distance metrics for selecting the optimal setting of the parameter of kernel functions in kernel method-based algorithms. Kernel method is established in the reproducing kernel Hilbert space, the angle and distance are two metrics in such space. We analyse and investigate the relationship between the parameter of kernel function and the metrics (distance or angle) in the reproducing kernel Hilbert space. We design a target function of optimization to model the relationship between these two variables, and found that (1) the landscape shapes of parameter and the metrics are the same in Gaussian kernel function because the norm of all the vectors are equal to one in reproducing kernel Hilbert space; (2) the landscape monotonicity of that are opposite in polynomial kernel function from that of Gaussian kernel. The monotonicity of designed target functions of optimization using Gaussian kernel and polynomial kernel is different as well. The distance metric and angle metric have different distribution characteristics for the decision of parameter setting in kernel function. It needs to balance these two metrics when selecting a proper parameter of the kernel function in kernel-based algorithms. We use evolutionary multi-objective optimization algorithms to obtain the Pareto solutions for optimal selection of the parameter in kernel functions. We found that evolutionary multi-objective optimization algorithms are useful tools to balance the distance metric and angle metric in the decision of parameter setting in kernel method-based algorithms.