Active testing for SVM parameter selection

The Support Vector Machine algorithm is sensitive to the choice of parameter settings. If these are not set correctly, the algorithm may have a substandard performance. It has been shown that meta-learning can be used to support the selection of SVM parameters. However, it is very dependent on the quality of the dataset and the meta-features used to characterize the dataset. As alternative for this problem, a recent technique called Active Testing characterized a dataset based on the pairwise performance differences between possible solutions. This approach selects the most useful cross-validation tests. Each new cross-validation test will contribute information to a better estimate of dataset similarity, and thus better predict which algorithms are most promising on the new dataset. In this paper we propose the application of Active Testing for the SVM parameter problem. We test it on the problem of setting the RBF kernel parameters for classification problems and we compare its similarity strategy with based on data characteristics. The results showed the variants of Active Testing that rely on cross-validation tests to estimate dataset similarity provides better solutions than those that rely on data characteristics.