Support Vector Machine Pre-pruning Approaches on Decision Trees for Better Classification

Abstract

Incorporation of the structural risk minimization of Support Vector Machine to pre-prune the decision trees based on empirical risk minimization is conducted to develop a combined algorithm. It is named as Support Vector Machine Pruned Decision Trees (SVMPDT) algorithm. Pre-pruning of decision trees (DT) is applied to the datasets through the synergistically adjusted regularization parameter of SVM. This is done by the proposed new approach derived from the study on the synergy effects between the pre-pruning weighting fraction of DT and the regularization parameter of SVM. The regularization parameter of SVM is customized and adjusted based on the different features and characteristics of DT from each applied dataset. After applying the proposed algorithms to the assigned datasets, it is shown to be more accurate in classification when compared with typical SVM without getting its parameter adjusted accordingly and the typical DT classification without applying pre-pruned weighting fraction as well as the default SVMDT algorithms without getting the DT to be pre-pruned. This is because its regularization parameter of SVM can be optimally adjusted with the newly proposed formulations on the pre-pruned weighting fraction of DT in a synergy way such that the classification accuracies can significantly be improved.