Studies on classification models using decision boundaries

Abstract

A classification model is obtained after a classifier is trained on training data. Decision region is the region in which data are predicted the same class label by a classifier. Decision boundary is the boundary between regions of different classes. We view classification as dividing the data space into decision regions. The formal definitions of decision region and decision boundary are presented in this paper, and then the relationship between classification models and decision boundaries are studied. We present the analytical expressions of decision boundaries of four typical classifiers, which are C4.5 algorithm, back propagation (BP) neural network, naive Bayes classifier and support vector machine (SVM). Comparative experiments are performed to illustrate different decision boundaries of these four classifiers. Decision boundaries of ensemble learning are discussed. The concept of probability gradient region is introduced for probability based classifiers, and SOMPGRV algorithm is proposed for visualizing probability gradient regions.