Determining the Structure of Decision Directed Acyclic Graphs for Multiclass Classification Problems

Abstract

An usual strategy to solve multiclass classification problems in Machine Learning is to decompose them into multiple binary sub-problems. The final multiclass prediction is obtained by a proper combination of the outputs of the binary classifiers induced in their solution. Decision directed acyclic graphs (DDAG) can be used to organize and to aggregate the outputs of the pairwise classifiers from the one-versus-one (OVO) decomposition. Nonetheless, there are various possible DDAG structures for problems with many classes. In this paper evolutionary algorithms are employed to heuristically find the positions of the OVO binary classifiers in a DDAG. The objective is to place easier sub-problems at higher levels of the DDAG hierarchical structure, in order to minimize the occurrence of cumulative errors. For estimating the complexity of the binary sub-problems, we employ two indexes which measure the separability of the classes. The proposed approach presented sound results in a set of experiments on benchmark datasets, although random DDAGs also performed quite well.