Maximum Bayes Boundary-Ness Training For Pattern Classification

Abstract

The ultimate goal of pattern classifier parameter training is to achieve its optimal status (value) that produces Bayes error or a corresponding Bayes boundary. To realize this goal without unrealistically long training repetitions and strict parameter assumptions, the Bayes Boundary-ness-based Selection (BBS) method was recently proposed and its effectiveness was clearly demonstrated. However, the BBS method remains cumbersome because it consists of two stages: the first generates many candidate sets of trained parameters by carefully controlling the training hyperparameters so that those candidate sets can include the optimal target parameter set; the second stage selects an optimal set from candidate sets. To resolve the BBS method's burden, we propose a new one-stage training method that directly optimizes a given classifier parameter set by maximizing its Bayes boundary-ness or increasing its accuracy during Bayes error estimation. We experimentally evaluate our proposed method in terms of its accuracy of Bayes error estimation over four synthetic or real-life datasets. Our experimental results clearly show that it successfully overcomes the drawbacks of the preceding BBS method and directly creates optimal classifier parameter status without generating too many candidate parameter sets.