Relaxed naïve Bayesian classifier based on maximum dependent attribute groups

Abstract

The utilisation of effective dependent attribute groups (DAGs) can benefit the construction of a high-performance naïve Bayesian classifier (NBC), to alleviate the conditional independence assumption of naïve Bayes. An NBC with optimised DAGs retains the simple NBC structure and significantly enhances NBC generalisation performance. However, it is extremely difficult to determine the appropriate DAGs for a given dataset when training an NBC with good generalisation capability. Therefore, this study proposes a relaxed NBC (RNBC) based on the maximum DAGs (MDAGs), that relaxes the attribute independence assumption by constructing an NBC with a series of MDAGs generated from the original condition attribute set. To determine the MDAGs, the RNBC includes an effective objective function to determine the degree of membership of conditional attributes belonging to different DAGs. Unlike the regular computation of class-conditional probability in traditional NBCs with whole condition attributes, the RNBC calculates multiple class-conditional probabilities corresponding to non-overlapping MDAGs and their products are utilised to construct the classification system. Exhaustive experiments were conducted to systematically verify the feasibility, rationality, and effectiveness of RNBC. The results demonstrate that (1) the objective function used to determine the MDAGs is convergent, and that MDAGs can be obtained with low time consumption; (2) the RNBC with MDAGs achieves a lower classification risk than traditional NBCs with the independence assumption; and (3) the RNBC achieves statistically higher training/testing accuracy and probability estimation quality with lower classification risk compared with eight representative Bayesian classifiers spanning 22 benchmark datasets. The best average testing accuracy, probability mean square error, and area under the curve for the RNBC were 0.76, 0.35, and 0.85, respectively. These results systematically confirmed that the proposed RNBC is an efficient NBC variant with high structural stability, strong correlation expression, and good generalisability.