Ensemble modeling through multiplicative adjustment of class probability

Abstract

We develop a new concept for aggregating items of evidence for class probability estimation. In Naive Bayes, each feature contributes an independent multiplicative factor to the estimated class probability. We modify this model to include an exponent in each factor in order to introduce feature importance. These exponents are chosen to maximize the accuracy of estimated class probabilities on the training data. For Naive Bayes, this modification accomplishes more than what feature selection can. More generally, since the individual features can be the outputs of separate probability models, this yields a new ensemble modeling approach, which we call APM (Adjusted Probability Model), along with a regularized version called APMR.