Effect of separate sampling on classification and the minimax criterion

Abstract

It is commonplace in bioinformatics (and elsewhere) to build a classifier from sample data in which the sample sizes of the classes are not random; that is, they are selected prior to sampling. The result is that there is no estimate of the prior class probabilities available from the data. In this paper, we find an analytic result for the minimax solution for the class prior probabilities for a general Neyman-Pearson induced classifier. From that we derive Anderson's classical minimax prior probability “estimate.” Using synthetic and real data, we demonstrate the degradation in classifier performance from using inaccurate values for the prior probabilities.