Assessing accuracy for multi-class classification when subclasses are involved.

Abstract

Classifications that involve subclasses are common in many applied fields. "Compound multi-class classification" refers to the settings which involve three or more main classes and at least one of the main classes has multiple subclasses. In this paper, we propose an accuracy metric proper for "compound $M$-class classification," namely "hypervolume under compound $ROC$ manifold $\left(HU{M}_{C,M}\right)$." The proposed $HU{M}_{C,M}$ evaluates the overall accuracy of a biomarker measured on continuous scale correctly identifying $M$ main classes without requiring specification of an ordering in terms of marker values for subclasses relative to each other within each main class. The probabilistic interpretation of $HU{M}_{C,M}$ is analytically derived. A network-based computing algorithm which enables efficient computation of the empirical estimate of $HU{M}_{C,M}$ is developed. Non-parametric bootstrap percentile confidence intervals of $HU{M}_{C,M}$ are assessed through extensive simulation studies. Lastly, a real data example is included to illustrate the usage of our proposed method.