Distribution Consistency Penalty in the Quadratic Kappa Loss for Ordinal Regression of Imbalanced Datasets

Abstract

Ordinal regression is a typical deep learning problem, which involves inherently ordered labels that are common in practical applications, especially in medical diagnosis tasks. To overcome the neglect of ordered or non-stationary property by merely exploiting classification or regression, quadratic weighted kappa (QWK) is proposed to be employed in the QWK loss function design as an efficient evaluation metric for ordinal regression. However, the paradox that kappa will be higher with an asymmetrical marginal histogram leads the QWK loss function to get the local optimal solution with all-zero-column in the confusion matrices during training. In practice, the all-zero column problem will result in a certain category not being detected at all, which can have serious consequences for the exclusion of pathology. To address this limitation, a new form of penalty term is proposed for the QWK loss function by penalizing the distance of marginal histogram to effectively avoid all-zero-column of the models. The experiments on the category-imbalanced datasets demonstrate that our penalty terms solve all-zero-column problem. On Adience dataset our penalty terms achieve 0.915 QWK, 0.446 MAE and 0.612 accuracy, while on DR dataset our penalty terms achieve 0.744 QWK, 0.281 MAE and 0.810 accuracy. Besides, experiments on the category-balanced datasets HCI show that our penalty terms achieve 0.810 QWK, 0.499 MAE and 0.610 accuracy.