Penalized Entropy: a novel loss function for uncertainty estimation and optimization in medical image classification

In medical image classification, uncertainty estimation providing confidence of decision is part of interpretability of prediction model. Based on estimated uncertainty, physicians can pick out cases with high uncertainty for further inspection. However, in this uncertainty-informed decision referral, models may make wrong predictions with high certainty which leads to omission of false predictions. Therefore, we propose a method to set up a model which could make correct prediction with low uncertainty and wrong prediction with high uncertainty. We integrate uncertainty estimation into training phase and design a novel loss function “penalized entropy” by penalizing wrong but certain samples to improve the models' certainty performance. Experiments were conducted on three datasets: optical coherence tomography (OCT) image dataset for anti-vascular endothelial growth factor (anti- VEGF) effectiveness classification, OCT image dataset for diagnostic classification, and chest X-ray dataset for pneumonia classification. Performances were evaluated on both accuracy metrics such as accuracy, sensitivity, specificity, area under the curve (AVC), and certainty metrics which are accuracy vs. uncertainty (AvV), probability of correct results among certain predictions (PCC), and probability of uncertain results among wrong predictions (PUW). Results show that the method using the proposed loss function can achieve better or comparable accuracy and state-of-the-art certainty performance.