BreastNet: Entropy-Regularized Transferable Multi-task Learning for Classification with Limited Breast Data

We describe a framework to automatically separate malignant from benign breast lesions using limited breast ultrasound data. The main uniqueness of this framework includes: (1) in terms of the unique shape features of breast lesions, two types of image patches are designed to fine-tune pre-trained models, aiming to characterize the overall appearance and heterogeneity in shapes of breast lesions. (2) taking the BI-RADS regression task as an auxiliary task, a multi-task architecture is proposed to improve the accuracy of classification. (3) instead of prevalent cross-entropy loss, we introduce training with confusion by means of regularizing prediction entropy to prevent overfitting. Extensive experimental results on small-scale breast ultrasound dataset corroborate that the proposed framework is superior to the state-of-the-art approaches in breast lesions classification with limited data. Besides, we provide detailed analysis of the choice of regularizing parameter and visual evidence that introduction of confusion leads to increase in feature generalization.