Multi-Part Knowledge Distillation for the Efficient Classification of Colorectal Cancer Histology Images

Abstract

Colorectal cancer is the most common type of cancer after breast cancer in women and third in men after lungs and prostrate cancer. The disease rank third in incidence and second in terms of mortality, hence early diagnosis is necessary for the correct line of treatment. Knowledge distillation based models boost the performance of small neural network and are performing efficiently for various image classification based tasks. In this work, a novel knowledge distillation based technique is developed to efficiently classify colorectal cancer histology images. Unlike traditional distillation, out method performs distillation in parts. Instead of supervising the student with a converged knowledge of teacher, the proposed method is fetching the teacher's knowledge at regular intervals and providing these knowledge to the student model during student training process. Through this multi-part distillation technique student can effectively learn the intermediate representational knowledge rather than the abstract knowledge of the teacher and hence boost the overall performance of the model. The the proposed model has achived 92.10% accuracy.