From ensemble to knowledge distillation: Improving large-scale food recognition

Abstract

Food recognition on a large scale presents significant challenges due to high intra-category similarity and inter-category variability. Addressing these challenges is crucial for developing robust and accurate food recognition systems, which have applications in health monitoring, dietary assessment, and automated food logging. This study aims to tackle these issues by employing ensemble learning and knowledge distillation. We use ensemble learning to effectively combine the local perception capability of convolutional neural networks (CNNs) and the global modeling capability of Vision Transformers. The synergistic ensemble enhances the model's ability to discern subtle differences within categories and capture a spectrum of diverse patterns across various categories. To reduce the number of base models in an ensemble, we employed a method combining knowledge distillation and re-ensembling. Specifically, we used the collective knowledge of four base models to guide the re-learning process of student models. Subsequently, we re-ensembled these distilled models, significantly enhancing the recognition performance of the ensemble while maintaining the same computational efficiency. Finally, we fine-tuned the optimal ensemble weights to further boost the recognition performance of the ensemble model. We conducted extensive experiments on the large-scale food datasets Food2k and CNFood241, achieving state-of-the-art performance. Specifically, on the Food2k dataset, our method achieved a top-1 accuracy of 86.22 % with 131.56M parameters, outperforming the state-of-the-art algorithms by 2.1 %, demonstrating its effectiveness.