Fine-grained food image recognition using a convolutional neural network and swin transformer hybrid model

Abstract

With increasing public emphasis on dietary monitoring and quality of life, fine-grained food image recognition has become an important research area in computer vision. However, distinguishing visually similar food items remains challenging, as traditional classification methods often fail to achieve satisfactory accuracy. To address this, this paper proposes a novel CNN-Transformer-based model that integrates convolutional neural networks (CNNs) with attention mechanisms. Specifically, the model introduces a Global Attention and Local Covariance Convolutional Feature Fusion module into the Swin Transformer framework. This module combines a deep convolutional network, a multi-layer perceptron, and a feature fusion component, enabling better capture of fine-grained details while integrating global context. Extensive experiments conducted on two public fine-grained food image datasets, FoodX-251 and UEC Food-256, demonstrate the superior performance of the proposed model. It achieves accuracy rates of 81.47 % and 83.44 %, respectively, outperforming most existing methods under the same experimental conditions.