Transformer: Image Classification Based on Constitutional Neural Networks

Abstract

To solve the problem of excessive calculation caused by inputting images with a large size when using ViT network structure to implement image classification tasks, this paper proposes a ViT network model based on a convolutional neural network (CNN). Its network structure first uses CNN to extract a low-resolution feature map and then uses ViT structure to process the low-resolution feature map. At this time, the computational pressure is greatly relieved. In this paper, the author uses VGG16 as the Backbone and ViT network structure to build the VGG16-TE network and implements an image classification task on the ImageNet-1k dataset. Compared with the VGG16 model, the accuracy of Top1 and Top5 image classification is improved by 2.5 points and 1.7 points respectively. Besides, this paper builds a ResNet34-TE network with ResNet34 as the Backbone and ViT network and implements an image classification task on the ImageNet-1k dataset. Compared with the ResNet34 model, the accuracy of Top1 and Top5 image classification is improved by 2.1 points and 1.2 points respectively. VGG16-TE and ResNet34-TE parameters decrease by 68M and 61.5M compared with that of the ViT-Base model.