A Novel Cross Grouping CG MLP based on local mechanism

Abstract

Recently, Google proposed the MLP-Mixer – a simple multi-layer fully connected network, proving that convolutional and attention mechanisms are not irreplaceable. Although MLP-Mixer is simple, training it requires a lot of resources. In this paper, a network model——Cross Grouping MLP(CG MLP) based on local mechanism is proposed. The CG MLP module is a general visual task backbone that replaces the original MLP’s spatial mixing module. CG MLP introduces vertical and horizontal bar grouping in different channels of feature map to extract local information. CG MLP also introduces pyramid structure. For the input image, this model reduces the computational complexity of MLP from the square of the area(the fourth power of the side length) to the third power of the side length. CG MLP with 64M parameters achieved 82.5% accuracy on Imagenet-1K, and it reaches the SOTA performance of MLP models.