Adaptive Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition

Fine-grained recognition is still a difficult task in pattern recognition applications due to the challenge of accurate localization of discriminative parts. Recent CNN-based methods generally utilize attention mechanism to produce attention masks without part labels/annotations and extract corresponding image parts from them. However, these methods extract the attention parts by using fixed-size rectangles to crop images regardless of the size of objects to be recognized, which will hinder the feature expression of the following Part-CNNs. In this paper, we propose an adaptive cropping module based on the information of attention masks to adjust size of cropping rectangles. The trainingprocessofadaptivecroppingmoduleandPart-CNNscan reinforce each other with the proposed rank loss and the classic softmax loss. To further balance and fuse all attention parts, we propose a part weighting module to evaluate part contributions. Under the optimization of sort loss, the part weighting module will produce part weights in the same order as prediction scores learned by attention parts. The backbone of our network is MA-CNN. Different from MA-CNN, the new proposed adaptive cropping module and part weighting module can jointly guide the framework to produce a more discriminative fine-grained feature. Experiments show that the AMA-CNN outperforms MA-CNN by 1.1% on CUB200-2011 bird dataset.