Score-based Aggregation for Attention Modules in Image Classification Tasks

Deep Convolutional Neural Networks (CNNs) have been widely used for various computer vision tasks because they hierarchically extract bountiful features from a highdimensional image. Also, some CNNs incorporate channel attention mechanisms that re-scale each channel of intermediate feature maps based on their importance. The channel attention modules squeeze the spatial information of a feature into a representative value to transform it as a re-scaling value. In order to reduce the amount of information, attention modules have utilized hand-designed pooling functions such as max pooling or average pooling which have been widely adopted in CNNs, because they add negligible computational complexity. However, a significant amount of spatial information is lost due to these pooling functions. In this paper, we propose a generalized pooling function that scales down spatial information with respect to the importance of each pixel. Unlike max pooling or average pooling, our score-based aggregation is capable of flexibly adjusting to input. Also, the score-based aggregation function learns how to squeeze the spatial information into the must appropriate representative value, which will convert the pooling into a spatial attention mechanism. Finally, we propose a novel method called Score-based Aggregated Attention Module (SAAM) that utilizes the proposed score-based aggregation. Our experimental results on CIFAR-10 and CIFAR-100 datasets demonstrate that SAAM achieves the highest classification accuracy improvement among existing channel attention modules since the score-based aggregation in SAAM is a more dynamic and effective method than the hand-designed aggregations.