Pooling Map Adaptation in Convolutional Neural Network for Facial Expression Recognition

Abstract

In this work, we proposed adaptive pooling maps (APMs) for CNNs to aid facial expression recognition. Inspired by superpixels, which represent the image content more naturally, pooling maps consisting of irregular pooling regions are learned from training images as part of training a CNN model. The APMs preserve the local structural information and thus are more capable of capturing subtle facial appearance and geometrical changes caused by facial expression. Furthermore, we developed an efficient algorithm to learn the APMs efficiently. Experiments on three benchmark datasets have shown that the proposed APM-based CNN model outperforms the one with the standard pooling map and achieves state-of-the-art recognition performance for facial expression recognition in the wild.