Deep Learning with Maxout Activations for Visual Recognition and Verification

Visual recognition is one of the most active research topics in computer vision due to its potential applications in self-driving cars, healthcare, social media, manufacturing, etc. For image classification tasks, deep convolutional neural networks have achieved state-of-the-art results, and many activation functions have been proposed to enhance the classification performance of these networks. We explore the performance of multiple maxout activation variants on image classification, facial recognition and verification tasks using convolutional neural networks. Our experiments compare rectified linear unit, leaky rectified linear unit, scaled exponential linear unit, and hyperbolic tangent to four maxout variants. Throughout the experiments, we find that maxout networks train relatively slower than networks comprised of traditional activation functions. We found that on average, across all datasets, rectified linear units perform better than any maxout activation when the number of convolutional filters is increased six times.