Orientation and Scale Based Weights Initialization Scheme for Deep Convolutional Neural Networks

Abstract

Image classification is generally about the understanding of information in the images concerned. The more the system able to understand the image contains, the more effective it will be in classifying desired images. Recent work has shown that the convolutional neural network (CNN) paradigm is useful for obtaining more accurate image classification results. A crucial component in the CNN is the convolution filters which consist of a series of predefined filter weight initialization values. The filter weights are then automatically learned by the neural network throughout the back-propagation training algorithm. However, most initialization schemes used in the deep convolutional neural networks are mainly to deal with vanishing gradient problems. Thus, selecting optimal weights are crucial to improve convergence and minimize the complexity which can enhance the generalization performance. One possible solution is to replace the standard weights with parameterized filters that proven to be efficient in extracting useful features such as Gabor filter bank. The Gabor filter bank is popular due to its ability in dealing with spatial transformation, especially on edges and texture information of different scales and directions. Thus, in this paper, we investigate the effect of utilizing Gabor and convolutional filters on small size kernels of deep VGG-16 architecture. The standard VGG-16 filter is replaced with the Gabor filter bank to obtain uniform distribution at all layers of the network. The result shows that the orientation and scale weights initialization scheme outperforms the standard filter weights on an image classification problem.