Hybrid CNNs: A Rotation Equivariant Framework for High Resolution Spherical Images

Abstract

With the prevalence of virtual reality, augmented reality and autonomous robots, the high resolution spherical images they produced make the standard convolutional neural networks (CNNs), which have been proven powerful on perspective images, non-trivial. The classic solution to utilize CNNs on spherical images is to project the spherical images onto plane and learning the planar images using conventional CNNs. But the distortion generated by the projection of spherical images to planar images invalidates the projection based models. Besides, these models are not robust to rotations which are the basic transformation of spherical images. Another type of solution based on spherical harmonics recently proposed by Cohen et al. [1] is rotation equivariant, but can't handle high resolution spherical images with its expensive computational cost. To process high resolution spherical images, we proposed the Hybrid CNNs. Our framework is both computationally efficient and rotation equivariant with two kinds of convolution operations defined in this paper. We compared our method with several baseline models in two classification tasks. The experimental results demonstrate the computational efficiency and rotation equivariance of the Hybrid CNNs.