Nested compression of convolutional neural networks with Tucker-2 decomposition

Abstract

The topic of convolutional neural networks (CNN) compression has attracted increasing attention as new generations of neural networks become larger and require more and more computing performance. This computational problem can be solved by representing the weights of a neural network with low-rank factors using matrix/tensor decomposition methods. This study presents a novel concept for compressing neural networks using nested low-rank decomposition methods. In this approach, we alternately perform decomposition of the neural network weights with fine-tuning of the network. The numerical experiments are performed on various CNN architectures, ranging from small-scale LeNet-5 trained on the MNIST dataset, through medium-scale ResNet-20, ResNet-56, and up to large-scale VGG-16, VGG-19 trained on the CIFAR-10 dataset. The obtained results show that using the nested compression, we can achieve much higher parameter and FLOPS compression with a minor drop in classification accuracy.