Fine-grained hierarchical singular value decomposition for convolutional neural networks compression and acceleration

Abstract

Convolutional neural networks (CNNs) still remain crucial in the field of computer vision, especially in industrial-embedded scenarios. Although modern artificial intelligence chips such as embedded graphics processing units (GPUs) and neural process units (NPUs) are equipped with sufficient computability, making CNNs more lightweight always has non-negligible significance. Until now, many researchers have made multiple corresponding achievements, in which a series of tensor decomposition methods have represented their unique advantages such as concision, flexibility, and low-rank approximation theory. However, balancing the compression, acceleration, and precision, is still an open issue, because the traditional tensor decompositions are hard to deal with the trade-off between approximation and compression ability, while the so-called fine-grained tensor decompositions such as Kronecker canonical polyadic (KCP) have not created a way to merge the factors for efficient inference. In this paper, we first review related works on convolutional neural network (CNN) compression and the necessary prior knowledge. We then propose a novel matrix decomposition method, termed hierarchical singular value (HSV) decomposition, and validate its effectiveness. Subsequently, we introduce a fast contraction strategy based on the merged factors of HSV and explain how our method addresses the inefficiencies in inference associated with traditional contraction processes. Additionally, we validate the advantages of HSV by comparing its complexity with that of other classical tensor decomposition methods. Thereafter, we apply HSV to CNN compression and acceleration by transforming convolution operations into matrix multiplication. We also propose a self-adaptive rank selection algorithm tailored to standard CNN architecture and conduct a theoretical analysis of the convergence of our method. Multiple experiments on CIFAR-10, ImageNet, COCO, and Cityscapes benchmark datasets show that the proposed HSV-Conv can simultaneously gain considerable compression ratio and acceleration ratio, while the precision loss is almost non-existent. We also make a comprehensive comparison with the other related works, and the superiority of our method is further validated. Besides, we give a deep discussion about the rank selection issue of HSV in the aspects of practice and theory, which explains the strategy of the proposed self-adaptive rank selection and the reason for choosing fine-tuning rather than training from scratch.