An Accuracy-Preserving Neural Network Compression via Tucker Decomposition

Abstract

Deep learning has made remarkable progress across many domains, enabled by the capabilities of over-parameterized neural networks with increasing complexity. However, practical applications often necessitate compact and efficient networks because of device constraints. Among recent low-rank decomposition-based neural network compression techniques, Tucker decomposition has emerged as a promising method which effectively compresses the network while preserving the high-order structure and information of the parameters. Despite its promise, designing an efficient Tucker decomposition approach for compressing neural networks while maintaining accuracy is challenging, due to the complexity of setting ranks across multiple layers and the need for extensive fine-tuning. This paper introduces a novel accuracy-aware network compression problem under Tucker decomposition, which considers both network accuracy and compression performance in terms of parameter size. To address this problem, we propose an efficient alternating optimization algorithm that iteratively solves a network training sub-problem and a Tucker decomposition sub-problem to compress the network with performance assurance. The proper Tucker ranks of multiple layers are selected during network training, enabling efficient compression without extensive fine-tuning. We conduct extensive experiments, implementing image classification on five neural networks using four benchmark datasets. The experimental results demonstrate that, without the need for extensive fine-tuning, our proposed method significantly reduces the model size with minimal loss in accuracy, outperforming baseline methods.