Companding quantization for deep neural networks

Abstract

Deep learning models typically require substantial computational resources and exhibit large model sizes to achieve high accuracy, presenting challenges for deployment on resource-constrained hardware, such as mobile and edge devices. To mitigate these challenges, various strategies have been proposed, including model compression techniques aimed at reducing the precision of weights and activations. Among these, quantization stands out as a prevalent method that significantly reduces model size; however, it often results in an accuracy loss. This work focuses on narrowing the accuracy gap between full-precision models and their low-bit quantized counterparts. A comprehensive investigation is presented into the application of $\mu$-law companding for the quantization of diverse neural network architectures. The research explores the impact of optimizing the $\mu$ parameter within the companding framework used to quantize weights and activations. Specifically, three different configurations for the $\mu$ parameter are examined: a single $\mu$ for all weights and activations, separate $\mu$ values for weights and activations, and individual $\mu$ parameters for each layer’s weights and activations. A comprehensive analysis was conducted to compare two distinct arrangements of these composite blocks, focusing on their performance metrics and overall effectiveness. Extensive experiments were conducted utilizing benchmark models, including VGG19 and AlexNet, tested on the CIFAR-10 and CIFAR-100 datasets. The results illustrate significant enhancements in top-1 prediction accuracy when compared to traditional quantization methods, with some configurations surpassing the accuracy of the full-precision model by as much as 0.11 %. For a 4-bit quantized model applied to the CIFAR-10 dataset, the loss was maintained below 1 %.