A Neural Network Binary Quantization Method Based on W-Regularization and Variable Cosine Momentum

Abstract

To solve the problem of insufficient extraction of weight information in binary quantization, this paper proposes a new training module based on W-regularization and variable cosine momentum. W-regularization is achieved by adjusting the network weights so that the weight values are optimised to ±1 and the parameters at different positions are optimised according to different functions. In addition, variable cosine momentum is designed so that parameters farther away from ±1 approach zero at high speed, which can significantly increase the speed of convergence and further improve quantization accuracy. Specifically, it outperforms the highest accuracy of bnn-free by 0.83% and 2.15% on the CIFAR-10, CIFAR-100 datasets, it also improved on both SVHN and TinyImage.