DIMQ: Distribution-Induced Multi-Peak Quantization for Deep Neural Networks

Abstract

Model quantization is critical for deploying deep neural networks on resource constrained devices. However, conventional quantization schemes typically rely on passive adaptation to inherent weight distributions, resulting in significant quantization noise and codebook collapse due to the structural mismatch with discrete levels. To address this limitation, we propose a weight-side Distribution-Induced Multi-peak Quantization (DIMQ) framework that actively reshapes weight distributions to align with the quantizer topology. DIMQ employs an entropy-regularized soft assignment to progressively organize weights into compact and well-separated peaks, together with a separation constraint that prevents codebook collapse under limited bit budgets. The resulting objective integrates task loss with a differentiable distortion surrogate, facilitating the simultaneous and stable optimization of quantization centers and weight structures via closed-form Gibbs assignment and gradient-stabilized center optimization. Theoretically, we derive rigorous distortion bounds proving that the induced multi-peak distribution yields strictly lower quantization error than gaussian and refined uniform baselines under specific variance conditions. Extensive experiments on ImageNet and COCO benchmarks demonstrate that DIMQ significantly outperforms baselines on CNN architectures and achieves highly competitive accuracy on Transformer models, particularly demonstrating superior robustness in extremely low bit regimes.