Improving DNN Accuracy on MLC PIM via Non-Ideal PIM Device Fine-Tuning

Abstract

Resistive random access memory (RRAM) emerges as a promising technology for developing energy-efficient deep neural network (DNN) accelerators, owing to its analog computing paradigm for matrix-vector multiplication. However, the inherent nonideal device features of RRAM cells, such as device variation, read disturbances, and limited on/off ratio, present challenges for model deployment. Therefore, to ensure accurate storage and computing precision for RRAM-based accelerators, a widely used practice is encoding a DNN weight by multiple cells, resulting in significant memory overhead and underutilization. This challenge is further exacerbated by the rapid increases in model size witnessed in recent years. While the one-to-one weight-cell mapping strategy can improve memory utilization, it inevitably introduces deviations in the mapped DNN weight from the desired value due to RRAM variation issues, leading to model accuracy degradation. In response to this challenge, we abstract the model optimization on RRAM chips as a non-ideal PIM device optimization problem, aimed at optimizing model accuracy without the requirement of precise weight programming. We systematically analyze the model optimization behavior on multilevel RRAM devices by investigating the accuracy recovery process of various fine-tuning strategies in recovering model performance under the non-ideal PIM device setting. Based on the analysis, we propose a non-ideal PIM device finetune scheme to recover the model performance for multilevel RRAM under the non-ideal PIM device setting. Our proposed scheme leverages knowledge distillation and exploits input/output information of the model on RRAM to guide the fine-tuning process, finally restoring its accuracy. Experimental results demonstrate the efficacy of our non-ideal PIM device fine-tuning scheme, achieving nearly complete recovery of model performance. Our approach yields over a 3% improvement in model accuracy compared to variation-aware training approaches.