IDWA: A Importance-Driven Weight Allocation Algorithm for Low Write–Verify Ratio RRAM-Based In-Memory Computing

Abstract

Resistive random access memory (RRAM)-based in-memory computing (IMC) architectures are currently receiving widespread attention. Since this computing approach relies on the analog characteristics of the devices, the write variation of RRAM can affect the computational accuracy to varying degrees. Conventional write–verify (W&V) procedures are performed on all weight parameters, resulting in significant time overhead. To address this issue, we propose a training algorithm that can recover the offline IMC accuracy impacted by write variation with a lower cost of W&V overhead. We introduce a importance-driven weight allocation (IDWA) algorithm during the training process of the neural network. This algorithm constrains the values of less important weights to suppress the diffusion of variation interference on this part of the weights, thus reducing unnecessary accuracy degradation. Additionally, we employ a layer-wise optimization algorithm to identify important weights in the neural network for W&V operations. Extensive testing across various deep neural networks (DNNs) architectures and datasets demonstrates that our proposed selective W&V methodology consistently outperforms current state-of-the-art selective W&V techniques in both accuracy preservation and computational efficiency. At same accuracy levels, it delivers a speed improvement of $6\times \sim 32\times $ compared to other advanced methods.