ADCIM: scalable construction of approximate digital compute-in-memory MACRO for energy-efficient attention computation

Abstract

Digital compute-in-memory (DCIM) performs energy-efficient computation without accuracy loss, which has been proven to be a promising way to break the memory wall commonly existing in Transformer accelerators with von Neumann architecture. Approximate computing is also widely utilized to boost computation efficiency by exploiting error tolerance in neural networks. In this paper, we perform algorithm-hardware co-optimization to incorporate approximate multiplication into original full-precision DCIM, resulting in a more energy-efficient computing paradigm. First, a coarse-grained error compensation method is proposed to balance the error of partial product generation and partial product reduction, achieving almost zero mean error during multiplication operations. Secondly, a fine-grained error compensation is developed for accumulation operations, further suppressing the error of multiply-and-accumulate by 2-3 orders of magnitude. Additionally, based on the proposed approximate algorithm design, the structure of Static Random Access Memory (SRAM) cell is fully exploited to implement efficient approximate digital compute-in-memory (ADCIM), which can be scaled to different bit-widths. Finally, one value-adaptive error controller is utilized to match the error tolerance of the self-attention mechanism and enhance computation efficiency. The proposed ADCIM has been verified on Transformer models with different quantization precisions, and obtains peak energy efficiency of 14.91 tera-operations per second per watt (TOPS/W) @ 16-bit, 22.84 TOPS/W @ 12-bit, and 39.89 TOPS/W @ 8-bit, with negligible accuracy loss.