ReCIM: A SRAM-Based Digital–Analogue Hybrid CIM Reformer Accelerator Macro

Reformer reduces redundant self-attention computations via hash bucketing. In this study, we introduce a SRAM-based digital-analogue hybrid reformer computing-in-memory (ReCIM) accelerator macro. This macro presents an absolute maximum value addressing circuit which facilitates the hash bucketing process and enables the utilisation of strongly-correlated (S-C) vectors for attention mechanism computations, thereby improving computational efficiency and saving memory space. Additionally, we introduce a reusable weight array which is suitable for matrix operations across various processes of self-attention, minimising unnecessary area overhead and enhancing device reusability. The proposed 4 Kb ReCIM macro was analysed using 28-nm CMOS technology. Simulation results demonstrate that the macro achieves a frequency of 500 MHz at a supply voltage of 0.9 V. During the hash bucketing process, energy efficiency reaches 9.74 TOPS/W.