A High-SNR SRAM-LUT-Based In-Memory-Computing Macro Supporting Energy-Efficient INT8 and FP8 Computation

Abstract

Recent quantized neural networks require various dynamic ranges and resolutions to enable better inference accuracy, necessitating mixed support of energy-efficient integer and floating point computation. This work presents an SRAM look-up table (LUT) based INT8/FP8 dual mode in-memory-computing (IMC) macro which supports configurable FP8 (E4M3 and E5M2) multiply-accumulate (MAC) operation and 2:8 structural sparsity pruning. Instead of suffering either excessive overheads of shifting and alignment logic or accuracy degradation for extreme energy efficiency, this macro exploits fine-grain exponent sharing and efficient LUT-based MAC circuits to balance the two aspects. The key features are: 1) a pipelined and SNR-oriented mantissa and exponent execution circuit for FP8 multiplication; 2) a LUT-based shifter and 1-stage adder to reduce digital logic overhead in FP MAC; 3) an in-macro decoder to support 2:8 structural sparsity pruning. The silicon prototype in 28nm CMOS achieves 81.69% accuracy for DeiT-base model for ImageNet dataset in dense FP8 mode, and 76.15% accuracy for ResNet-34 model in CIFAR100 dataset in sparse FP8 mode. The 2:8 pruning provides a 31.42 TFLOPS/W peak energy efficiency and 3.56 TFLOPS/mm² peak area efficiency in sparse FP8 mode.