BeaCIM: A Digital Compute-in-Memory DNN Processor With Bi-Directional Exponent Alignment for FP8 Training

Abstract

Previous digital Compute-In-Memory (DCIM) exhibits limitations in Deep Neural Network (DNN) training with FP8, which is playing an increasingly important role in model training. Most of the previous DCIM purely align exponents to the maximum one when computing floating-point data. This method struggles to balance accuracy and energy efficiency when using FP8. In this brief, we propose BeaCIM, a DCIM processor with bi-directional exponent alignment. There are several contributions. First, we propose a new exponent alignment mechanism, which can dynamically adjust the shared exponent $(E^{*})$ towards the numerical distribution center of exponents. Second, we use the Shift-on-Product (SOP) method to address the limitation of data bitwidth, and present $E^{*}$ calculator based on the Ordinary Least Squares (OLS). Finally, we performed RTL-level circuit implementation and evaluated BeaCIM using various datasets and models. Our experiments show the following results: (1) Compared with the standard hybrid FP8 training, our bi-directional exponent alignment FP8 training exhibits an average top-1 accuracy drop less than 0.75% across different models and datasets. (2) BeaCIM can achieve an energy efficiency of 31.5 TFLOPS/W at FP8, which is 2.6- $19.3\times $ better than state-of-the-art works.