An energy-efficient data-reused on-the-fly Im2col for OS-dataflow systolic array-based accelerator

Abstract

The rapid development of AI technology has made the systolic array, capable of massively parallel computations, one of the most commonly used computational architectures for AI inference. The im2col transformation is required to convert the operation into matrix multiplication for convolution, resulting in inflated input data and increased power consumption for data transmission. This work proposes a low-power, on-the-fly im2col scheme based on data reuse. By delaying or preloading certain input rows, redundant data can be aligned in timing, eliminating the need for extensive intermediate buffers in the im2col module. Additionally, this paper discusses the design of the systolic array for irregular input dataflow and provides hardware support for 1 $\times$ 1 and 3 $\times$ 3 convolution modes. The proposed design has been tested and implemented on the ZYNQ UltraScale+ ZU4EV. Experiment results show that the hardware implementation of the im2col module consumes only 610 LUTs and 465 FFs, with a power consumption of just 6.51 mW, demonstrating better power consumption than previous works.