3D CNN Acceleration using Block Circulant Matrix in Frequency Domain

Abstract

Recently, 3D CNNs have proven to be outstanding in applications such as video analysis, 3D geometric data, and 3D medical image diagnosis. However, the forbidden storage and computational overhead of 3D CNNs hinder their deployment on edge devices. To tackle this challenge, we propose 3D FCirCNN, an algorithm-hardware co-design approach to realize efficient 3D CNNs acceleration. At the algorithm level, 3D FCirCNN applies block circulant matrix to compress 3D CNNs for the first time and further accelerates the computation with Fast Fourier Transform (FFT), significantly reducing the storage and computational overhead of 3D CNNs. Besides, to avoid the extra computation overhead caused by frequent spatial/frequency domain switching, we introduce several 3D CNN operators in the frequency domain, thus realizing the full frequency domain computation. At the hardware level, we design an FPGA-based dedicated hardware architecture to accelerate 3D FCirCNN. Experiments on the Xilinx ZCU102 demonstrate that with only an acceptable accuracy loss, 3D FCirCNN can achieve a performance of 2692.81 GOPS with a 3.349 GOPS/DSPs efficiency, which outperforms prior works significantly.