Support of Sparse Tensor Computing for MLIR HLS

Abstract

Nowadays, sparse tensor computations are widely used in machine learning. Without the multiplications in zero values, sparse tensor computation can significantly reduce the latency and power consumption. Famous frameworks like TensorFlow, PyTorch, Pandas, etc., already have the support of sparse tensor computing. MLIR also integrated this idea and implemented the compilation flow. Integrating sparse tensor computing and MLIR into High-level Synthesis (HLS) can generate more powerful RTL and further implement specified hardware. However, MLIR flow isn’t well done now while translating into LLVM IR and does not fully support HLS tools, which are not supporting MLIR. In this paper, we propose a flow in MLIR to lower sparse tensor computations into HLS-readable LLVM IR, which can then be synthesized into RTL. To demonstrate the effectiveness of our proposed flow, we devise experiments by implementing matrix multiplication operations in the convolution layers, and the data is pruned to maximize the sparsity. Our proposed flow speeds up the latency by about 3.6 times, as demonstrated in our experiment with Xilinx Vitis HLS.