A Configurable FPGA Accelerator of Bi-LSTM Inference with Structured Sparsity

To deploy Bi-directional Long Short-Term Memory (Bi-LSTM) on resource-constrained embedded devices, this work presents a configurable FPGA-based Bi-LSTM accelerator enabling structured compression. Firstly, a dense Bi-LSTM model is thoroughly slimed by a hybrid quantization scheme and a structured top-k pruning. Secondly, the energy consumption on external memory access is significantly reduced by the proposed row-reuse computing pattern. Finally, the proposed accelerator is capable of handling a structured sparse Bi-LSTM model benefitting from the algorithm-hardware co-design workflow. It is also flexible to perform inference tasks on Bi-LSTM models with any feature dimension, sequence length, and number of layers. Implemented on the Intel Cyclone V SXC5 SoC FPGA platform, the proposed accelerator can achieve 189.69 GOPs on structured sparse Bi-LSTM networks without batching. Compared with the implementations on CPU and GPU, the low-cost FPGA accelerator achieves 43.5x and 6.3x speedup on latency, 520.9x and 46.5 x improvement on energy efficiency, respectively.