Efficient CORDIC-Based Activation Functions for RNN Acceleration on FPGAs

Abstract

Recurrent neural networks (RNNs), particularly long short-term memory (LSTM) networks, have emerged as standard tools for tackling a wide range of time series applications, such as natural language processing. However, deploying these models on edge devices presents great challenges due to limited computational resources. Additionally, the implementation of RNN activation functions on low-end hardware devices significantly impacts the overall network performance, as activations constitute the dominant part of execution time. In this work, we propose an efficient approach for implementing commonly used RNN activations, leveraging an optimized coordinate rotation digital computer algorithm (CORDIC). Moreover, we propose a unified hardware architecture for mapping the CORDIC-based method onto field-programmable gate arrays (FPGAs), which can be configured to implement multiple nonlinear activation functions. Our architecture reduces the computational time with fewer iterations in CORDIC compared with existing methods, rendering it particularly suitable for resource-constrained edge devices. Our design is implemented on a Xilinx Zynq-7000 device and evaluated across three RNNs and benchmark datasets. Experimental results demonstrate that our design achieves up to a 2$\boldsymbol{\times}$ speedup while maintaining model accuracy compared with the state-of-the-art designs.