Accelerating the Activation Function Selection for Hybrid Deep Neural Networks – FPGA Implementation

Much of the work in literature about hardware implementations of deep neural networks illustrates the multiplication of input signal with weights and summing up the data. Work in this paper is focused on the hardware implementation of non-linear functions (activation functions), along with hardware rendition of the automatic selection of activation function for each layer in the neural network in order to increase the accuracy. We have used Field Programmable Gate Array (FPGA) based hardware development platform to add in the advantages of the power efficiency and edge deployment. Our novel hardware design modules to this extent are capable of accelerating the entire process of activation function selection and activation function output generation along with its derivative. The tabulated results in this paper, for power and resource utilization generated through Xilinx® Vivado platform by targeting our design modules towards Avnet® Ultra96 v2 Evaluation board prove the sheer hardware novelty in terms of less hardware foot print and energy efficiency in comparison to the GPU (Graphics Processing Unit) and CPU (Central Processing Unit) based executions along with FPGA based implementations of few activation functions reported in literature.