A Configurable FPGA Implementation of the Tanh Function Using DCT Interpolation

Abstract

Efficient implementation of non-linear activationfunctions is essential to the implementation of deep learningmodels on FPGAs. We introduce such an implementation basedon the Discrete Cosine Transform Interpolation Filter (DCTIF). The proposed interpolation architecture combines simple arithmeticoperations on the stored samples of the hyperbolic tangentfunction and on input data. It achieves almost 3 better precisionthan previous works while using a similar amount computationalresources and a small amount of memory. Various combinationsof DCTIF parameters can be chosen to trade off the accuracy andthe overall circuit complexity of the tanh function. In one case, the proposed architecture approximates the hyperbolic tangentactivation function with 0.004 maximum error while requiringonly 1.45 kbits BRAM memory and 21 LUTs of a Virtex-7 FPGA.