FPGA based Implementation of Binarized Neural Network for Sign Language Application

In the last few years, there is an increasing demand for developing efficient solutions for computer vision-related tasks on FPGA hardware due to its quick prototyping and computing capabilities. Therefore, this work aims to implement a low precision Binarized Neural Network (BNN) using a Python framework on the Xilinx PYNQZ2 embedded platform to tackle the challenging problem of Sign Language recognition. More specifically, the FINN framework is adopted and the BNN topology is modified to adapt large resolution (i.e 64x64) to perform classification of proposed Indian Sign Language (ISL) gestures into corresponding numbers. In addition, data augmentation techniques are also applied to improve the overall performance of the neural network. Furthermore, hardware/software co-verification of BNN topology is performed to validate the accuracy after implementing it onto hardware. Extensive experimental results show that it achieves a classification rate of 843.8 frames per second (FPS) on PYNQ-Z2 FPGA which delivers higher performance as compared to previous works. Also, the post-implementation results are analyzed in terms of resource utilization and power consumption.