Implementation of convolutional-LSTM network based on CPU, GPU and pynq-zl board

Abstract

Deep learning is among the most commonly investigated approach in computer vision area. Quite recently, considerable attention has been paid to develop an end-to-end deep learning approach for action recognition. According to the developments of these time and resource consuming deep learning models, there is now a growing interest in implementing an accelerator low-power hardware architecture. The main objective of this paper is to implement an optimized convolutional-Long Short Term Memory (LSTM) architecture based a low-cost pynq-zl design tool for human action recognition applications. Firstly, the pre-trained Convolutional Neural Network (CNN) model is applied to extract relevant features from videos. Secondly, the classification of these sequences is done using the LSTM with optimized parameters. Finally, the model testing step is performed on the ARM of the pynq-zl FPGA platform and compared with the performances obtained by the central processing unit and graphics processing tools. The experimental results, performed in UTD-MHAD dataset, prove the efficiency of our proposed approach.