Analysis of the acceleration of deep learning inference models on a heterogeneous architecture based on OpenVINO

Abstract

Convolutional neural networks (CNN) are a very powerful tool for many different applications. This capability is highly demanded in the field of embedded systems for video surveillance, speech recognition, and image analysis. Due to its high computational intensity, the application of CNN is limited to real-time research areas where computational speed is extremely important. Therefore, an appropriate accelerator is required to fulfill the requirements of these limitations. On the one hand, GPUs are widely used to accelerate the CNN under high power dissipation. On the other hand, the trend for FPGA implementation is increasing rapidly due to its low power consumption and facile re-configurability. In this work, we evaluate the inference performance of 10 classification models and 9 object detection models using the OpenVINO toolkit. In addition, we analyzed the implementation of these models on the DE5a-Net DDR4 equipped with an Arria 10 GX FPGA. The results show that the performance of Full-Precision FP32 classification models on a heterogeneous architecture FPGA/CPU is on average 3.6X faster than the CPU.