Accelerating real-time computer vision applications using HW/SW co-design

Video processing applications have become increasingly difficult to implement on hardware, owing to the complex computer vision algorithms involved. This paper presents a real-time video processing architecture based on hardware/software co-design that improves execution speed and reduces the time to market of applications. We have implemented this framework for handwritten digit recognition on the Zybo Zynq-7000 ARM/FPGA SoC using Vivado High Level Synthesis (HLS) and Xillybus tools. Histogram of Oriented Gradients (HOG) feature extraction algorithm has been optimised for hardware execution and acceleration techniques have been applied on Vivado HLS to achieve a speed up of 38.89 for the HOG algorithm and recognition accuracy of 95.6%. Low precision arithmetic along with our approximations for costly functions, produced this significant gain in throughput by reducing 90% of the hardware resources required with just a marginal accuracy reduction by 1%. An overall performance improvement of 77% is obtained through hardware/software co-design over software execution. The framework identified digits seamlessly in a real-time video stream at 30 frames per second and enabled high frame rate video processing.