A Real-time Image Processing Hardware Acceleration Method based on FPGA

Abstract

Real-time image sensed by the visual sensor usually contains a lot of noise information. Model reasoning, and pattern recognition-oriented CNNs face such thorny issues as excessive computation, poor accuracy and high resource occupancy. Hence, CNN architecture was heterogeneously deployed on the Zynq platform to realize hardware acceleration for the image processing algorithm. MNIST dataset was adopted to train CNN for extracting network parameters on PC terminal under the Caffe framework; the convolutional layer responsible for heavy computational load was deployed onto FPGA for parallel computing to increase system speed; input layer and output layer responsible for a small amount computation were placed on ARM terminal to reduce resource consumption; real-time image acquired by the camera was binarized to highlight image features and improve the recognition accuracy; the hardware acceleration performance of the heterogeneously deployed CNN was verified with numerous experiments on image recognition of handwritten numerals. Experimental results indicated that: CNN hardware accelerator kept an image recognition accuracy up to 99.02% which is largely equivalent to that of client PC; When recognizing a single piece of handwritten numerical sample, under the use of optimized instructions and 100MHz clock frequency, the recognition time of a single image is 0.53s, which is 16 times faster than pure ARM operation; the maximum power consumption of the system is 2.606W, which is far Lower than general-purpose processors.