VLSI Architecture of Generalized Pooling for Hardware Acceleration of Convolutional Neural Networks

Convolutional Neural Networks (CNNs) handle a massive variety of datasets with great accuracy for various image processing and computer vision applications. However, it comes at the cost of the requirement of large hardware resources, which are computational and energy extensive. There is a need for efficient hardware and algorithmic optimization for real-time CNN inference while deploying the CNN model. This paper proposes a novel VLSI architecture of generalized pooling operation for hardware acceleration of CNN inference in real-time. Generalized pooling operation adaptively downsamples the huge parameter set generated by the convolutional layer by generating weights as per the input features. It is capable of accommodating varying feature maps and preserves significant features, unlike other counterparts maximum and average pooling. In order to efficiently compute the output of the generalised pooling operation, the proposed hardware design makes use of the Newton-Raphson reciprocal approximation for division operations, a low number of comparators, and a high degree of parallelism. The proposed architecture is developed and tested for performance evaluation on Xilinx Vivado 2018.3, and the target device chosen is Zynq UltraScale + MPSoC ZCU104 Evaluation board.