Pipeline ShiftAddNet: An FPGA-Based CNN Implementation With Low Hardware Consumption Targeting Constrained Devices

ShiftAddNet is a recently proposed multiplier-less CNN that replaces conventional multiplication with cheaper shift and add operations, which makes it suitable for hardware implementation. In this paper, we present the first implementation of ShiftAddNet FPGA inference core, which achieves low area consumption and fast computation. ShiftAddNet combined the convolutional layer of the DeepShift-PS (denoted as Shift-Accumulate, sac) and AdderNet (denoted as Add-Accumulate, aac) into a single computational stage. Due to this reason, there are data dependencies between the sac and aac, which prohibits them from being executed in parallel, resulting in $2\times$ more operations compared to other multiplier-less CNNs like DeepShift-PS and AdderNet. To overcome this performance bottleneck, we proposed a novel technique to allow pipeline processing between sac and aac, effectively reducing the latency. The proposed ShiftAddNet-18 was evaluated on a small ResNet-18, achieving 11.37 ms of latency per image, which is $\sim$69.21% faster than the original version that takes 19.24 ms. On a denser network, the proposed pipeline ShiftAddNet-101 requires only 61.92 ms as compared to the original version of 98.85 ms, showing a latency reduction of $\sim$37.1%. Compared to the state-of-the-art multiplier-less CNN core (e.g., AdderNet), our work is 20% slower in latency but provides higher accuracy and consumes $2.2\times$ less DSP.