FCLNN: A Flexible Framework for Fast CNN Prototyping on FPGA with OpenCL and Caffe

The CNN algorithms are still in rapid evolution, while the traditional RTL level programming on FPGA is relatively slow and requires great efforts and expertise. In this paper, we propose a flexible HW/SW co-design framework for both fast and high-throughput CNN prototyping with commercial high-level OpenCL language and the standard open-source deep learning framework Caffe. We build up a parameterizable stream-architected convolution engine and extend it to support any input size and filter depth. For iterative development process, we provide both layer-based and subgraph-based execution schedule. While for competitive performance, both on-chip and off-chip communication are optimized. Using our framework with Intel Arria 10 GX1150 FPGA, we achieve 69.2 fps and 18.6 fps on official YOLOv2-tiny-voc and YOLOv2-voc respectively. To the best of our knowledge, this is the first work to accelerate the state-of-the-art YOLOv2 with both real-time performance and < 1% accuracy drop on FPGA.