Low-Cost FPGA-Enhanced CNN Accelerator for Real-Time YOLO Object Detection and Classification

Abstract

This work presents a low-cost FPGA-based accelerator for real-time object detection and classification using a compressed YOLOv3-Tiny model. Existing FPGA-based CNN accelerators excel in one critical performance metric but sacrifice either throughput, accuracy, or power efficiency. This is particularly the case for low-cost devices that are resource-constrained and often heavily rely on off-chip memory which hinders performance. To address these limitations, we introduce three novel contributions: 1) an iterative structured hardware pruning algorithm that removes the least important filters from the YOLO model in small increments; 2) a quantization-aware training (QAT) algorithm that adapts the scaling factor per layer; and 3) a custom RTL memory-mapping controller that prioritizes on-chip BRAM/URAM memory allocation to improve throughput while decreasing power consumption. With this approach, the model size was reduced by $13.3\times $ while preserving accuracy. Implemented on a low-cost Kria KV260 FPGA, the approach achieved 93.8% detection accuracy, 24.3 FPS throughput, 41.59 ms latency, and just 2.13W of power consumption. The result was a high-performing, efficient system that directly outperformed comparable low-cost designs. These results demonstrate that balanced, high-performance YOLO inference is attainable on low-cost FPGA hardware without reliance on off-chip memory.