CGR-NPU: A Hybrid CGRA and NPU Architecture for Adaptive Neural Computing Workloads

Abstract

To exploit apabilities of processors in contemporary heterogeneous systems, frequent data movement across disparate memory domains is required. The substantial overhead incurred by this data transfer is now a principal bottleneck for system performance, especially for computationally intensive, large-scale AI workloads. To address this, we propose CGR-NPU, a hybrid architecture that synergistically combines a Coarse-Grained Reconfigurable Array (CGRA) with a Neural Processing Unit (NPU). The architecture features tightly-coupled CGRA and NPU operational modes with fast switching, and leverages a reinforcement learning-based mechanism for dynamic resource allocation tailored to workload characteristics. Evaluations on diverse ML benchmarks show our integrated architecture improves average 1.62× performance speed up and reduces silicon area by 48.4% compared to standalone CGRA or NPU baselines. The overhead for mode switching and internal data handling is minimal, constituting only 7% of the total execution time.