Design and Evaluation of Energy-Efficient Whisper Dot-Product Kernel Offloading on a CGLA Architecture

Abstract

In this paper, we implement and evaluate Whisper dot-product kernel offloading on IMAX, a programmable coarse-grained linear arrays (CGLAs) architecture. Whisper-tiny.en profiling on an ARM Cortex-A72 shows that dot-product operations account for 90.6% of FP16 execution time and 87.1% of Q8_0 execution time. To address this kernel bottleneck, we combine kernel mapping, local-memory sizing, and burst scheduling. The implementation uses inline FP16-to-FP32 conversion, 2-way SIMD FMA on a 64-bit datapath, column-wise multithreading, and mixed execution in which aligned vector segments run on IMAX and residual segments run concurrently on the host CPU. We evaluate the design with an FPGA prototype and a 28 nm ASIC projection at 840 MHz. For Whisper-tiny.en, 32 KB local memory and burst length 16 jointly minimize PDP and EDP. Under a TDP-based cross-platform comparison, the projected IMAX records a PDP of 11.58 J for Whisper-tiny.en Q8_0, 2.35$\times$ lower than Jetson AGX Orin (27.16 J) and 10.48$\times$ lower than RTX 4090 (121.38 J). The same design extends to Whisper-base.en and Whisper-small.en, where the PDP gap narrows as 32 KB local-memory coverage drops from 93.8% for tiny to about 66.5% for base and small. These results position IMAX as a programmable architecture for lower-PDP local ASR in the tiny-model regime.