SNAP: Selective NTV Heterogeneous Architectures for Power-Efficient Edge Computing

Abstract

While there is a growing need to process ML inference on the edge for improved latency and extra security, general-purpose solutions alone cannot cope with the increasing performance demand under power restrictions. Considering that systolic arrays are a prominent, but also power-hungry solution, we propose a methodology to enable their use in edge devices. For that, we propose SNAP, a selective Near-Threshold Voltage (NTV) strategy to explore heterogeneous MPSoCs with two voltage islands, one at NTV, and another at nominal voltage. By adopting a dynamic programming approach, SNAP may selectively apply NTV to the systolic array and to an optimal subset of cores in RISC- V-based MPSoCs, enabling ML acceleration on the edge. Combined with a smart application mapping, the strategy increases performance by up to 18.9 % over a nominal design within the same power limits.