AIA: A Customized Multi-Core RISC-V SoC for Discrete Sampling Workloads in 16 nm

Probabilistic models (PMs) are essential in advancing machine learning capabilities, particularly in safety-critical applications involving reasoning and decision-making. Among the methods employed for inference in these models, sampling-based Markov chain Monte Carlo (MCMC) techniques are widely used. However, MCMC methods come with significant computational costs and are inherently challenging to parallelize, resulting in inefficient execution on conventional CPU/GPU platforms. To overcome these challenges, this article presents an approximate inference accelerator (AIA), a multi-core RISC-V system-on-chip (SoC) design fabricated using Intel’s 16 nm process technology. Our AIA is specifically designed to empower edge devices with robust decision-making and reasoning abilities. The AIA architecture incorporates an RISC-V host processor to manage chip-to-chip data communication and a 2-D mesh of 16 custom versatile RISC-V cores optimized for high-efficiency approximate inference. Each core features: 1) custom instructions and datapath blocks for non-normalized Knuth-Yao (KY) sampling, as well as for the interpolation of non-linear functions (e.g., logarithmic and exponential), and 2) direct data-access to the register file (RF) of each neighboring core, to reduce the data movement costs of frequent data exchanges between nearby cores. To further capitalize on the parallelism potential in MCMC algorithms, we developed a specialized compile chain that enables efficient spatial mapping and scheduling across the cores. As a result, AIA attains a peak sampling rate of 1277 MSamples/s at 0.9 V and achieves an energy efficiency of 20 GSamples/s/W at 0.7 V, surpassing the previous state-of-the-art (SotA) ASIC accelerator for probabilistic inference by up to $6\times $ in speed and $5\times $ in energy efficiency. Furthermore, the AIA’s versatility is demonstrated through the successful mapping of different types of PM workloads onto the chip.