Superscalar Time-Triggered Versatile-Tensor Accelerator

Abstract

Integrating AI hardware accelerators into safety-critical real-time systems to speed up the inference execution of safety-critical AI applications demands rigorous assurance to prevent potentially catastrophic outcomes, especially in environments where timely and accurate results are crucial. Even in cases where AI models are potentially designed and constructed correctly using AI frameworks, the system’s safety will also rely on the real-time behavior of the AI hardware accelerator. While AI hardware accelerators can achieve the necessary throughput, conventional accelerators, such as the versatile tensor accelerator (VTA) encounter significant challenges in predictability and reliability. These challenges stem from the variability in event-driven inference execution and insufficient timing control, posing considerable risks in safety-critical scenarios where delays in providing inference results can have severe consequences. To address this challenge, previous work introduced the time-triggered VTA (TT-VTA) to ensure timely execution of tensor operations. Nonetheless, the TT-VTA exhibited a slightly longer average inference time of 53 ms compared to the conventional VTA’s 51 ms, underscoring the ongoing need for optimization in this crucial domain to speed up the inference execution, while sustaining the deterministic and predictable behavior of the TT-VTA. This article proposes a novel superscalar TT-VTA (STT-VTA) architecture specifically designed to address the deficiencies of conventional VTAs and TT-VTAs. The STT-VTA architecture employs pattern-based timing schedules generated by an extended software simulator and an architecture configuration manager to analyze tensor operations within a given AI model and determine the required number of additional VTA modules for faster inference than a single (TT-)VTA setup. It integrates DRAMSim2 for memory instructions and a cycle-accurate simulator for nonmemory instructions. Evaluation using various models demonstrates that the STT-VTA achieves identical classification accuracy as the conventional VTA and TT-VTA, while improving performance and reducing inference time by 20%–41%. Moreover, it ensures deterministic temporal use of shared resources, such as memories and memory-buses and precise timing control to avoid interference. These results contribute toward safety and reliability of AI systems deployed in a safety-critical environment.