Centralized Generic Interfaces in Hardware/Software Co-design for AI Accelerators

A hardware/software co-design for AI accelerators such as Neural Processing Unit (NPU) is essential not only to support the required functionality but also to meet primary goals of improved performance and power efficiency. However, their ever-changing requirements often introduce undesirable development costs. Indeed, it is quite challenging for developers from different backgrounds to efficiently work together to construct a full HW/SW stack to develop AI accelerators. This paper addresses these challenges, and proposes a centralized collaboration methodology for efficient full-stack development, especially targeting NPU HW. The proposal is inspired based on the observations from our experiences, presented later as a case study. As not all of the involved developers have enough knowledge of software engineering, this approach suggests making a central development group (e.g., runtime system software) have a higher priority to organize and devise common interfaces including APIs for each layer in the full-stack. This aims to minimize unnecessary discussions between development groups and hide any minor updates introduced with each new design, reducing the overall development costs and improving the quality of products. More importantly, each development group can focus on their work as much as possible with this approach.