A Strategy to Maximize the Utilization of AI Neural Processors on an Automotive Computing Platform

Abstract

Advancements in AI are transforming the automotive industry, creating opportunities for AI-powered software and hardware. AI-driven features in automobiles are increasingly embraced due to their potential to significantly improve the driving experience. High-performance computing, particularly with NPUs, becomes crucial for executing the AI features. To maximize the efficiency and utilization of NPUs, DAIMO-NPU optimizes the inference sequence of the DNN models that form the backbones of the AI features. Not only does it organize and schedule the model inference tasks but also supports the tasks to be executed on heterogeneous NPU settings. Three main components are involved in the implementation of DAIMO-NPU. The schedule-table generator is responsible for creating a detailed plan for the model inference tasks, which is to be updated whenever an AI feature is added, removed, or upgraded. The onboard operator reads the schedule table and carries out the tasks accordingly. And, by dividing models into smaller segments, while not mandatory, the schedule table can be further optimized. In the subsequent developments, the integration of additional NPU hardware properties into DAIMO-NPU will be pursued.