Compare with the Traditional Heterogeneous Solution: Accelerate Neural Network Algorithm through Heterogeneous Integrated CPU+NPU Chip on Server

Abstract

The increasing popularity of artificial intelligence (AI) requires the ability to process intensive data and efficient heterogeneous computing power. As a result, a heterogeneous integration scheme involving both central processing units (CPUs) and neural processing units (NPUs) has become increasingly prevalent in various edge terminals, such as mobile phones. Compared with traditional separated heterogeneous solutions, the integration scheme can effectively reduce the distance and number of data transmissions, thereby accelerating deep neural network (DNN) models and improving energy efficiency. Due to the low power requirements of cloud computing, heterogeneous integration solutions are beginning to be used in the design of processor architectures for servers. The TF16110 integrates NPUs into server CPUs, creating an efficient parallel computing solution for servers that lack GPUs or other AI acceleration devices. In this paper, we evaluate and analyze commonly used DNN models. Compared with NVIDIA’s TX2 GPU, the heterogeneous integrated CPU+NPU design can provide similar computational power and achieve 5x higher energy efficiency and 10x cost-effectiveness under the premise of ensuring accuracy