Optimizing Convolutions for an Inference Accelerator: Case Study: Intel’s NNP-I 1000 DL Compute Grid

Abstract

With Deep Learning (DL) surpassing humans in Image Recognition and Machine Translation related tasks, the demand for specialized hardware has increased in the recent past. DL Accelerators belong to a category of such purpose-built hardware that promise compelling performance for Neural Net computations. But a specialized hardware needs a powerful compiler to unlock its full potential. This paper discusses the Code Generator and Optimizer (CGO) that produces optimized tiling as well as schedule of Convolution operations for the DL Compute Grid in Intel’s NNP-I 1000 platform. This paper also presents some of the key optimization techniques used and the associated performance gains across a rich variety of Deep Learning workloads.