Efficient and Fast High-Performance Library Generation for Deep Learning Accelerators

Abstract

The widespread adoption of deep learning accelerators (DLAs) underscores their pivotal role in improving the performance and energy efficiency of neural networks. To fully leverage the capabilities of these accelerators, exploration-based library generation approaches have been widely used to substantially reduce software development overhead. However, these approaches have been challenged by issues related to sub-optimal optimization results and excessive optimization overheads. In this paper, we propose Heron to generate high-performance libraries of DLAs in an efficient and fast way. The key is automatically enforcing massive constraints through the entire program generation process and guiding the exploration with an accurate pre-trained cost model. Heron represents the search space as a constrained satisfaction problem (CSP) and explores the space via evolving the CSPs. Thus, the sophisticated constraints of the search space are strictly preserved during the entire exploration process. The exploration algorithm has the flexibility to engage in space exploration using either online-trained models or pre-trained models. Experimental results demonstrate that Heron averagely achieves 2.71 $\times$ speedup over three state-of-the-art automatic generation approaches. Also, compared to vendor-provided hand-tuned libraries, Heron achieves a 2.00 $\times$ speedup on average. When employing a pre-trained model, Heron achieves 11.6 $\times$ compilation time speedup, incurring a minor impact on execution time.