Introduction to the Special Section on Energy-Efficient AI Chips

Energy efficiency is one of the most important metrics in AI system designs on both servers and mobile devices. Especially, mobile and edge devices require 10-100X better energy-efficient computing for immersive AR/VR applications as well as AI-based apps on smartphones, smart cameras, etc. Due to battery and cost reasons, such emerging applications demand extreme energy efficiency and high performance to run dozens of heavy neural network models in real time and under stringent power budgets. In order to realize 100X improvements in energy efficiency, we require innovative ideas in both software and hardware. In this special issue, which originated from the Highly Efficient Neural Processing (HENP) workshop held in ESWEEK 2020, we aimed at covering state-of-the-art industrial and academic efforts to achieve orders of magnitude better energy efficiency in software and hardware designs for AI chips. Recent neural networks adopt special layers for better compute efficiency. “MVP: An Efficient CNN Accelerator with Matrix, Vector, and Processing-Near-Memory Units” by Lee et al. proposes an approach to improve area-efficiency of systolic array accelerator for depth-wise separable convolution and squeeze-and-excitation layers which are widely adopted on networks for mobile and embedded systems due to efficiency reasons. Reusing computation results is one of the representative methods in reducing computation cost thereby improving energy efficiency. “Energy Efficient Boosting of GEMM Accelerators for DNN via Reuse” by Cicek et al. proposes a novel reuse-centric hardware accelerator for CNN inference based on the proposed improved detection of neuron vector similarity. Embedded devices are often characterized by continuous sensory inputs and limited computing/programming capabilities. “A Low-Power Programmable Machine Learning Hardware Accelerator Design for Intelligent Edge Devices” by Kee et al. proposes a hardware accelerator, called intelligent boosting engine, to accelerate sensor fusion and the SVM-based motion recognition algorithm with limited programmability. Processing sequential inputs under timing and power budgets is a representative design problem in edge applications. In “Energy Efficient LSTM Inference Accelerator for Real-Time Causal Prediction” by Chen et al. the authors take advantage of fine-grained parallelism, pipelined feedforward and recurrent updates in LSTM and present a bit-sparse quantization to reduce the circuit cost by replacing the original multiplication with the bit-shift operation. Adopting reinforcement learning on edge devices for sequential decision-making and control based on image inputs is desirable, but challenging due to the low efficiency of training and the high cost of inference. In “E2HRL: An Energy-Efficient Hardware Accelerator for Hierarchical Deep Reinforcement Learning”, Shiri et al. proposes a scalable hardware architecture called E2HRL which boosts training speed by learning hierarchical policies and enables high energy efficiency by a cross-layer design methodology.