Neural Network Quantization is All You Need for Energy Efficient ISP

Abstract

It is common to devise and develop a Hand Crafted Feature for the modules constituting the CIS ISP Chain. This not only requires specialized domain knowledge, but also has a limitation in the degree of image quality improvement, and there have been many attempts to introduce deep learning to overcome this. However, it is very challenging to circuit the deep learning model directly on the CMOS sensor. Therefore, this study improves energy efficiency through Ultra Low Bit Quantization for demosaicing functions existing in the ISP chain, and proposes an ultra-high-speed/ultra-light model that enables Integer Only Inference. The proposed methodology can be utilized as an element technology that can be applied to deep learning models for other functions constituting ISP.