AI-assisted ISP hyperparameter auto tuning

Abstract

Images and videos are vital visual information carriers, and the image signal processor (ISP) is an essential hardware component for capturing and processing these visual signals. ISPs convert raw data into high-quality color images, which requires various function modules to control different aspects of image quality. However, the results of these modules are interdependent and have crosstalk with each other, making it tedious and time-consuming for manual tuning to obtain a set of ideal parameter configurations to achieve stable performance. In this paper, we introduce xkISP, a self-developed open-source ISP project which includes both a C model and hardware implementation of an 8-stage ISP pipeline. Most importantly, we present a novel proxy function-based AI-assisted ISP tuning solution that is demonstrated to accelerate the ISP parameter configuration process and improve performance for both human vision and computer vision tasks.