Real-time vignetting compensation and exposure correction for panoramic images by optimizing irradiance consistency

Abstract

Abstract Image-based object detection is a crucial task in autonomous driving. In many cases, objects are not correctly detected and classified if they are only partially visible due to a limited field of view. Also, even if stitched panoramic images are used, errors in object detection can still occur if the seam between individual images is visible. This happens due to vignetting or different exposure, although the images are optimally aligned. In this article, we present a real-time capable and effective method for vignetting compensation and exposure correction. Before runtime, the camera response function is determined and the vignetting model is preliminarily approximated. We obtain the irradiance from the intensity values of incoming images. Then, the vignetting model is applied. Afterwards, the pixels at the seam are used to correct the exposure. Finally, we convert the corrected irradiance back to intensity values. We evaluate our approach by measuring the image stitching accuracy in the overlapping area by the IoU of grayscale histograms and the mean absolute error of intensity values. The metrics are applied both on data recorded with our experimental vehicle and on the publicly available nuScenes dataset. Finally, we demonstrate that our approach runs in real-time on GPU.