LInear Optimization of Neurons (LION) for computational photography augmentation

Abstract

Imaging in complex lighting conditions such as low light and underwater scenarios remains challenging. Computational photography algorithms can enhance image quality in complex environments. However, conventional methods may suffer from imprecise models or priors, while deep learning ones face limitations in data-unaccessible scenarios. To address these challenges, we present a generalized augmenting technique for computational photography, termed Linear Optimization of Neurons (LION), designed to perform case-specific optimization while maintaining the network’s structure and weights. We deeply investigated the robust semantic constraints on the deep feature domain and found that linear transformations of pre-trained convolutional neural network (CNN) features can enhance the quality of output images without necessitating modifying network architecture. Building upon this finding, we developed a generalized workflow to augment both conventional and deep learning methods. We first fit the encoder–decoder CNN to output the pre-reconstruction from an off-the-shelf method. Then, fixing the network weights, we optimize the linear combinations of features using a generalized color and texture regularization to further enhance the output image quality. A series of experiments on various public datasets confirmed the effectiveness of this technique, particularly in underwater and low-light imaging scenarios.