Mask-Q attention network for flare removal

Abstract

Lens flare is a common optical phenomenon that is typically undesirable because it significantly degrades image quality, thus affecting certain visual tasks. The existing main methods using CNN-based models and transformer-based models aimed at removing lens flare, however, perform poorly in removing large-scale flares because they lack the inductive bias for spatial equivariance and the ability to capture both global and local features simultaneously. This deficiency prevents networks from quickly pinpointing the locations of flares and from fully learning the contextual content necessary for repairing contaminated areas, and they also struggle with restoring uncontaminated regions. In this paper, we introduce the Mask-Q Attention Network, a multi-scale framework designed to address the flare removal problem. Our approach focuses on extracting both global and local features, leveraging ResBlock for local feature extraction and Mask-Q Attention for capturing global features. In the Mask-Q Attention, we enhance the localization capability of the attention mechanism by integrating the flare mask with the query vector (Q) in the self-attention process. The flare mask is obtained by binarizing the initial image. This integration with the flare mask effectively resolves the issue of the lack of spatial equivariance in Transformer blocks, providing the network with prior knowledge of the flare locations. Extensive experiments demonstrate MQANet’s superiority, achieving minimum gains of 0.5% PSNR and 0.6% SSIM on synthetic datasets, alongside a 0.1% SSIM improvement on real-world data. It also performs well in terms of the LPIPS metric.