All-in-one weather removal via Multi-Depth Gated Transformer with gradient modulation

Abstract

All-in-one weather removal methods have made impressive progress recently, but their ability to recover finer details from degraded images still needs to be improved, since (1) the difficulty of Convolutional Neural Networks (CNNs) in providing long-distance information interaction or Visual Transformer with simple convolutions in extracting richer local details, makes them unable to effectively utilize similar original texture features in different regions of a degraded image, and (2) under complex weather degradation distributions, their pixel reconstruction loss functions often result in losing high-frequency details in restored images, even when perceptual loss is used. In this paper, we propose a Multi-Depth Gated Transformer Network (MDGTNet) for all-in-one weather removal, with (1) a multi-depth gated module to capture richer background texture details from various weather noises in an input-adaptive manner, (2) self-attentions to reconstruct similar background textures via long-range feature interaction, and (3) a novel Adaptive Smooth ${\text{L}}_1$ (${\text{ASL}}_1$) loss based on gradient modulation to prompt finer detail restoration. Experimental results show that our method achieves superior performance on both synthetic and real-world benchmarks. Source code is available at https://github.com/xiangLi-bit/MDGTNet.