Enhancing Perception for Autonomous Vehicles: A Multi-Scale Feature Modulation Network for Image Restoration

Abstract

Accurate environmental perception is essential for the effective operation of autonomous vehicles. However, visual images captured in dynamic environments or adverse weather conditions often suffer from various degradations. Image restoration focuses on reconstructing clear and sharp images by eliminating undesired degradations from corrupted inputs. These degradations typically vary in size and severity, making it crucial to employ robust multi-scale representation learning techniques. In this paper, we propose Multi-Scale Feature Modulation (MSFM), a novel deep convolutional architecture for image restoration. MSFM modulates multi-scale features in both frequency and spatial domains to make features sharper and closer to that of clean images. Specifically, our multi-scale frequency attention module transforms features into multiple scales and then modulates each scale in the implicit frequency domain using pooling and attention. Moreover, we develop a multi-scale spatial modulation module to refine pixels with the guidance of local features. The proposed frequency and spatial modules enable MSFM to better handle degradations of different sizes. Experimental results demonstrate that MSFM achieves state-of-the-art performance on 12 datasets for a range of image restoration tasks, i.e., image dehazing, image defocus/motion deblurring, and image desnowing. Furthermore, the restored images significantly improve the environmental perception of autonomous vehicles.