Self-adaptive and learnable detail enhancement network for efficient image super resolution

Abstract

In recent years, single image super-resolution (SISR) methods based on deep learning have advanced significantly. However, their high computational complexity and memory demands hinder deployment on resource-constrained devices. Although numerous lightweight super-resolution (SR) methods have been proposed to address this issue, most fail to distinguish between flat and detailed regions in images, treating them uniformly. This lack of targeted design for detailed regions, which are critical to SR performance, results in redundancy and inefficiency in existing lightweight methods. To address these challenges, we propose a simple yet effective network Self-adaptive and Learnable Detail Enhancement Network (LDEN) that specifically focuses on the reconstruction of detailed regions. Firstly, we present two designs for the reconstruction of detailed regions: (1) we design the Learnable Detail Extraction Block (LDEB), which can pay special attention to detailed regions and employ a larger convolution kernel in LDEB to obtain a larger receptive field; (2) we design a lightweight attention mechanism called Detail-oriented Spatial Attention (DSA) to enhance the network's ability to reconstruct detailed regions. Secondly, we design a hierarchical refinement mechanism named Efficient Hierarchical Refinement Block (EHRB) which can reduce the inadequate information extraction and integration caused by rough single-layer refinement. Extensive experiments demonstrate that LDEN achieves state-of-the-art performance on all benchmark datasets. Notably, for 4 × magnification tasks, LDEN outperforms BSRN - the champion of the model complexity track of NTIRE 2022 Efficient SR Challenge - by achieving gains of 0.11 dB and 0.12 dB while reducing parameters by nearly 10 %.