CCTFaceNet: Enhancing face super-resolution with cascaded CNN-transformer and dual-path feature fusion

The convolutional neural networks (CNNs) have significantly advanced face super-resolution techniques, enabling the restoration of degraded facial details. However, these methods often encounter limitations related to computational cost. Additionally, the limited receptive fields of CNNs can hinder the realistic and natural reconstruction of facial images. Transformer-based models counter this issue by global feature learning using multi-head self attention, but lack the incremental feature learning capabilities of CNNs. This paper proposes a Cascaded CNN-Transformer-based Face image super-resolution Network (CCTFaceNet) to deal with the above-mentioned issues. In the proposed network, to enrich input low-resolution face images, a Preliminary Super-Resolution, i.e., PreSR network is placed at the beginning of CCTFaceNet. The output of PreSR is then fed to a deep feature extraction block consisting of a dual-path feature fusion block (DPFF). This block internally has two paths, one for CNN and the other for the cascaded attention transformer (CAT). DPFF also has a context-resolving unit responsible for filtering out redundant information. CAT consists of a shifted window multi-head self-attention, a multi scale edge attention, and a channel importance recalibration module; they are assembled in a cascaded manner. This assembly can reconstruct highly accurate details spatially and along the channel with crisp edges. A feed-forward layer is also sandwiched between the above attention cascade. The extracted deep features are upsampled using pixel-shuffle and sub-pixel convolutional layers. Extensive experiments conducted on several benchmark datasets affirm the supremacy of the proposed network.