Burst denoising transformer with multi-task optical flow estimation

Abstract

Burst denoising focuses on producing a clean image from a series of noisy frames captured in rapid succession. A major challenge during burst capturing is the misalignment between frames, caused by subtle movements of the camera or the scene. To deal with this difficulty, in this paper we introduce a novel Burst Denoising Transformer (BDFormer) network. First, we introduce a Transformer-based Multi-task Optical Flow Estimation module (TMOFE) to align the frames, where an auxiliary denoising task is used to reduce the impact of noise during optical flow estimation. Next, the aligned frames are passed through a Transformer-based Feature Enrichment module (TFE). The core unit of TFE lies in a specially-designed Spatial and Channel-wise Transformer Block (SCTB), which combines an FFT-based Spatial Transformer Block (FSTB) and a Channel-wise Transformer Block (CTB), in order to fully leverage both spatial and channel-wise global information across inter- and intra-frames. Extensive experiments show that BDFormer outperforms other transformer-based methods, achieving superior performance while maintaining low computational complexity.