Staggered HDR video reconstruction with a real-world benchmark dataset for night scenes

Abstract

Capturing night scenes with full visibility is attractive. Due to the limited dynamic range of camera sensors, we cannot record details for both light-source regions and dark regions. A practical solution is utilizing multi-exposure fusion to get high dynamic range (HDR) results. However, the lack of real-world NightHDR dataset hinders the development of deep learning based NightHDR video reconstruction. To solve this problem, we first construct a real-world NightHDR video dataset, which contains 57 LDR-HDR video pairs captured under night scenes in both raw and sRGB formats, where the LDR frame includes short and long exposures. Different from previous alternating exposure based or frame based HDR video reconstruction, we turn to staggered HDR reconstruction, which is more applicable in real scenarios. Correspondingly, we propose an efficient NightHDRNet, which contains single-exposure enhancement (stage I), two-exposure fusion (stage II), and two stage selective fusion modules. In this way, our network can improve the dynamic range and reduce ghosting artifacts. Extensive experiments show that our approach outperforms state-of-the-art methods qualitatively and quantitatively. We will release our dataset and code after the acceptance of this work.