E2D-GS: Event-enhanced deblurring gaussian splatting

Abstract

In recent years, implicit neural representations and explicit 3D Gaussian Splatting(3DGS) have demonstrated substantial advancements in the domain of novel view synthesis. Nevertheless, the efficacy of these approaches is predominantly contingent upon the availability of well-defined, clear imagery and precise camera pose information. Consequently, they exhibit a pronounced susceptibility to motion blur, which impedes the rendering of sharp images. Event cameras, which measure intensity changes with microsecond temporal precision, possess an inherent robustness to motion-induced blur. This characteristic offers new avenues for 3D reconstruction in challenging scenarios characterized by high-speed motion or low-light conditions. This paper introduces E2D-GS, a novel algorithm for deblurring and reconstruction based on event cameras and 3D Gaussian Splatting. To enhance reconstruction accuracy, our proposed framework leverages event streams to physically model the formation process of motion blur. This is achieved by optimizing the discrepancy between synthesized data and the observed blurry images, while simultaneously recovering the camera’s motion trajectory. Additionally, to enhance robustness in real-world scenarios, this paper proposes a differential consistency module. This module effectively mitigates noise within the event data and regularizes the optimization of Gaussian parameters, thereby improving reconstruction quality under non-ideal conditions. Comprehensive experimental evaluations on both simulated and real-world benchmarks validate the proposed method’s capability to reconstruct latent sharp imagery via the learned 3DGS representations, and further demonstrate its capacity for stable reconstruction under adverse scenarios. The results show that our approach surpasses the performance of previous works.