4DStyleGaussian: Generalizable 4D style transfer with Gaussian splatting

Abstract

3D neural style transfer has gained significant attention for its potential to provide user-friendly stylization with 3D spatial consistency. However, existing 3D style transfer methods often struggle with inference efficiency, generalization, and maintaining temporal consistency when handling dynamic scenes. In this paper, we introduce 4DStyleGaussian, a novel 4D style transfer framework designed to achieve real-time stylization of arbitrary style references while maintaining reasonable content affinity, multi-view consistency, and temporal coherence. Our approach leverages an embedded 4D Gaussian Splatting technique, which is trained utilizing a reversible neural network for reducing content loss and artifacts in the feature distillation process. With the pre-trained 4D embedded Gaussians for efficient and view-consistent rendering, we predict a 4D style transformation matrix that facilitates spatially and temporally consistent style transfer. Experiments demonstrate that our method can achieve high-quality and generalizable stylization for 4D scenarios with enhanced efficiency and spatial-temporal consistency, with 7.1 % lower LPIPS and 2.5× faster inference compared to existing methods.