Phase shift guided dynamic view synthesis from monocular video

Abstract

This paper endeavors to address the challenge of synthesizing novel views from monocular videos featuring moving objects, particularly in complex scenes with non-rigid deformations. Existing implicit representations rely on motion estimation in the spatial domain, which often struggle to capture correct temporal dynamics under such conditions. To mitigate the drawback, we propose dynamic positional encoding to represent temporal dynamics as learnable phase shifts and leverage the implicit neural representation (INR) network for iterative optimization. Utilizing optimized phase shifts as guidance enhances the representational capability of the dynamic radiance field, thereby alleviating motion ambiguity and reducing artifacts around moving objects in novel views. This paper also introduces a rational evaluation metric, referred to as “dynamic only+”, for the quantitative assessment of the rendering quality in novel views, focusing on dynamic objects and surrounding regions impacted by motion. Experimental results on multiple challenging datasets demonstrate the favorable performance of the proposed approach over state-of-the-art dynamic view synthesis methods.