Frame rate up conversion of 3D video by motion and depth fusion

Abstract

This paper presents a new frame-rate up conversion method for video plus depth representation. The proposed method improves the quality of video using additional depth information. The depth information is utilized in three ways to improve the accuracy of motion estimation. One usage is that depth frame is added in block matching criterion to estimate initial motion. Another usage is to detect object boundary region combining depth value variation and local depth distribution adjustment technique. Finally, motion vector field is refined adaptively by segmenting macroblocks into object regions. Any existing block-based motion-compensated frame interpolation methods may utilize our method to refine motion vector field by using depth frames. Experimental results verify that the proposed method outperforms the conventional motion-compensated frame interpolation algorithms, while preserving object structure.