A Novel Hierarchical Model-Based Frame Rate Up-Conversion via Spatio-temporal Conditional Random Fields

Abstract

In this paper, a hierarchical model-based approach to frame rate-up conversion is presented. Given a sequence of consecutive video frames, a Spatio-Temporal Conditional Random Field (ST-CRF) is trained to capture both the motion and shape characteristics of objects within consecutive frames. A hierarchical tree is then constructed via hierarchical segmentation that sub-divides frames into regions based on color intensity and regional velocity. A hierarchical sampling approach is then introduced to construct new intermediate frames between adjacent video frames, where estimated intermediate frames are constructed at each level of a hierarchical tree constructed such that the probability of the ST-CRF is maximized. Preliminary results using videos with different motion characteristics show that the proposed approach has potential for producing intermediate frames with high visual quality.