Detail-Preserving Video-based Virtual Try-On (DPV-VTON)

Abstract

Virtual Try-on systems enable the try-on of a desired clothing on a target person image. These systems have led to vast research and have attracted commercial interest. However, the existing techniques are image-based systems limited to using an in-shop target clothing from a pre-defined dataset. To address this, we propose a video-based virtual try-on network DPV-VTON, that simulates the try-on using the target cloth extracted from the fashion videos on a target person image, while preserving the details and the characteristics. The core of the DPV-VTON pipeline is made up of (i) Best Frame Selection (BFS) module that extracts the best frame from the video (ii) Clothing Extraction module (CEM) extracts the target clothing from the selected best frame and generates a binary mask. (iii) A virtual try-on module synthesizes a final virtual try-on. Experiments on the existing benchmark datasets and a curated video dataset demonstrate that DPV-VTON generates photo-realistic and visually promising results. The proposed model obtains the lowest FID, LPIPS and the highest SSIM scores compared to the existing systems.