Self-supervised feature matched virtual try-on

Abstract

Abstract Virtual try-on is a technology that enables users to preview the effect of wearing a target garment without wearing the actual garment. However, existing image-based virtual try-on methods often require additional human parsing or segmentation operations to generate intermediate representations required for garment deformation and texture fusion. These operations not only increase the computational complexity and memory consumption, but also limit the real-time and portability of virtual try-on. Additionally, inaccurate parsing results can lead to misleading final generated images. To overcome these challenges, we propose a self-supervised feature matched virtual try-on network, which can directly generate high-quality try-on results from human body images and target clothing images without any additional input. Specifically, we design an optical flow warp module, which focuses on the optical flow changes between the person image and the clothing image to achieve accurate clothing alignment and deformation. Furthermore, a feature refine warp module is designed to enhance the features of the extracted optical flow information and the original character segmentation and analysis operations, reducing the influence of background clutter features on the content, and ensuring that the wrinkles and deformation of the replacement clothes are close to the original clothes. The feature match module is developed to calculate the feature matching loss of the converted clothing and the generated results of the teacher network and the student network, and the corresponding knowledge is distilled and passed to the student network to assist in self-supervised training. We conduct experiments on the VITON dataset and show that our model can generate high quality and high resolution, and our proposed method outperforms the state-of-the-art virtual try-on methods both qualitatively and quantitatively.