Unsupervised Fashion Style Learning by Solving Fashion Jigsaw Puzzles

Abstract

Fashion style learning is the basis for many tasks in fashion AI, such as clothing recommendations, fashion trend analysis and popularity prediction. Most of the existing methods rely on the quality and quantity of the annotations. This paper proposes an efficient two-step unsupervised fashion style learning framework with "Fashion Jigsaw" task and centroid-based density clustering algorithm. First, we design the "Fashion Jigsaw" unsupervised learning task according to the distribution of fashion elements in full-body fashion images. By splitting and recovering fashion images, we pre-train a model that can extract both intra-image and inter-image information. Second, we propose a centroid-based density clustering algorithm and introduce the concept of "centroid" to cluster fashion image features and represent fashion styles. Meanwhile, we keep the noise features to discover the newly sprouted fashion styles. Experiment results demonstrate the effectiveness of our proposed method.