Clothing Image Retrieval Based on Parts Detection and Segmentation

Abstract

With the rapid development of E-commerce, more and more users are buying clothes through the Internet, and "image search" for clothing images has become a popular research direction. The current "image search" technology mainly relies on the results of feature extraction of the whole image, but cannot focus on the parts of the clothing, and the background of the clothing image is generally complex, resulting in low accuracy of clothing image retrieval, so we propose a retrieval method based on clothing image detection and segmentation. Firstly, Mask R-CNN is used to detect and segment the image to get the information of garment body, collar parts, sleeve category and pocket positions, then VGG16 is used to extract 512-dimensional features from the garment body and collar parts, based on this information, the similarity between the garment to be retrieved and the garment in the database is calculated one by one. We calculate the similarity by weighting the cosine similarity of 512-dimensional features of the garment body and collar, as well as the similarity of the sleeves and pockets. The search results are presented to the user according to the descending order of similarity. The experimental results show that the method can focus on the whole garment as well as their parts, thus enabling retrieval based on garment style. It also allows users to adjust the weights of each part and can return the search results that best meet their individual needs