Multi-order attributes information fusion via hypergraph matching for popular fashion compatibility analysis

Abstract

Popular fashion compatibility modeling aims to quantitatively assess the compatibility of a set of wearable items for everyday pairings and clothing purchases to assist human decision-making, which has garnered extensive academic attention. Earlier approaches that studied garments as a whole could only discern loose relationships between items, resulting in low accuracy and poor interpretability. Although existing state-of-the-art methods attempt to reveal the mechanism of garment compatibility at a fine-grained level by quantifying pairwise attribute compatibility, they overlook the fact that multi-attribute combinations between apparel items tend to play a more salient role in compatibility. Considering the complex and high-order characteristics of compatibility data, we propose a network named MAIF to deeply mine and reveal the intricate compatibility mechanisms of clothing by fusing multi-order attributes compatibility information through hypergraph matching. Specifically, we use the compatibility modeling of top-item and bottom-item as an example. First, we construct an adaptive hypergraph representation module to model the multi-attribute association combinations of individual clothing items and fuse single-attribute variable information to form multi-order attribute association information. Second, we learn the multi-order compatibility information of attributes between clothing items through spatial similarity matching. Considering the varying compatibility impacts caused by different attribute combinations, we construct a dynamic cross-plot matching mechanism to model the impact weights of multi-order attribute compatibility information. Finally, personalized ranking loss is designed to optimize the model parameters using fashion context information. Experimental and user survey studies conducted on the FashionVC and Polyvore-Maryland datasets verified the validity and superiority of MAIF in accurately assessing apparel compatibility, demonstrating its ability to interpret multi-order attribute compatibility information.