Enhancing fashion e-commerce retrieval: A self-supervised graph-integrated framework for cross-modal image–text alignment

Abstract

Cross-modal retrieval is pivotal for enhancing user experiences in the rapidly evolving fashion e-commerce sector. However, conventional methods often struggle with fine-grained feature alignment and maintaining modal consistency. To overcome these challenges, we propose G-SimX, an innovative framework that synergizes self-supervised contrastive learning via SimCLR with Graph Convolutional Networks (GCN) for robust cross-modal image–text retrieval. By harnessing SimCLR, G-SimX significantly reduces reliance on annotated data, while the integration of GCN facilitates structured relational modeling to capture intricate semantic associations between product images and textual descriptions. Extensive experiments on the FashionGenAttnGAN dataset demonstrate that G-SimX achieves outstanding mAP scores of 0.856 for image-to-text and 0.842 for text-to-image retrieval—improving upon the CLIP baseline by over 20%. Moreover, our approach exhibits exceptional stability with reduced training loss fluctuations (0.161 vs. 0.189) and markedly enhanced robustness against data perturbations (0.92 vs. 0.76). Ablation studies further affirm the contributions of each module, revealing mAP drops of 11.7% and 16.9% upon removing SimCLR and GCN, respectively. With a convergence rate 95.8% faster than existing methods, G-SimX offers a scalable and efficient solution for real-world fashion e-commerce applications.