SemAlignRec: A Large Language Model-Driven Cross-Modal Collaborative–Semantic Recommendation Framework

Abstract

Recommender systems are increasingly vital in e-commerce and social media, yet traditional collaborative filtering and deep learning methods still face challenges in fine-grained interest modeling and data sparsity. Large language models (LLMs), with strong semantic understanding and reasoning capabilities, excel in cold-start and long-tail scenarios. However, the modality gap between low-dimensional collaborative signals and LLMs’ contextual embeddings leads to fusion difficulties and semantic distortion. To address this, we propose SemAlignRec, a unified recommendation framework that enhances fusion through a model-agnostic collaborative–semantic alignment mechanism. Specifically: 1) deep semantic encoding and graph structural enhancement: leveraging LLMs to encode user and item textual attributes into high-quality semantic embeddings, which are further refined by a multilayer deep semantic encoder; a graph enhancement module incorporating residual connections, node attention, and graph perturbations strengthens collaborative embeddings and facilitates their fusion with semantic embeddings; and 2) transmodal fusion and self-supervised optimization: integrating multihead attention, gated fusion, and contrastive self-supervision to achieve precise fusion of collaborative and semantic embeddings. Experiments on multiple datasets show that SemAlignRec delivers substantial performance gains, achieving an overall average improvement of about 6.6% over the compared baselines. In particular, compared with the latest baseline DALR, SemAlignRec obtains an average gain of approximately 9.8% on key metrics for the Amazon and Yelp datasets, demonstrating its strong practical potential.