GSRDR-GAN: Global search result diversification ranking approach based on multi-head self-attention and GAN

Search result diversification ranking aims to generate rankings that comprehensively cover multiple subtopics, but existing methods often struggle to balance ranking diversity with relevance and face challenges in modeling document interactions and dealing with limited high-quality training data. While GAN have proven highly successful in fields like computer vision, their application to search result diversification has been limited due to the discrete nature of ranking items and the complex interactions among documents. To address these challenges, we propose GSRDR-GAN, a novel approach that integrates multi-head self-attention with GAN. Our method consists of four key components designed to address the limitations of traditional approaches: the Selected Document State Retriever, the Subtopic Encoder with Multi-head Self-Attention, the Subtopic Decoder with Multi-head Self-Attention, and the Relevance Predictor. First, a self-attention-based feature extraction module is employed to enhance document representations, enabling the model to capture both global and local context effectively. Second, a GAN framework is introduced to improve generalization by generating diverse rankings, mitigating limited high-quality training data. Third, a carefully designed reward function optimizes the trade-off between ranking diversity and relevance, allowing the model to adaptively prioritize these competing objectives during training. Notably, the method improves the generator’s stability and the diversity of search results by reducing training variance, even without pre-trained models. Extensive experiments on the TREC Web Track dataset demonstrate that the proposed GSRDR-GAN method significantly enhances result diversity, achieving relative improvements of 1.7% in $\alpha$-nDCG, 3.0% in ERR-IA, 3.3% in NRBP, and 0.9% in S-rec over strong baseline methods. Ablation studies and comparative analyses of different reward computation methods further validate the effectiveness of the proposed approach.