Improving Conversational Literature Retrieval Quality via Personalized Profile-Based Re-Ranking

Abstract

Academic literature retrieval is constrained by the paradox of “information overload” versus “evidence scarcity”, a tension that deepens when researchers iteratively refine their queries in multiturn conversational settings. To address this challenge, we propose Conversational Literature Personalized Reranking (CLPR), a personalized framework that unifies dense semantic retrieval with personalized user profiling. CLPR first performs a broad high-recall retrieval to collect candidate documents, then compresses conversational history into a concise textual profile that encodes sequential continuity, immediate focus, and longterm research background via a large language model. The generated profile serves as a pseudo-query for a neural crossencoder to produce the final ranking. Cross-domain testing on the public LitSearch (computer science) benchmark confirms its robust generalization, yielding an NDCG@10 of 0.4793. On MedCorpus, a new multi-turn biomedical conversational retrieval benchmark constructed for this study, CLPR attains state-of-the-art performance with P@1 = 0.9497 and NDCG@10 = 0.9271, surpassing the strongest baseline by substantial margins. Ablation shows longterm background cues contribute most, and maintaining a short, uptodate profile across turns outperforms a static one. CLPR therefore delivers accurate, personalized literature retrieval and can accelerate evidence synthesis across scientific domains.