Towards building Urdu language document retrieval framework

Abstract

Research on document retrieval has mainly focused on high-resource languages like English, with limited attention to low-resource languages such as Urdu. Urdu, the 10th-most spoken language globally, with 230 million speakers, faces significant IR challenges due to its complex script, word variations, and lack of standardization. This study introduces U-RR² (Urdu Document Retrieval & Ranking Framework), a two-stage retrieval framework integrating traditional IR models with advanced feature representation techniques to improve Urdu document retrieval. In the first stage, documents and queries are represented using a combination of TF-IDF and embedding methods such as Word2Vec, FastText, and multilingual BERT (mBERT). After that, multiple retrieval models—Vector Space Model, BM25, DFR, and Jelinek-Mercer smoothing—are evaluated, and the best-performing model is selected for initial retrieval. The second stage refines rankings using SVMrank with a feature ensemble technique for improved relevance. Evaluation on CURE, ROSHNI, and UIR_21 benchmarks demonstrates our framework's effectiveness. Our weighted Word2Vec approach achieves MAP scores of 0.78, 0.81, and 0.80, and P@10 scores of 0.79, 0.82, and 0.81. The feature ensemble further improves performance, reaching F₁ scores of 0.89, 0.90, and 0.81, significantly outperforming baselines and establishing a new state-of-the-art for Urdu document retrieval.