On the modelling of ranking algorithms in probabilistic datalog

Abstract

TF-IDF, BM25, language modelling (LM), and divergence-from-randomness (DFR) are popular ranking models. Providing logical abstraction for information search is important, but the implementation of ranking algorithms in logical abstraction layers such as probabilistic Datalog leads to many challenges regarding expressiveness and scalability. Though the ranking algorithms have probabilistic roots, the ranking score often is not probabilistic, leading to unsafe programs from a probabilistic point of view. In this paper, we describe the evolution of probabilistic Datalog to provide concepts required for modelling ranking algorithms.