Decentralized Ranking in Large-Scale Overlay Networks

Abstract

Modern distributed systems are often characterized by very large scale, poor reliability, and extreme dynamism of the participating nodes, with a continuous flow of nodes joining and leaving the system. In order to develop robust applications in such environments, middleware services aimed at dealing with the inherent unpredictability of the underlying networks are required. One such service is aggregation. In the aggregation problem, each node is assumed to have attributes. The task is to extract global information about these attributes and make it available to the nodes. Examples include the total free storage, the average load, or the size of the network. Efficient protocols for computing several aggregates such as average, count, and variance have already been proposed. In this paper, we consider calculating the rank of nodes, where the set of nodes has to be sorted according to a numeric attribute and each node must be informed about its own rank in the global sorting. This information has a number of applications, such as slicing. It can also be applied to calculate the median or any other percentile. We propose T-Rank, a robust and completely decentralized algorithm for solving the ranking problem with minimal assumptions. Due to the characteristics of the targeted environment, we aim for a probabilistic approach and accept minor errors in the output. We present extensive empirical results that suggest near logarithmic time complexity, scalability and robustness in different failure scenarios.