Distributed spectral cluster management: a method for building dynamic publish/subscribe systems

Abstract

In recent years peer-to-peer (P2P) networking has gained high popularity for large-scale content distribution. Prominent systems expect a large user base with rather diversified demands. Yet it is highly challenging to achieve scalability without sacrificing the expressiveness of queries in such systems. This paper proposes distributed spectral cluster management, an approach which adapts the techniques from spectral graph theory to work in distributed settings. The proposed approach is applied to content-based publish/subscribe to i) significantly reduce the cost for event dissemination by clustering subscribers exploiting the similarity of events, ii) preserve the expressiveness of the subscription language, and iii) perform robustly in the presence of workload variations. The evaluations analyze the accuracy of the proposed distributed spectral mechanisms and show their effectiveness to significantly reduce the efforts to disseminate events under many practical workloads.