PIRD: P2P-Based Intelligent Resource Discovery in Internet-Based Distributed Systems

Abstract

Internet-based distributed systems enable globally scattered resources to be collectively pooled and used in a cooperative manner to achieve unprecedented petascale super computing capabilities. Numerous resource discovery approaches have been proposed to help achieve this goal. To report or discover a multi-attribute resource, most approaches use multiple messages with each message for an attribute, leading to high overhead. Anther approach can reduce multi-attribute to one index, but it is not practically effective in an environment with a large number of different resource attributes. Furthermore, few approaches are able to locate resources geographically close to the requesters, which is critical to system performance. This paper presents a P2P-based intelligent resource discovery (PIRD) mechanism that weaves all attributes into a set of indices using locality sensitive hashing, and then maps the indices to a structured P2P. It further incorporates Lempel-Ziv-Welch algorithm to compress attribute information for higher efficiency. In addition, it helps to search resources geographically close to requesters by relying on a hierarchical P2P structure. PIRD significantly reduces overhead and improves the efficiency and effectiveness of resource discovery. Theoretical analysis and simulation results demonstrate the efficiency of PIRD in comparison with other approaches. It dramatically reduces overhead and yields significant improvements on the efficiency of resource discovery.