Advancing Inter-cloud Resource Discovery Based on Past Service Experiences of Transient Resource Clustering

Abstract

Resource discovery in large-scale distributed systems is a challenging issue, mainly due to the traditional centralized topologies in which the nodes of these systems are typically organized. This unified approach has been proven to be effectual for cluster-based grids and clouds but questionable for large scale, heterogeneous and dynamically interoperable e-infrastructure such as grids and inter-clouds. The latter e-infrastructures are less tolerant with regards to scalability, elasticity and flexibility. In this work, we explore the decentralized distributed search protocols that transform nodes of large size distributed systems to act as both clients and servers. Specifically, we extend the use of a job profile specification that is generated during the user job submission process. The exploitation of the job profile will allow us to orchestrate and group different user submissions into transient inter-cloud brokering groups that represent a temporary resource cluster according to the current service submission characteristics. We suggest the clustering submissions in an inter-cloud system with the view of forming notional short-term grouped nodes that may reform over the time. In this way, the resource discovery process is becoming dynamically, that is to say, a transient group of nodes is required advancing a single request based on current time, opposed to multiple requests as currently happens. We further propose an architecture that implies nodes orchestration based on previous resource requests as well as we model a service meta-registry for inter-cloud systems to store relevant information to service submission.