Cluster Spanning with Virtual Environments

Abstract

Summary form only given. The ability to easily span parallel and distributed jobs over multiple physical clusters is a potentially attractive proposal. Such an ability would allow researchers to pool all available cluster resources at a given site. Multiple clusters at a single research site has become the norm whether in an academic, government, or industrial environment. However, there are substantial barriers to transparently spanning clusters in this environment, not the least of which are authentication, authorization, sharing data, library mismatching, version skew, compiler access, environment variables, and networking. These problems arise from the heterogeneity of software environments, and in order to circumvent many of these issues, we look at visualization, and its potential use to create dynamic virtual clusters in order to provide a consistent, transparent environment for job spanning. The Grid computing community has addressed the problem of spanning jobs across distributed resources. However, grid solutions tend to focus on authorization, authentication, and data migration problems associated with far flung resources communicating over public network links. In a campus grid environment, where clusters can be directly connected via internal networks, many of these problems disappear. An alternative to the grid approach way to accomplish this spanning is to use common visualization techniques to create virtual clusters made up of one or more physical clusters. In this work, we examine the ontology of visualization techniques from the least amount of visualization to the most, and evaluate their suitability for building dynamic virtual clusters