Kenin Coloma, A. Ching, A. Choudhary, W. Liao, R. Ross, R. Thakur, L. Ward
{"title":"A New Flexible MPI Collective I/O Implementation","authors":"Kenin Coloma, A. Ching, A. Choudhary, W. Liao, R. Ross, R. Thakur, L. Ward","doi":"10.1109/CLUSTR.2006.311865","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311865","url":null,"abstract":"The MPI-IO standard creates a huge opportunity to break out of the traditional file system I/O methods. As a software layer between the user and the file system, an MPI-IO library can potentially optimize I/O on behalf of the user with little to no user intervention. This is all possible because of the rich data description and communication infrastructure MPI-2 offers. Powerful data descriptions and some of the other desirable features of MPI-2, however, make MPI-IO challenging to implement. By creating a new collective I/O implementation that allows developers to easily tinker and play with new optimizations or combinations of different techniques, research can proceed faster and be quickly and reliably deployed","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116752411","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Marching Towards Nirvana: Configurations for Very High Performance Parallel File Systems","authors":"P. Andrews, C. Jordan, W. Pfeiffer","doi":"10.1109/CLUSTR.2006.311895","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311895","url":null,"abstract":"Over the past 7 years, the San Diego Supercomputer Center has worked to produce the highest possible performance file systems available to the National Science Foundation community in the USA. Most of this was done with GPFS, IBM's parallel file system, but several distinctly different configurations were designed and implemented with numerous lessons learned in the process. All of these systems provided transfer rates in the multiple GB/s range. In this paper, we detail the configurations and their intended modes of operation and, as much as possible, show the resulting performance. We attempt to describe the advantages and disadvantages of each approach with an emphasis on the implications for future systems","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"109 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117200401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fairness and Performance Isolation: an Analysis of Disk Scheduling Algorithms","authors":"Seetharami R. Seelam, P. Teller","doi":"10.1109/CLUSTR.2006.311899","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311899","url":null,"abstract":"An I/O system using a sharing model provides concurrently executing applications shared access to the underlying I/O resources. Although the existing sharing models of these I/O systems are purported to be fair, none of them results in performance isolation. Failing to provide performance isolation results in unpredictable application performance. Unpredictability in application performance hampers providing quality of service guarantees. In this paper, we present a formal analysis of the fairness properties of various disk scheduling algorithms and an experimental evaluation of their performance isolation properties. We show that none of the existing \"fair\" scheduling algorithms provides performance isolation","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126166743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Rex, F. Mietke, W. Rehm, Christoph Raisch, H. Nguyen
{"title":"Improving Communication Performance on InfiniBand by Using Efficient Data Placement Strategies","authors":"R. Rex, F. Mietke, W. Rehm, Christoph Raisch, H. Nguyen","doi":"10.1109/CLUSTR.2006.311864","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311864","url":null,"abstract":"Despite using high-speed network interconnection systems like InfiniBand, the communication overhead for parallel applications is still high. In this paper we show, how such costs can be reduced by choosing appropriate data placement strategies. For large buffers, we propose a transparent placing in hugepages as it can dramatically decrease memory registration overhead and may increase network bandwidth. Thus, we developed a new library that can be preloaded for applications at load time and cares about drawbacks of using hugepages. So we believe that it is the most suitable one in the HPC area for Linux today. But we do not only refer to large buffers as small communication buffers also play a significant role for application behaviour. We show that transfer latencies vary, depending on data placement. All current communication library implementations for InfiniBand do not utilize scatter-gather lists for send and receive operations, but we show that this feature can have a positive impact on latency for small buffers and data aggregation can perform better. Our results show that communication performance of applications may improve more than 10 % using the presented improvements","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115656376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Resource Management for Interactive Jobs in a Grid Environment","authors":"E. Fernández, E. Heymann, M. A. Senar","doi":"10.1109/CLUSTR.2006.311892","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311892","url":null,"abstract":"Most recent grid middleware technologies have been aimed at the execution of sequential batch jobs. However, some users require interactive access when running jobs on grid sites. Execution of these applications on a grid environment is a challenging problem that requires the cooperation of several middleware tools and services. Additional problems arise when this interactive support is intended for parallel applications, which may run remotely across several sites. We provide transparent and reliable support for such applications. Our solution, based on the notion of split execution and interposition agents, allows running applications on a remote machine while some I/O operations are sent back to a home machine. The paper describes how we have applied interposition agents transparently to interactive applications and also describes a simple multiprogramming mechanism that is used to start interactive applications as fast as possible even under heavy occupancy of resources. We provide a performance evaluation of the key elements involved in the execution of interactive jobs","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125146950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Autonomic Management of Clustered Applications","authors":"S. Bouchenak, N. D. Palma, D. Hagimont, C. Taton","doi":"10.1109/CLUSTR.2006.311842","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311842","url":null,"abstract":"Distributed software environments are increasingly complex and difficult to manage, as they integrate various legacy software with proprietary management interfaces. Moreover, the fact that management tasks are performed by humans leads to many configuration errors and low reactivity. This paper presents Jade, a middleware for self-management of distributed software environments. The main principle is to wrap legacy software pieces in components in order to provide a uniform management interface, thus allowing the implementation of management applications. Management applications are used to deploy distributed applications and to autonomously reconfigure them as required","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125174321","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multi-Objective Models for Scheduling Jobs on Parallel Computer Systems","authors":"S. Vasupongayya, S. Chiang","doi":"10.1109/CLUSTR.2006.311873","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311873","url":null,"abstract":"This paper is concerned with the design of goal-oriented scheduling policies that deal with multiple goals on production parallel systems. Several objective models are compared, including two new models that allow an explicit specification of tradeoff among different goals and a hierarchical model studied in our previous paper. We focus on two performance goals commonly placed on production systems: starvation prevention and favoring shorter jobs. In this paper, we study how to best formulate high-level performance goals, in an attempt to provide guideline for defining the objectives","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"41 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114373015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Virtual Structured P2P Network Topology for Distributed Computing","authors":"E. Edi, Mohand Tahar Kechadi, R. McNulty","doi":"10.1109/CLUSTR.2006.311907","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311907","url":null,"abstract":"P2P and grid computing are two paradigms more and more used in today computing environments; their potential to provide better quality of service to users is very promising compared to the cost it involves. This paper presents a hierarchical virtual network topology, built on top of the real existing one and which is used to manage distributed resources in grid environment. The distributed resources are found on the Web following P2P techniques; and so they are very volatile. The virtual topology constructs an efficient and robust virtual machine which will serve as a distributed computing platform. This topology is called TreeP and it is exploited in DGET by Hudzia, B. et al, (2005); a data-grid middleware environment. Here, we study this virtual topology both theoretically and experimentally. We show that this topology is very scalable, robust, load-balanced, and easy to construct and maintain","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"28 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114517364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploiting RDMA operations for Providing Efficient Fine-Grained Resource Monitoring in Cluster-based Servers","authors":"K. Vaidyanathan, Hyun-Wook Jin, D. Panda","doi":"10.1109/CLUSTR.2006.311916","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311916","url":null,"abstract":"Efficiently capturing the resource usage in a shared server environment has been a critical research issue in the past several years. With the amount of resources used by each application becoming more and more divergent and unpredictable, the solution to this problem is becoming increasingly important. In the past, several researchers have come up with a number of techniques which rely on coarse-grained monitoring of resources in order to avoid the overheads associated with fine-grained monitoring. In this paper, we propose a low-overhead efficient fine-grained resource monitoring scheme using the advanced Remote Direct Memory Access (RDMA) operation provided by RDMA-enabled interconnects such as InfiniBand (IBA). We evaluate the relative benefits of our approach against traditional approaches in various environments (including micro-benchmarks as well as real applications such as an auction server based on the RUBiS benchmark and the Ganglia distributed monitoring tool). Our results indicate that our approach for fine-grained monitoring can significantly improve the overall system utilization, thereby resulting in up to 25% improvement in the number of requests the cluster-system can admit","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128654764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Graham, G. Shipman, Brian W. Barrett, R. Castain, G. Bosilca, A. Lumsdaine
{"title":"Open MPI: A High-Performance, Heterogeneous MPI","authors":"R. Graham, G. Shipman, Brian W. Barrett, R. Castain, G. Bosilca, A. Lumsdaine","doi":"10.1109/CLUSTR.2006.311904","DOIUrl":"https://doi.org/10.1109/CLUSTR.2006.311904","url":null,"abstract":"The growth in the number of generally available, distributed, heterogeneous computing systems places increasing importance on the development of user-friendly tools that enable application developers to efficiently use these resources. Open MPI provides support for several aspects of heterogeneity within a single, open-source MPI implementation. Through careful abstractions, heterogeneous support maintains efficient use of uniform computational platforms. We describe Open MPI's architecture for heterogeneous network and processor support. A key design features of this implementation is the transparency to the application developer while maintaining very high levels of performance. This is demonstrated with the results of several numerical experiments","PeriodicalId":259649,"journal":{"name":"2006 IEEE International Conference on Cluster Computing","volume":"94 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2006-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124755497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}