{"title":"Gang Migration of Virtual Machines Using Cluster-wide Deduplication","authors":"Umesh Deshpande, Brandon Schlinker, Eitan Adler, Kartik Gopalan","doi":"10.1109/CCGrid.2013.39","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.39","url":null,"abstract":"Gang migration refers to the simultaneous live migration of multiple Virtual Machines (VMs) from one set of physical machines to another in response to events such as load spikes and imminent failures. Gang migration generates a large volume of network traffic and can overload the core network links and switches in a data center. In this paper, we present an approach to reduce the network overhead of gang migration using global deduplication (GMGD). GMGD identifies and eliminates the retransmission of duplicate memory pages among VMs running on multiple physical machines in the cluster. We describe the design, implementation and evaluation of a GMGD prototype using QEMU/KVM VMs. Evaluations on a 30-node Gigabit Ethernet cluster having 10GigE core links shows that GMGD can reduce the network traffic on core links by up to 65% and the total migration time of VMs by up to 42% when compared to the default migration technique in QEMU/KVM. Furthermore, GMGD has a smaller adverse performance impact on network-bound applications.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132305823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Simulating Application Workflows and Services Deployed on the European Grid Infrastructure","authors":"S. Camarasu-Pop, T. Glatard, H. Benoit-Cattin","doi":"10.1109/CCGrid.2013.13","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.13","url":null,"abstract":"This paper presents an end-to-end SimGrid-based simulation of a Monte-Carlo computation deployed on the European Grid Infrastructure. We describe a simulation framework allowing to replay executions from real traces. Middleware services, namely the file catalog, storage elements, the DIRAC pilot-job system, and the MOTEUR workflow engine are simulated by SimGrid processes. The deployment of pilot jobs, performed on EGI by the gLite WMS and batch queues, is simulated by a random selection of platform hosts, and a matching of their latencies and failure times with real traces. The Monte-Carlo application workflow is calibrated to address performance discrepancies between the real and simulated network and CPUs. The simulation is evaluated against real executions of the GATE Monte-Carlo application. Results show that SimGrid can be used to study the performance of applications running on EGI. Simulated and real make spans are consistent, and conclusions drawn in production about the influence of application parameters such as the check pointing frequency and the number of mergers are also made in simulation. This opens the door to better and faster experimentation on production grids.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117028223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Energy Efficient VM Scheduling for Cloud Data Centers: Exact Allocation and Migration Algorithms","authors":"Chaima Ghribi, Makhlouf Hadji, D. Zeghlache","doi":"10.1109/CCGrid.2013.89","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.89","url":null,"abstract":"This paper presents two exact algorithms for energy efficient scheduling of virtual machines (VMs) in cloud data centers. Modeling of energy aware allocation and consolidation to minimize overall energy consumption leads us to the combination of an optimal allocation algorithm with a consolidation algorithm relying on migration of VMs at service departures. The optimal allocation algorithm is solved as a bin packing problem with a minimum power consumption objective. It is compared with an energy aware best fit algorithm. The exact migration algorithm results from a linear and integer formulation of VM migration to adapt placement when resources are released. The proposed migration is general and goes beyond the current state of the art by minimizing both the number of migrations needed for consolidation and energy consumption in a single algorithm with a set of valid inequalities and conditions. Experimental results show the benefits of combining the allocation and migration algorithms and demonstrate their ability to achieve significant energy savings while maintaining feasible convergence times when compared with the best fit heuristic.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125106798","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tetsuya Hoshino, N. Maruyama, S. Matsuoka, R. Takaki
{"title":"CUDA vs OpenACC: Performance Case Studies with Kernel Benchmarks and a Memory-Bound CFD Application","authors":"Tetsuya Hoshino, N. Maruyama, S. Matsuoka, R. Takaki","doi":"10.1109/CCGrid.2013.12","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.12","url":null,"abstract":"OpenACC is a new accelerator programming interface that provides a set of OpenMP-like loop directives for the programming of accelerators in an implicit and portable way. It allows the programmer to express the offloading of data and computations to accelerators, such that the porting process for legacy CPU-based applications can be significantly simplified. This paper focuses on the performance aspects of OpenACC using two micro benchmarks and one real-world computational fluid dynamics application. Both evaluations show that in general OpenACC performance is approximately 50% lower than CUDA. However, for some applications it can reach up to 98% with careful manual optimizations. The results also indicate several limitations of the OpenACC specification that hamper full use of the GPU hardware resources, resulting in a significant performance gap when compared to a fully tuned CUDA code. The lack of a programming interface for the shared memory in particular results in as much as three times lower performance.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122524793","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Linlin Wu, S. Garg, R. Buyya, Chao-Wen Chen, Steven Versteeg
{"title":"Automated SLA Negotiation Framework for Cloud Computing","authors":"Linlin Wu, S. Garg, R. Buyya, Chao-Wen Chen, Steven Versteeg","doi":"10.1109/CCGrid.2013.64","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.64","url":null,"abstract":"A Service Level Agreement (SLA) is a legal contract between parties to ensure the Quality of Service (QoS) are provided the providers to the customers. A SLA negotiation between participants assists in defining the QoS requirements of critical service-based processes. However, the negotiation process for customers is a significant task particularly when there are multiple SaaS providers in the Cloud market, as service cost and quality are constantly changing and consumers have varying needs. Therefore, we propose a novel automated negotiation framework where a SaaS broker is utilized as the one-stop-shop for customers to achieve the required service efficiently when negotiating with multiple providers. The automated negotiation framework facilitates intelligent bilateral bargaining of SLAs between a SaaS broker and multiple providers to achieve different objectives for different participants. To maximize profit and improve customer satisfaction levels for the broker, we propose the design of counter offer generation strategies and decision making heuristics that take into account time, market constraints and trade-off between QoS parameters. Our negotiation heuristics are evaluated by extensive experimental studies of our framework using data from a real Cloud provider.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121603748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Stretch Out and Compact: Workflow Scheduling with Resource Abundance","authors":"Young Choon Lee, Albert Y. Zomaya","doi":"10.1109/CCGrid.2013.55","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.55","url":null,"abstract":"Resource abundance is apparent in today's multi-core era. Workflow applications common in science and engineering can take great advantage of such ample resource capacity. Yet, existing workflow scheduling algorithms have been mostly designed on the traditional and contrasting premise of scarce resource capacity relative to what applications require. In this paper, we address the problem of workflow scheduling exploiting resource abundance not only for performance, but also for resource efficiency. We first present the critical-path-first scheduling algorithm, which efficiently stretches out the schedule to proactively preserve critical path length, the shortest possible time of completion. We then develop an algorithm to compact the schedule for resource efficiency. The schedule is compacted by rearranging tasks making use of idle/inefficiency slots present in the schedule due to precedence constraints (synchronization). We have compared the performance of our scheduling algorithm with three previous algorithms and evaluated the efficacy of our schedule compaction algorithm by applying it to those four scheduling algorithms including our own. Experiments were conducted in a simulated environment with various real-world scientific workflows. Results show that our scheduling algorithm achieves a great performance to resource usage ratio. Further, our schedule compaction algorithm reduces resource usage by 33% on average ranging from 11.3% to 90.7% with no make span increases.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130828524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Novel Checkpointing Scheme for Amazon EC2 Spot Instances","authors":"Sunirmal Khatua, N. Mukherjee","doi":"10.1109/CCGrid.2013.71","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.71","url":null,"abstract":"Amazon's spot instances allow customers to bid on unused Amazon EC2 capacity and run those instances for as long as their bid exceeds the current spot price. Customers may expect their services at lower cost with spot instances compared to on-demand or reserved. However, the reliability is compromised since the instances providing the service may become unavailable at any time. In this paper, we study various check pointing schemes that can be used with spot instances. Also we devise some algorithms for check pointing scheme on top of application-centric resource provisioning framework that increase the reliability while reducing the cost significantly.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131070512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"User-Level Remote Memory Paging for Multithreaded Applications","authors":"H. Midorikawa, Yuichiro Suzuki, M. Iwaida","doi":"10.1109/CCGrid.2013.63","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.63","url":null,"abstract":"The new page swap mechanism is introduced to resolve an inconsistent page problem for multithreaded applications in user-level remote paging systems. According to the evaluations, its overhead is limited and it can be applicable to actual use for multithreaded applications.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132737022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bi-Hadoop: Extending Hadoop to Improve Support for Binary-Input Applications","authors":"Xiao Yu, Bo Hong","doi":"10.1109/CCGrid.2013.56","DOIUrl":"https://doi.org/10.1109/CCGrid.2013.56","url":null,"abstract":"The MapReduce programming model, along with its open-source implementation - Hadoop - has provided a cost effective solution for many data-intensive applications. Hadoop stores data distributively and exploits data locality by assigning tasks to where data is stored. Many data-intensive applications, however, require two (or more) input data for each of their tasks. Such applications pose significant challenges for Hadoop as the inputs to one task often reside on multiple nodes, and Hadoop is unable to discover data locality in this scenario. This often leads to excessive data transfers and significant degradations in application performance. In this paper, we present Bi-Hadoop, an efficient extension of Hadoop to better support binary-input applications. Bi-Hadoop integrates an easy-to-use user interface, a binary-input aware task scheduler, and a caching subsystem. Extensive experiments show that Bi-Hadoop can significantly improve the execution of binary-input applications by reducing the data transfer overhead, and outperforms existing Hadoop by up to 3.3x.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124389467","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards an Optimized Big Data Processing System","authors":"Bogdan Ghit, A. Iosup, D. Epema","doi":"10.1109/CCGRID.2013.53","DOIUrl":"https://doi.org/10.1109/CCGRID.2013.53","url":null,"abstract":"Scalable by design to very large computing systems such as grids and clouds, MapReduce is currently a major big data processing paradigm. Nevertheless, existing performance models for MapReduce only comply with specific workloads that process a small fraction of the entire data set, thus failing to assess the capabilities of the MapReduce paradigm under heavy workloads that process exponentially increasing data volumes. The goal of my PhD is to build and analyze a scalable and dynamic big data processing system, including storage (distributed file system), execution engine (MapReduce), and query language (Pig). My contributions for the first two years of PhD research are the following: 1) the design and implementation of a resource management system part of a MapReduce-based processing system for deploying and resizing MapReduce clusters over multicluster systems, 2) the design and implementation of a benchmarking tool for the MapReduce processing system, and 3) the evaluation and modeling of MapReduce using workloads with very large data sets. Furthermore, based on the first two years research, we will optimize the MapReduce system to efficiently process terabytes of data.","PeriodicalId":313983,"journal":{"name":"2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129044480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}