2016 IEEE International Conference on Cloud Engineering (IC2E)最新文献_第4页

Awan: Locality-Aware Resource Manager for Geo-Distributed Data-Intensive Applications Awan:地理分布数据密集型应用的位置感知资源管理器

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-04-04 DOI: 10.1109/IC2E.2016.15

A. Jonathan, A. Chandra, J. Weissman

{"title":"Awan: Locality-Aware Resource Manager for Geo-Distributed Data-Intensive Applications","authors":"A. Jonathan, A. Chandra, J. Weissman","doi":"10.1109/IC2E.2016.15","DOIUrl":"https://doi.org/10.1109/IC2E.2016.15","url":null,"abstract":"Today, many organizations need to operate on data that is distributed around the globe. This is inevitable due to the nature of data that is generated in different locations such as video feeds from distributed cameras, log files from distributed servers, and many others. Although centralized cloud platforms have been widely used for data-intensive applications, such systems are not suitable for processing geo-distributed data due to high data transfer overheads. An alternative approach is to use an Edge Cloud which reduces the network cost of transferring data by distributing its computations globally. While the Edge Cloud is attractive for geo-distributed data-intensive applications, extending existing cluster computing frameworks to a wide-area environment must account for locality. We propose Awan : a new locality-aware resource manager for geo-distributed data-intensive applications. Awan allows resource sharing between multiple computing frameworks while enabling high locality scheduling within each framework. Our experiments with the Nebula Edge Cloud on PlanetLab show that Awan achieves up to a 28% increase in locality scheduling which reduces the average job turnaround time by approximately 18% compared to existing cluster management mechanisms.","PeriodicalId":430893,"journal":{"name":"2016 IEEE International Conference on Cloud Engineering (IC2E)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127994033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12

LOGAN: Problem Diagnosis in the Cloud Using Log-Based Reference Models LOGAN:使用基于日志的参考模型进行云中的问题诊断

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-04-01 DOI: 10.1109/IC2E.2016.12

Byungchul Tak, S. Tao, L. Yang, Chao Zhu, Yaoping Ruan

{"title":"LOGAN: Problem Diagnosis in the Cloud Using Log-Based Reference Models","authors":"Byungchul Tak, S. Tao, L. Yang, Chao Zhu, Yaoping Ruan","doi":"10.1109/IC2E.2016.12","DOIUrl":"https://doi.org/10.1109/IC2E.2016.12","url":null,"abstract":"Problem diagnosis is one crucial aspect in the cloud operation that is becoming increasingly challenging. On the one hand, the volume of logs generated in today's cloud is overwhelmingly large. On the other hand, cloud architecture becomes more distributed and complex, which makes it more difficult to troubleshoot failures. In order to address these challenges, we have developed a tool, called LOGAN, that enables operators to quickly identify the log entries that potentially lead to the root cause of a problem. It constructs behavioral reference models from logs that represent the normal patterns. When problem occurs, our tool enables operators to inspect the divergence of current logs from the reference model and highlight logs likely to contain the hints to the root cause. To support these capabilities we have designed and developed several mechanisms. First, we developed log correlation algorithms using various IDs embedded in logs to help identify and isolate log entries that belong to the failed request. Second, we provide efficient log comparison to help understand the differences between different executions. Finally we designed mechanisms to highlight critical log entries that are likely to contain information pertaining to the root cause of the problem. We have implemented the proposed approach in a popular cloud management system, OpenStack, and through case studies, we demonstrate this tool can help operators perform problem diagnosis quickly and effectively.","PeriodicalId":430893,"journal":{"name":"2016 IEEE International Conference on Cloud Engineering (IC2E)","volume":"53 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133438739","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 39

Not All Joules are Equal: Towards Energy-Efficient and Green-Aware Data Processing Frameworks 并非所有焦耳都是平等的:迈向节能和绿色意识数据处理框架

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-04-01 DOI: 10.1109/IC2E.2016.17

Zhaojie Niu, Bingsheng He, Fangming Liu

{"title":"Not All Joules are Equal: Towards Energy-Efficient and Green-Aware Data Processing Frameworks","authors":"Zhaojie Niu, Bingsheng He, Fangming Liu","doi":"10.1109/IC2E.2016.17","DOIUrl":"https://doi.org/10.1109/IC2E.2016.17","url":null,"abstract":"Interests have been growing in integrating renewable energy into data centers, which attracts many research efforts in developing green-aware algorithms and systems. However, little attention was paid to the efficiency of each joule consumed by data center workloads. In fact, not all joules are equal in the sense that the amount of work that can be done by a joule can vary significantly in data centers. Ignoring this fact leads to significant energy waste (by 25% of the total energy consumption in Hadoop YARN on a Facebook production trace according to our study). In this paper, we investigate how to exploit such joule efficiency to maximize the benefits of renewable energy for MapReduce framework. We develop job/task scheduling algorithms with a particular focus on the factors on joule efficiency in the data center, including the energy efficiency of MapReduce workloads, renewable energy supply and the battery usage. We further develop a simple yet effective performance-energy consumption model to guide our scheduling decisions. We have implemented GreenMR, an energy-efficient and green-aware MapReduce framework, on top of Hadoop YARN. The experiments demonstrate the accuracy of our models, and the effectiveness of our energy-efficient and green-aware optimizations over Hadoop YARN and a state-ofthe-art green-aware Hadoop YARN implementation.","PeriodicalId":430893,"journal":{"name":"2016 IEEE International Conference on Cloud Engineering (IC2E)","volume":"27 8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133659709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 13

A Reinforcement Learning-Based Power Management Framework for Green Computing Data Centers 基于强化学习的绿色计算数据中心电源管理框架

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-04-01 DOI: 10.1109/IC2E.2016.33

X. Lin, Yanzhi Wang, Massoud Pedram

引用次数: 28

Supporting On-demand Elasticity in Distributed Graph Processing 支持分布式图处理中的按需弹性

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-04-01 DOI: 10.1109/IC2E.2016.31

Mayank Pundir, Manoj Kumar, Luke M. Leslie, Indranil Gupta, R. Campbell

{"title":"Supporting On-demand Elasticity in Distributed Graph Processing","authors":"Mayank Pundir, Manoj Kumar, Luke M. Leslie, Indranil Gupta, R. Campbell","doi":"10.1109/IC2E.2016.31","DOIUrl":"https://doi.org/10.1109/IC2E.2016.31","url":null,"abstract":"While distributed graph processing engines have become popular for processing large graphs, these engines are typically configured with a static set of servers in the cluster. In other words, they lack the flexibility to scale-out or scale-in the number of servers, when requested to do so by the user. In this paper, we propose the first techniques to make distributed graph processing truly elastic. While supporting on-demand scale-out/in operations, we meet three goals: i) perform scale-out/in without interrupting the graph computation, ii) minimize the background network overhead involved in the scale-out/in, and iii) mitigate stragglers by maintaining load balance across servers. We present and analyze two techniques called Contiguous Vertex Repartitioning (CVR) and Ring-based Vertex Repartitioning (RVR) to address these goals. We implement our techniques in the LFGraph distributed graph processing system, and incorporate several systems optimizations. Experiments performed with multiple graph benchmark applications on a real graph indicate that our techniques perform within 9% and 21% of the optimum for scale-out and scale-in operations, respectively.","PeriodicalId":430893,"journal":{"name":"2016 IEEE International Conference on Cloud Engineering (IC2E)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130752705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 16

Cost-Aware Scalability of Applications in Public Clouds 公共云中应用程序的成本感知可扩展性

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-04-01 DOI: 10.1109/IC2E.2016.23

D. Moldovan, Hong Linh Truong, S. Dustdar

{"title":"Cost-Aware Scalability of Applications in Public Clouds","authors":"D. Moldovan, Hong Linh Truong, S. Dustdar","doi":"10.1109/IC2E.2016.23","DOIUrl":"https://doi.org/10.1109/IC2E.2016.23","url":null,"abstract":"Scalable applications deployed in public clouds can be built from a combination of custom software components and public cloud services. To meet performance and/or cost requirements, such applications can scale-out/in their components during run-time. When higher performance is required, new component instances can be deployed on newly allocated cloud services (e.g., virtual machines). When the instances are no longer needed, their services can be deallocated to decrease cost. However, public cloud services are usually billed over predefined time and/or usage intervals, e.g., per hour, per GB of I/O. Thus, it might not be cost efficient to scale-in public cloud applications at any moment in time, without considering their billing cycles. In this work we aid developers of scalable applications for public clouds to monitor their costs, and develop cost-aware scalability controllers. We introduce a model for capturing the pricing schemes of cloud services. Based on the model we determine and evaluate the application's costs depending on its used cloud services and their billing cycles. We further evaluate cost efficiency of cloud applications, analyzing which application component is cost efficient to deallocate and when. We evaluate our approach on a scalable platform for IoT, deployed in Flexiant, one of the leading European public cloud providers. We show that cost-aware scalability can achieve higher application stability and performance, while reducing its operation costs.","PeriodicalId":430893,"journal":{"name":"2016 IEEE International Conference on Cloud Engineering (IC2E)","volume":"98 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126101026","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

Barrier-Aware Max-Min Fair Bandwidth Sharing and Path Selection in Datacenter Networks 数据中心网络中障碍感知的最大最小公平带宽共享和路径选择

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-04-01 DOI: 10.1109/IC2E.2016.35

Li Chen, Baochun Li, Bo Li

{"title":"Barrier-Aware Max-Min Fair Bandwidth Sharing and Path Selection in Datacenter Networks","authors":"Li Chen, Baochun Li, Bo Li","doi":"10.1109/IC2E.2016.35","DOIUrl":"https://doi.org/10.1109/IC2E.2016.35","url":null,"abstract":"In production datacenters operated by Web service providers such as Google, multiple data parallel applications, such as MapReduce, are employed to facilitate data processing at a large scale, with a strong demand for intra-datacenter bandwidth in their communication stages. A noteworthy phenomenon in these applications is the presence of barriers, which implies that a job will not finish until the last task completes. Existing flow-level sharing in datacenter networks is not designed and optimized to meet such application-level needs. In this paper, we promote the awareness of application barriers in the design of both bandwidth allocation and path selection strategies. In particular, we propose the notion of application-level fairness when bandwidth is allocated, with favorable properties of performance-centric max-min fairness and Pareto efficiency. Further, we show that both application-level performance and resource utilization can be further improved by considering path selection as well. With our implementation in the Mininet emulation testbed and large-scale simulations, we demonstrate that our new barrier-aware strategy for fair bandwidth sharing and path selection significantly outperforms barrier-agnostic strategy when application performance is concerned.","PeriodicalId":430893,"journal":{"name":"2016 IEEE International Conference on Cloud Engineering (IC2E)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133046263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 9

Container-Based Cloud Virtual Machine Benchmarking 基于容器的云虚拟机基准测试

2016 IEEE International Conference on Cloud Engineering (IC2E) Pub Date : 2016-01-15 DOI: 10.1109/IC2E.2016.28

B. Varghese, Lawan Thamsuhang Subba, Long Thai, A. Barker

{"title":"Container-Based Cloud Virtual Machine Benchmarking","authors":"B. Varghese, Lawan Thamsuhang Subba, Long Thai, A. Barker","doi":"10.1109/IC2E.2016.28","DOIUrl":"https://doi.org/10.1109/IC2E.2016.28","url":null,"abstract":"With the availability of a wide range of cloud Virtual Machines (VMs) it is difficult to determine which VMs can maximise the performance of an application. Benchmarking is commonly used to this end for capturing the performance of VMs. Most cloud benchmarking techniques are typically heavyweight - time consuming processes which have to benchmark the entire VM in order to obtain accurate benchmark data. Such benchmarks cannot be used in real-time on the cloud and incur extra costs even before an application is deployed. In this paper, we present lightweight cloud benchmarking techniques that execute quickly and can be used in near real-time on the cloud. The exploration of lightweight benchmarking techniques are facilitated by the development of DocLite - Docker Container-based Lightweight Benchmarking. DocLite is built on the Docker container technology which allows a user-defined portion (such as memory size and the number of CPU cores) of the VM to be benchmarked. DocLite operates in two modes, in the first mode, containers are used to benchmark a small portion of the VM to generate performance ranks. In the second mode, historic benchmark data is used along with the first mode as a hybrid to generate VM ranks. The generated ranks are evaluated against three scientific high-performance computing applications. The proposed techniques are up to 91 times faster than a heavyweight technique which benchmarks the entire VM. It is observed that the first mode can generate ranks with over 90% and 86% accuracy for sequential and parallel execution of an application. The hybrid mode improves the correlation slightly but the first mode is sufficient for benchmarking cloud VMs.","PeriodicalId":430893,"journal":{"name":"2016 IEEE International Conference on Cloud Engineering (IC2E)","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121363757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 36