{"title":"Auto-scaling of virtual resources for scientific workflows on hybrid clouds","authors":"Younsun Ahn, Yoonhee Kim","doi":"10.1145/2608029.2608036","DOIUrl":"https://doi.org/10.1145/2608029.2608036","url":null,"abstract":"Cloud computing technology enables applications to employ scalable resources dynamically. Scientists can promote large-scale scientific computational experiments over cloud environment. It is essential for many-task-computing (MTC) to certificate stable executions of applications even rapid changes of vital status of physical resources and furnish high performance resources in a long period. Auto-scaling with virtualization provides efficient and integrated cloud resource utilization. Auto-scaling issues have been actively studied as effective resource management in order to utilize large-scale data center in a good shape but most of the auto-scaling methods just easily support performance metrics such as CPU utilization and data transfer latency but seldom consider execution deadline or characteristics of an application. We propose an auto-scaling method that finishes all tasks by user specified deadline. We accomplish our goal by dynamically allocating VMs to maximize resource utilization while meeting a deadline and considering task dependency and data transfer time in workflow application. We have evaluated our auto-scaling method with protein annotation workflow application which tasks are specified as a workflow in hybrid cloud environment. The results of a simulation show the method performs automatically resource allocation actually needed satisfying deadline constraints.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126472875","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
F. Berghaus, R. Desmarais, I. Gable, C. Leavett-Brown, M. Paterson, R. P. Taylor, A. Charbonneau, R. Sobie
{"title":"HEP computing in a context-aware cloud environment","authors":"F. Berghaus, R. Desmarais, I. Gable, C. Leavett-Brown, M. Paterson, R. P. Taylor, A. Charbonneau, R. Sobie","doi":"10.1145/2608029.2608035","DOIUrl":"https://doi.org/10.1145/2608029.2608035","url":null,"abstract":"This paper describes the use of a distributed cloud computing system for high energy physics (HEP) applications. The system is composed of IaaS clouds integrated into a unified infrastructure that has been in production for over two years. It continues to expand in scale and sites, encompassing more than twenty clouds on three continents. We are prototyping a new context-aware architecture that enables the virtual machines to make connections to both software and data repositories based on geolocation information. The new design will significantly enhance the ability of the system to scale to higher workloads and run data-intensive applications. We review the operation of the production system and describe our work towards a context-aware cloud system.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129292291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A distributed architecture for intra- and inter-cloud data management","authors":"I. Kelley","doi":"10.1145/2608029.2608037","DOIUrl":"https://doi.org/10.1145/2608029.2608037","url":null,"abstract":"When envisioning \"The Cloud,\" one is often presented with an idyllic black box of functionality that seamlessly stores arbitrary amounts of data while providing endless CPU cycles.\u0000 However, Cloud deployments are often partitioned along institutional and middleware boundaries that link computation and storage infrastructures. Decoupling storage from computation resource providers allows for greater flexibility in resource provisioning and new data storage paradigms to emerge.\u0000 This paper proposes a decentralized data management architecture that facilitates interoperability between Clouds and other heterogeneous systems. The goals of this research are to augment the latent storage capacity within provisioned Cloud VMs with existing institutional resources to build low-cost Storage Clouds for scientific computing.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133104304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dennis Gannon, D. Fay, Daron Green, Kenji Takeda, Wenming Yi
{"title":"Science in the cloud: lessons from three years of research projects on microsoft azure","authors":"Dennis Gannon, D. Fay, Daron Green, Kenji Takeda, Wenming Yi","doi":"10.1145/2608029.2608030","DOIUrl":"https://doi.org/10.1145/2608029.2608030","url":null,"abstract":"Microsoft Research is now in its fourth year of awarding Windows Azure cloud resources to the academic community. As of April 2014, over 200 research projects have started. In this paper we review the results of this effort to date. We also characterize the computational paradigms that work well in public cloud environments and those that are usually disappointing. We also discuss many of the barriers to successfully using commercial cloud platforms in research and ways these problems can be overcome.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"43 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124472957","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pierre Riteau, Myunghwa Hwang, Anand Padmanabhan, Yizhao Gao, Yan Y. Liu, K. Keahey, Shaowen Wang
{"title":"A cloud computing approach to on-demand and scalable cybergis analytics","authors":"Pierre Riteau, Myunghwa Hwang, Anand Padmanabhan, Yizhao Gao, Yan Y. Liu, K. Keahey, Shaowen Wang","doi":"10.1145/2608029.2608032","DOIUrl":"https://doi.org/10.1145/2608029.2608032","url":null,"abstract":"Spatial data analysis has become ubiquitous as geographic information systems (GIS) are widely used to support scientific investigations and decision making in many fields of science, engineering, and humanities (e.g., ecology, emergency management, environmental engineering and sciences, geosciences, and social sciences). Tremendous data and computational capabilities are needed to handle and analyze massive quantities of spatial data that are collected across multiple spatiotemporal scales and used for diverse purposes. CyberGIS has emerged as a new-generation GIS based on advanced cyberinfrastructure to seamlessly integrate such capabilities into scalable geospatial analytics and modeling tools. One of the key challenges and opportunities of CyberGIS research is to build an on-demand service framework that can manage underlying cyberinfrastructure resources dynamically, in order to provide responsive support for interactive online CyberGIS analytics for which users can generate massive service requests in a short amount of time. This paper presents a cloud computing approach to implementing CyberGIS analytics using cloud computing services in the CyberGIS Gateway, a multiuser and collaborative online problem-solving environment. The primary purpose of this research is to address the question of how to achieve on-demand and scalable CyberGIS analytics that provide a stable response time to the user. We do that through integration with the Nimbus Phantom cloud platform. We then investigate how the cloud platform is able to adaptively handle fluctuating requests for analytics while providing a stable response time.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128181366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jiaan Zeng, Guangchen Ruan, Alexander Crowell, A. Prakash, Beth Plale
{"title":"Cloud computing data capsules for non-consumptiveuse of texts","authors":"Jiaan Zeng, Guangchen Ruan, Alexander Crowell, A. Prakash, Beth Plale","doi":"10.1145/2608029.2608031","DOIUrl":"https://doi.org/10.1145/2608029.2608031","url":null,"abstract":"As digital data sources grow in number and size, they pose an opportunity for computational investigation by means of text mining, natural language processing (NLP), and other text analysis techniques. In this paper we propose a virtual machine (VM) framework and methodology for non-consumptive text analysis. Using a remote VM model, the VM is configured with software and tooling for text analysis. When completed, the VM is wiped out and resources released for other users to share. Our approach extends the VM by turning it into a data capsules that prevents leakage of copyrighted content in the event that the VM is compromised. The HathiTrust Research Center Data Capsules has seen early use in application against the HathiTrust repository of digitized books from university libraries nationwide.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114510163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
K. Maheshwari, J. Wozniak, Hao Yang, D. Katz, M. Ripeanu, V. Zavala, M. Wilde
{"title":"Evaluating storage systems for scientific data in the cloud","authors":"K. Maheshwari, J. Wozniak, Hao Yang, D. Katz, M. Ripeanu, V. Zavala, M. Wilde","doi":"10.1145/2608029.2608034","DOIUrl":"https://doi.org/10.1145/2608029.2608034","url":null,"abstract":"Infrastructure-as-a-Service (IaaS) clouds are an appealing resource for scientific computing. However, the bare-bones presentation of raw Linux virtual machines leaves much to the application developer. For many cloud applications, effective data handling is critical to efficient application execution. This paper investigates the capabilities of a variety of POSIX-accessible distributed storage systems to manage data access patterns resulting from workflow application executions in the cloud. We leverage the expressivity of the Swift parallel scripting framework to benchmark the performance of a number of storage systems using synthetic workloads and three real-world applications. We characterize two representative commercial storage systems (Amazon S3 and HDFS, respectively) and two emerging research-based storage systems (Chirp/Parrot and MosaStore). We find the use of aggregated node-local resources effective and economical compared with remotely located S3 storage. Our experiments show that applications run at scale with MosaStore show up to 30% improvement in makespan time compared with those run with S3. We also find that storage-system driven application deployments in the cloud results in better runtime performance compared with an on-demand data-staging driven approach.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126771017","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Mux-Kmeans: multiplex kmeans for clustering large-scale data set","authors":"Chen Li, Yanfeng Zhang, Ming-hai Jiao, Ge Yu","doi":"10.1145/2608029.2608033","DOIUrl":"https://doi.org/10.1145/2608029.2608033","url":null,"abstract":"Kmeans clustering algorithm is widely used in a number of scientific applications due to its simple iterative nature and ease of implementation. The quality of clustering result highly depends on the selection of initial centroids. Different selections of initial centroids result in different clustering results. In practice, people run a series of Kmeans processes with multiple initial centroid groups serially and return the best clustering result among them. However, in the era of big data, a Kmeans process is implemented on MapReduce to scale to large data sets. Even a single Kmeans process on MapReduce requires considerable long runtime. This paper proposes Mux-Kmeans. Rather than executing multiple Kmeans processes serially, Mux-Kmeans launches these Kmeans processes concurrently with multiple centroid groups. In each iteration, Mux-Kmeans (i) evaluates these Kmeans processes, (ii) prunes the low-quality Kmeans processes, and (iii) incubates new Kmeans processes. After a certain number of iterations, it finally obtains the best among these local optimal results. We implement Mux-Kmeans on MapReduce and evaluate it on Amazon EC2. The experimental results show that starting from the same initial centroid groups, the clustering result of Mux-Kmeans is always non-worse than the best of a series of Kmeans processes. Mux-Kmeans also saves elapsed time than serial multiple Kmeans processes.","PeriodicalId":443577,"journal":{"name":"Scientific Cloud Computing","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121805538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
