Auto-scaling of virtual resources for scientific workflows on hybrid clouds

Abstract

Cloud computing technology enables applications to employ scalable resources dynamically. Scientists can promote large-scale scientific computational experiments over cloud environment. It is essential for many-task-computing (MTC) to certificate stable executions of applications even rapid changes of vital status of physical resources and furnish high performance resources in a long period. Auto-scaling with virtualization provides efficient and integrated cloud resource utilization. Auto-scaling issues have been actively studied as effective resource management in order to utilize large-scale data center in a good shape but most of the auto-scaling methods just easily support performance metrics such as CPU utilization and data transfer latency but seldom consider execution deadline or characteristics of an application. We propose an auto-scaling method that finishes all tasks by user specified deadline. We accomplish our goal by dynamically allocating VMs to maximize resource utilization while meeting a deadline and considering task dependency and data transfer time in workflow application. We have evaluated our auto-scaling method with protein annotation workflow application which tasks are specified as a workflow in hybrid cloud environment. The results of a simulation show the method performs automatically resource allocation actually needed satisfying deadline constraints.