Task Assignment in a Virtualized GPU Enabled Cloud

Abstract

Cloud computing vendors are beginning to offer GPU based high performance computing as a service. One approach uses virtual machines (VM), running in a hypervisor like VMware vSphere, equipped with virtual GPUs like Nvidia's vGPU solution. In this approach, multiple VMs running concurrently can share a single GPU. The number of VMs that share the GPU can be configured by the user/system administrator. Further, VMs can be re-assigned to GPUs, if more than one is available, dynamically. This approach allows tasks/jobs that use GPUs to run in individual VMs guaranteeing isolation whilst sharing resources. In a typical cloud environment with multiple servers each with one or more GPUs, finding an efficient, fast solution to the problem of placing VMs (i.e. VM-placement) on GPUs and moving them around as needed is extremely important to achieve high throughput of tasks while maximizing server utilization and minimizing task wait times. In this paper, we present the simulator we built to compare different solutions to the problem of VM-placement together with some early results.