TECN: Task Selection and Placement in GPU Enabled Clouds Using Neural Networks

Abstract

Resource scheduling in cloud computing needs to be addressed effectively and efficiently to enable fair share, high throughput and low latency for large numbers of jobs that share the cloud. Currently, there are no known solutions for vGPU enabled VM placement. Virtualized GPUs present many opportunities and challenges; finding an optimal placement for VMs is an NP-hard problem. Our research focusses on using Machine Learning to address task/VM placement in a vGPU enabled cloud.We built a simulator to test and compare different strategies to select and place VMs. In this paper, we describe the simulator and discuss the results of the comparison of different heuristics. We present details of the dense neural networks (DNN) we built that out-perform all the heuristics. The DNNs learns the "best" heuristic at every system configuration and as such are "superior" to any individual heuristic. Our approach to using machine learning to solve the problem of selection and placement starts with heuristics, trains DNNs from these heuristics, and then out-performs them. We did a head-to-head comparison of the task selection by the DNNs with that generated by the heuristics. In this comparison, the DNNs show better task selection results for 76% of the test cases than the heuristics. These results obtained by using DNNs look promising and can be further improved by refining the neural networks.