Gaith Rjoub, J. Bentahar, O. A. Wahab, A. Bataineh
{"title":"深度智能调度:基于云的自动化大数据调度的深度学习方法","authors":"Gaith Rjoub, J. Bentahar, O. A. Wahab, A. Bataineh","doi":"10.1109/FiCloud.2019.00034","DOIUrl":null,"url":null,"abstract":"With the widespread adoption of Internet of Thing (IoT) and the exponential growth in the volumes of generated data, cloud providers tend to receive massive waves of demands on their storage and computing resources. To help providers deal with such demands without sacrificing performance, the concept of cloud automation had recently arisen to improve the performance and reduce the manual efforts related to the management of cloud computing workloads. In this context, we propose in this paper, Deep learning Smart Scheduling (DSS), an automated big data task scheduling approach in cloud computing environments. DSS combines Deep Reinforcement Learning (DRL) and Long Short-Term Memory (LSTM) to automatically predict the Virtual Machines (VMs) to which each incoming big data task should be scheduled to so as to improve the performance of big data analytics and reduce their resource execution cost. Experiments conducted using real-world datasets from Google Cloud Platform show that our solution minimizes the CPU usage cost by 28.8% compared to the Shortest Job First (SJF), and by 14% compared to both the Round Robin (RR) and improved Particle Swarm Optimization (PSO) approaches. Moreover, our solution decreases the RAM memory usage cost by 31.25% compared to the SJF, by 25% compared to the RR, and by 18.78% compared to the improved PSO.","PeriodicalId":268882,"journal":{"name":"2019 7th International Conference on Future Internet of Things and Cloud (FiCloud)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2019-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"26","resultStr":"{\"title\":\"Deep Smart Scheduling: A Deep Learning Approach for Automated Big Data Scheduling Over the Cloud\",\"authors\":\"Gaith Rjoub, J. Bentahar, O. A. Wahab, A. Bataineh\",\"doi\":\"10.1109/FiCloud.2019.00034\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"With the widespread adoption of Internet of Thing (IoT) and the exponential growth in the volumes of generated data, cloud providers tend to receive massive waves of demands on their storage and computing resources. To help providers deal with such demands without sacrificing performance, the concept of cloud automation had recently arisen to improve the performance and reduce the manual efforts related to the management of cloud computing workloads. In this context, we propose in this paper, Deep learning Smart Scheduling (DSS), an automated big data task scheduling approach in cloud computing environments. DSS combines Deep Reinforcement Learning (DRL) and Long Short-Term Memory (LSTM) to automatically predict the Virtual Machines (VMs) to which each incoming big data task should be scheduled to so as to improve the performance of big data analytics and reduce their resource execution cost. Experiments conducted using real-world datasets from Google Cloud Platform show that our solution minimizes the CPU usage cost by 28.8% compared to the Shortest Job First (SJF), and by 14% compared to both the Round Robin (RR) and improved Particle Swarm Optimization (PSO) approaches. Moreover, our solution decreases the RAM memory usage cost by 31.25% compared to the SJF, by 25% compared to the RR, and by 18.78% compared to the improved PSO.\",\"PeriodicalId\":268882,\"journal\":{\"name\":\"2019 7th International Conference on Future Internet of Things and Cloud (FiCloud)\",\"volume\":\"57 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2019-08-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"26\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2019 7th International Conference on Future Internet of Things and Cloud (FiCloud)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/FiCloud.2019.00034\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2019 7th International Conference on Future Internet of Things and Cloud (FiCloud)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/FiCloud.2019.00034","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Deep Smart Scheduling: A Deep Learning Approach for Automated Big Data Scheduling Over the Cloud
With the widespread adoption of Internet of Thing (IoT) and the exponential growth in the volumes of generated data, cloud providers tend to receive massive waves of demands on their storage and computing resources. To help providers deal with such demands without sacrificing performance, the concept of cloud automation had recently arisen to improve the performance and reduce the manual efforts related to the management of cloud computing workloads. In this context, we propose in this paper, Deep learning Smart Scheduling (DSS), an automated big data task scheduling approach in cloud computing environments. DSS combines Deep Reinforcement Learning (DRL) and Long Short-Term Memory (LSTM) to automatically predict the Virtual Machines (VMs) to which each incoming big data task should be scheduled to so as to improve the performance of big data analytics and reduce their resource execution cost. Experiments conducted using real-world datasets from Google Cloud Platform show that our solution minimizes the CPU usage cost by 28.8% compared to the Shortest Job First (SJF), and by 14% compared to both the Round Robin (RR) and improved Particle Swarm Optimization (PSO) approaches. Moreover, our solution decreases the RAM memory usage cost by 31.25% compared to the SJF, by 25% compared to the RR, and by 18.78% compared to the improved PSO.