{"title":"Improving scalability of multi-agent reinforcement learning with parameters sharing","authors":"Ning Yang, Bo Ding, Peichang Shi, Dawei Feng","doi":"10.1109/JCC56315.2022.00013","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00013","url":null,"abstract":"Improving the scalability of a multi-agent system is one of the key challenges for applying reinforcement learning to learn an effective policy. Parameter sharing is a common approach used to improve the efficiency of learning by reducing the volume of policy network parameters that need to be updated. However, sharing parameters also reduces the variance between agents’ policies, which further restricts the diversity of their behaviors. In this paper, we introduce a policy parameter sharing approach, it maintains a policy network for each agent, and only updates one of them. The differentiated behavior of agents is maintained by the policy, while sharing parameters are updated through a soft way. Experiments in foraging scenarios demonstrate that our method can effectively improve the performance and also the scalability of the multi-agent systems.","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122856774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiang Fang, Yi Zou, Yange Fang, Zhen Tang, Hui Li, Wei Wang
{"title":"A Query-Level Distributed Database Tuning System with Machine Learning","authors":"Xiang Fang, Yi Zou, Yange Fang, Zhen Tang, Hui Li, Wei Wang","doi":"10.1109/JCC56315.2022.00012","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00012","url":null,"abstract":"Knob tuning is important to improve the performance of database management system. However, the traditional manual tuning method by DBA is time-consuming and error-prone, and can not meet the requirements of different database instances. In recent years, the research on automatic knob tuning using machine learning algorithm has gradually sprung up, but most of them only support workload-level knob tuning, and the studies on query-level tuning is still in the initial stage. Furthermore, few works are focus on the knob tuning for distributed database. In this paper, we propose a query-level tuning system for distribute database with the machine learning method. This system can efficiently recommend knobs according to the feature of the query. We deployed our techniques onto CockroachDB, a distribute database, and experimental results show that our system achieves higher performance under typical OLAP workload. For all categories of queries, our system reduces the latency by 9.2% on average, and for some categories of queries, this system reduces the latency by more than 60%.","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129977899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Threshold Based Load Balancing Algorithm in Cloud Computing","authors":"Shusmoy Chowdhury, Ajay Katangur","doi":"10.1109/JCC56315.2022.00011","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00011","url":null,"abstract":"Cloud computing has become an emerging trend for the software industry with the requirement of large infrastructure and resources. The future success of cloud computing depends on the effectiveness of instantiation of the infrastructure and utilization of available resources. Load Balancing ensures the fulfillment of these conditions to improve the cloud environment for the users. Load Balancing dynamically distributes the workload among the nodes in such a way that no single resource is either overwhelmed with tasks or underutilized. In this paper we propose a threshold based load balancing algorithm to ensure the equal distribution of the workload among the nodes. The main objective of the algorithms is to stop the VMs in the cloud being overloaded with tasks or being idle for lack allocation of tasks, when there are active tasks. We have simulated our proposed algorithm in the Cloudanalyst simulator with real world data scenarios. Simulation results shows that our proposed threshold based algorithm can provide a better response time for the task/requests and data processing time for the datacenters compared to the existing algorithms such as First Come First Serve (FCFS), Round Robin(RR) and Equally Spread Current Execution Load Balancing algorithm(ESCELB).","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"359 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115899658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Uncertainty Estimation based Intrinsic Reward For Efficient Reinforcement Learning","authors":"Chao Chen, Tianjiao Wan, Peichang Shi, Bo Ding, Zijian Gao, Dawei Feng","doi":"10.1109/JCC56315.2022.00008","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00008","url":null,"abstract":"For reinforcement learning, the extrinsic reward is a core factor for the learning process which however can be very sparse or completely missing. In response, researchers have proposed the idea of intrinsic reward, such as encouraging the agent to visit novel states through prediction error. However, the deep prediction model can provide over-confident and miscalibrated predictions. To mitigate the impact of inaccurate prediction, previous research applied deep ensembles and achieved superior results, despite the increased computation and storage space. In this paper, inspired by the uncertainty estimation, we leverage Monte Carlo Dropout to generate intrinsic reward from the perspective of uncertainty estimation with the goal to decrease the demands for computing resources while retaining superior performance. Utilizing the simple yet effective approach, we conduct extensive experiments across a variety of benchmark environments. The experimental results suggest that our method provides a competitive performance in final score and is faster in running speed, while requiring much fewer computing resources and storage space.","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125779884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ming Chen, Peichang Shi, Xiang Fu, Feng Jiang, Fei Gao, Penghui Ma, Jinzhu Kong
{"title":"FSS: A Flexible Scaling Scheme for Blockchain Based on Stale Block Rate","authors":"Ming Chen, Peichang Shi, Xiang Fu, Feng Jiang, Fei Gao, Penghui Ma, Jinzhu Kong","doi":"10.1109/JCC56315.2022.00015","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00015","url":null,"abstract":"In blockchain, there has long been a contradiction between the limited ability and the uncertain requirements of processing transactions, which seriously restricts the practical application of blockchain. Therefore, how to improve the scalability of blockchain has become an urgent issue to be solved. Some existing works have achieved blockchain expansion through increasing the upper limit of block size permanently, which makes the trade-off of the “Mundellian Trilemma ” in blockchain (i.e. a blockchain system cannot be optimal in all the three dimensions of scalability, security and decentralization at the same time) fixed and thus not adapted to the dynamic environment. In this paper, we propose FSS, a flexible scaling scheme for blockchain based on stale block rate, which dynamically adjusts the upper limit of block size according to the stale block rate, not only expanding the blockchain when allowed, but also shrinking it when necessary. Experimental results indicate that FSS can reasonably improve the scalability of blockchain with required stale block rate.","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129848508","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fei Gao, Huaimin Wang, Peichang Shi, Xiang Fu, Tao Zhong, Jinzhu Kong
{"title":"MRASS: Dynamic Task Scheduling enabled High Multi-cluster Resource Availability in JointCloud","authors":"Fei Gao, Huaimin Wang, Peichang Shi, Xiang Fu, Tao Zhong, Jinzhu Kong","doi":"10.1109/JCC56315.2022.00014","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00014","url":null,"abstract":"As the new paradigm of JointCloud Computing matures, enterprises are trying to build multiple Kubernetes clusters on different clouds to deploy tasks, with the advantages of disaster backup, low latency, and avoidance of single vendor lock-in, etc. Tasks in a JointCloud environment, always have highly diversified resource demands on CPU, memory, disk, and network. However, the mismatch between these tasks and heterogeneous clusters can easily cause many resource fragments, resulting in low resource availability. Therefore, the task scheduling strategy is the key to solving the above problem. The existing task schedule strategies for multi-clusters are always aiming at clusters’ load balancing instead of increasing the resource availability. In this paper, we propose a dynamic task scheduling framework with the design of multi-cluster resource high-availability schedule strategy (MRASS) based on historical task resource consumption. MRASS conducts a cooperation model between multiple clusters and tasks, and proposes an indicator of resource availability, which is used to optimize the proportion of remaining resources of the cluster to keep approaching the proportion of resource requirements of future tasks, thereby execute more tasks within limited resources. Extensive numerical results confirm that the strategy has stable performance and performs well with different initial cluster resource setting, task resource type and task number. Compared with the existing algorithm, MRASS can place up to 20% more tasks, and the success rate of first placement of tasks can reach over 98%.","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124911385","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Youmei Song, Chao Li, Kuoran Zhuang, Tengyu Ma, Tianyu Wo
{"title":"An Automatic Scaling System for Online Application with Microservices Architecture","authors":"Youmei Song, Chao Li, Kuoran Zhuang, Tengyu Ma, Tianyu Wo","doi":"10.1109/JCC56315.2022.00018","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00018","url":null,"abstract":"Auto-scaling is an efficient technique to handle fluctuations of application workloads by acquiring or releasing resources. However, performing auto-scaling in a microservice system for online applications faces critical challenges, including unpredictably massive microservice requests, without fine-granularity performance metrics, and complex dependencies among services. In this paper, we design a cost-efficient autoscaling system, which pinpoints the scaling-needed services as quickly as possible and makes decisions on the right resource amount allocation toward them. Specifically, we first propose a multi-level microservice monitoring mechanism to capture historical and latest service-level performance metrics, and detect the over-provisioning services and under-provisioning services via jointly considering the changes of latency and throughput. For the overload anomalies, a random walk method is further adopted for detecting the root causes based on the dependency topology of microservices. When anomalies are detected, we design a threshold-based method by incorporating the ARIMI method for predicting resource usage status to allocate or recycle the right number of computation resources for them. Extensive and systematic evaluations of different algorithm modules with real-world and simulated workload data confirm the superiority of our mechanism over multiple algorithms.","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133143373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Welcome Message from the TPC Chairs of IEEE JCC 2022","authors":"","doi":"10.1109/jcc56315.2022.00006","DOIUrl":"https://doi.org/10.1109/jcc56315.2022.00006","url":null,"abstract":"","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129933783","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tiejun Wang, Xudong Mou, Juntao Hu, Rui Wang, Tianyu Wo
{"title":"Two-stage Scheduling of Stream Computing for Industrial Cloud-edge Collaboration","authors":"Tiejun Wang, Xudong Mou, Juntao Hu, Rui Wang, Tianyu Wo","doi":"10.1109/JCC56315.2022.00016","DOIUrl":"https://doi.org/10.1109/JCC56315.2022.00016","url":null,"abstract":"As the Industrial Internet of Things (IIoT) develops, intelligent services applying stream computing, such as industrial robot health management, are requiring higher timeliness of data processing, which may involve scheduling of stream tasks. However, traditional scheduling methods are no longer suitable for the currently widely used cloud-edge collaboration mode, not considering the cloud-edge heterogeneity, and focusing on the scheduling of single tasks instead of the optimization of the total tasks. To improve the performance of the cloud-edge collaboration, this paper establishes a practical model for task scheduling considering respectively cloud-edge environment collaboration models. We propose a novel two-stage scheduling method for IIoT. The algorithm utilizes the idea of maximum flow to divide the task into cloud-edge deployment schemes and find the best partitioning scheme, and then deploy the operator for the edge domain based on the network topology by using dynamic programming. Experimental results show that the proposed method could reduce 7.27% the cloud-edge bandwidth usage compared with the highest greedy algorithm for traffic difference, 24.33% end-to-end latency and 11.18% back-pressure rate compared with SBON.","PeriodicalId":239996,"journal":{"name":"2022 IEEE International Conference on Joint Cloud Computing (JCC)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114612629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}