Proceedings of the first Workshop on Emerging Technologies for software-defined and reconfigurable hardware-accelerated Cloud Datacenters最新文献

筛选

英文中文

An Experimental Comparison Between Genetic Algorithm and Particle Swarm Optimization in Spark Performance Tuning 遗传算法与粒子群算法在火花性能调优中的实验比较

Proceedings of the first Workshop on Emerging Technologies for software-defined and reconfigurable hardware-accelerated Cloud Datacenters Pub Date : 2017-04-08 DOI: 10.1145/3129457.3129494

Yuzhao Wang, Qixiao Liu, Junqing Yu, Zhibin Yu

{"title":"An Experimental Comparison Between Genetic Algorithm and Particle Swarm Optimization in Spark Performance Tuning","authors":"Yuzhao Wang, Qixiao Liu, Junqing Yu, Zhibin Yu","doi":"10.1145/3129457.3129494","DOIUrl":"https://doi.org/10.1145/3129457.3129494","url":null,"abstract":"The most popular in-memory computing framework --- Spark --- has a number of performance-critical configuration parameters. Manually tuning these parameters for optimized performance is not practical because the parameter tuning space is huge. Searching algorithms such as genetic algorithm can be used to automatically search the optimal configurations. However, there are several such algorithms and it is unclear which one is better in the case of Spark configuration parameter tuning. To address this issue, we experimentally compare two searching algorithms --- the Genetic Algorithm (GA) and the Particle Swarm Optimization (PSO) --- in searching the optimal configurations for Spark applications. We made several interesting observations. For one, PSO converges 2x faster than GA but the performance tuned by the configuration parameters found by PSO is slightly poorer than that by GA. Second, PSO shows better scalability with respect to the number of configuration parameters than GA. Finally, we find PSO is more robust than GA across different searching processes. Based on these observations, we recommend one to use PSO in Spark performance tuning context.","PeriodicalId":345943,"journal":{"name":"Proceedings of the first Workshop on Emerging Technologies for software-defined and reconfigurable hardware-accelerated Cloud Datacenters","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132249794","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Proceedings of the first Workshop on Emerging Technologies for software-defined and reconfigurable hardware-accelerated Cloud Datacenters 第一届软件定义和可重构硬件加速云数据中心新兴技术研讨会论文集

Proceedings of the first Workshop on Emerging Technologies for software-defined and reconfigurable hardware-accelerated Cloud Datacenters Pub Date : 1900-01-01 DOI: 10.1145/3129457

引用次数: 0

首页上一页