Proceedings of the 10th Workshop on Scientific Cloud Computing最新文献

Session details: Session 2: Scientific Computing Based on Cloud 会议详情:第二部分:基于云的科学计算

Proceedings of the 10th Workshop on Scientific Cloud Computing Pub Date : 2019-06-17 DOI: 10.1145/3341817

Dmitry Duplyakin

引用次数: 0

Horizontal or Vertical?: A Hybrid Approach to Large-Scale Distributed Machine Learning 水平还是垂直?大规模分布式机器学习的混合方法

Proceedings of the 10th Workshop on Scientific Cloud Computing Pub Date : 2019-06-17 DOI: 10.1145/3322795.3331461

Jinkun Geng, Dan Li, Shuai Wang

引用次数: 15

Session details: Session 1: Converged Computing Infrastructures 会议详情:会议1:融合计算基础设施

Proceedings of the 10th Workshop on Scientific Cloud Computing Pub Date : 2019-06-17 DOI: 10.1145/3341816

Bogdan Nicolae

引用次数: 0

Deconstructing the 2017 Changes to AWS Spot Market Pricing 解析2017年AWS现货市场定价变化

Proceedings of the 10th Workshop on Scientific Cloud Computing Pub Date : 2019-06-17 DOI: 10.1145/3322795.3331465

Matt Baughman, Simon Caton, C. Haas, Ryan Chard, R. Wolski, Ian T Foster, K. Chard

引用次数: 14

ElasticPipe: An Efficient and Dynamic Model-Parallel Solution to DNN Training 弹性管道:DNN训练的高效动态模型并行解决方案

Proceedings of the 10th Workshop on Scientific Cloud Computing Pub Date : 2019-06-17 DOI: 10.1145/3322795.3331463

Jinkun Geng, Dan Li, Shuai Wang

{"title":"ElasticPipe: An Efficient and Dynamic Model-Parallel Solution to DNN Training","authors":"Jinkun Geng, Dan Li, Shuai Wang","doi":"10.1145/3322795.3331463","DOIUrl":"https://doi.org/10.1145/3322795.3331463","url":null,"abstract":"Traditional deep neural network (DNN) training is executed with data parallelism, which suffers from significant communication overheads and GPU memory consumption. Considering this, recent pioneering works have attempted to train DNN with model parallelism. However, model partition remains as a major concern and a static partition fails to adapt to the ever-changing computation environment of the cloud cluster. This paper proposes ElasticPipe, which trains the neural network based on pipe-based model parallelism. Unlike data-parallel solutions, each node in ElasticPipe only holds part of the whole model, leading to much lower cost of communication and GPU memory. More importantly, ElasticPipe is able to dynamically tune the workload distribution among different nodes, so that it can mitigate the common straggler effect in cloud environment. Our primary experiment shows, compared to the data-parallel baselines, ElasticPipe can reduce the training time by up to 89.03% without considering straggler effect, and by up to 76.72% with the existence of stragglers. Besides, ElasticPipe also outperforms its static counterpart by up to 28.81% in training performance when stragglers are involved.","PeriodicalId":164694,"journal":{"name":"Proceedings of the 10th Workshop on Scientific Cloud Computing","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116069846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 29

Towards a Smart, Internet-Scale Cache Service for Data Intensive Scientific Applications 面向数据密集型科学应用的智能互联网级缓存服务

Proceedings of the 10th Workshop on Scientific Cloud Computing Pub Date : 2019-06-17 DOI: 10.1145/3322795.3331464

Yubo Qin, Anthony Simonet, Philip E. Davis, Azita Nouri, Zhe Wang, M. Parashar, I. Rodero

引用次数: 5

Proceedings of the 10th Workshop on Scientific Cloud Computing 第十届科学云计算研讨会论文集

Proceedings of the 10th Workshop on Scientific Cloud Computing Pub Date : 1900-01-01 DOI: 10.1145/3322795

引用次数: 0