2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)最新文献

筛选
英文 中文
[Title page] (标题页)
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/dls49591.2019.00001
{"title":"[Title page]","authors":"","doi":"10.1109/dls49591.2019.00001","DOIUrl":"https://doi.org/10.1109/dls49591.2019.00001","url":null,"abstract":"","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126456285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Scaling TensorFlow, PyTorch, and MXNet using MVAPICH2 for High-Performance Deep Learning on Frontera 在Frontera上使用MVAPICH2缩放TensorFlow, PyTorch和MXNet进行高性能深度学习
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/DLS49591.2019.00015
Arpan Jain, A. Awan, H. Subramoni, D. Panda
{"title":"Scaling TensorFlow, PyTorch, and MXNet using MVAPICH2 for High-Performance Deep Learning on Frontera","authors":"Arpan Jain, A. Awan, H. Subramoni, D. Panda","doi":"10.1109/DLS49591.2019.00015","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00015","url":null,"abstract":"Frontera is the largest NSF-funded cluster in the US and comprises of 8,008 nodes equipped with the latest Intel Xeon processors (Cascade-Lake). In this paper, we explore the potential of Frontera for training state-of-the-art Deep Learning (DL) models at scale. Most DL studies present performance data from large-scale GPU clusters that are equipped with NVIDIA GPUs. However, our earlier performance characterization studies have helped us achieve comparable performance with CPU-only clusters as well. Based on this, we configure three important DL frameworks; 1) TensorFlow, 2) PyTorch, and 3) MXNet, using Horovod and two Message Passing Interface (MPI) libraries on Frontera: 1) MVAPICH2 and 2) Intel MPI. We provide a systematic performance comparison for TensorFlow using MVAPICH2 and Intel MPI on 2,048 Frontera nodes. Using a four process per-node configuration, we observe near-linear scaling for ResNet-50 training for TensorFlow up to 8,192 MPI processes (on 2,048 nodes) offering a sustained performance of 250,000 images/second. In addition, we provide insights into process per node and batch size configurations for TensorFlow as well as for PyTorch and MXNet. Based on single-node performance behavior, we scale all three DL frameworks up to 1,024 processes (256 nodes) for various models like ResNet-50/101/152 and Inception-v3/v4.","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115951825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 15
Deep Facial Recognition using Tensorflow 使用Tensorflow的深度面部识别
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/DLS49591.2019.00011
C. Mattmann, Zhao Zhang
{"title":"Deep Facial Recognition using Tensorflow","authors":"C. Mattmann, Zhao Zhang","doi":"10.1109/DLS49591.2019.00011","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00011","url":null,"abstract":"Facial recognition is a tractable problem today because of the prevalence of Deep Learning implementations. Approaches for creating structured datasets from unstructured web data are more easily accessible as are GPUs that deep learning frameworks can use to learn from this data. In DARPA’s MEMEX effort, which sought to create better search capabilities for law enforcement to scan the deep and dark web, we are interested in leveraging the Tensorflow framework to reproduce a seminal Deep Learning facial recognition model called VGG- Face. On MEMEX we desired to build the VGG-Face model and to train feature extraction for use in prioritization of leads for possible law enforcement follow-up. We describe our efforts to recreate the VGG-Face dataset, along with our efforts to create the Deep Learning network implementation for it using Tensorflow. Though other implementations of VGG-Face on Tensorflow exist, none of them fully reproduce as much of the dataset as we do today (∼ 48% of the data still exists), nor have detailed documentation and steps for reproducing each step in the workflow. We contribute those instructions and leverage Texas Advanced Computing Center’s Maverick2 supercomputer to perform the work. We report experimental results on building the dataset, and training the network to achieve a 77.99% validation accuracy on the 2, 622 celebrity use case from VGG-Face. This paper can be a useful recipe in building new Tensorflow facial recognition.","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123765358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Deep Learning for Gap Crossing Ability of Ground Vehicles 地面车辆通过间隙能力的深度学习
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/DLS49591.2019.00009
Benjamin E. Parsons, Jing-Ru C. Cheng
{"title":"Deep Learning for Gap Crossing Ability of Ground Vehicles","authors":"Benjamin E. Parsons, Jing-Ru C. Cheng","doi":"10.1109/DLS49591.2019.00009","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00009","url":null,"abstract":"In this work we present our results designing a deep neural network (DNN) to act as a surrogate model for costly HPC simulations. In order to determine a ground vehicle's gap crossing ability in extreme weather scenarios, several HPC simulations are currently used. Hydrologic models are first run to determine the environmental conditions over an area of interest. Once these conditions are known they are given, along with the terrain data, to a vehicle simulation which determines if a particular vehicle can cross a stream at a given point. Every point of interest must be evaluated independently, which quickly becomes infeasible for a large numbers of crossing points. In order to accelerate this phase of the process, we have created a DNN that acts as a surrogate model for the vehicle simulator. Despite several challenges converting irregular data into a form that can be used with a DNN, and incorporating scalars into the models, we were able to produce DNN models that predicted the gap crossing ability of all vehicle types with over 95% accuracy.","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133338037","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
[Copyright notice] (版权)
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/dls49591.2019.00002
{"title":"[Copyright notice]","authors":"","doi":"10.1109/dls49591.2019.00002","DOIUrl":"https://doi.org/10.1109/dls49591.2019.00002","url":null,"abstract":"","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129844502","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Strategies to Deploy and Scale Deep Learning on the Summit Supercomputer 在Summit超级计算机上部署和扩展深度学习的策略
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/DLS49591.2019.00016
Junqi Yin, Shubhankar Gahlot, N. Laanait, K. Maheshwari, Jack Morrison, Sajal Dash, M. Shankar
{"title":"Strategies to Deploy and Scale Deep Learning on the Summit Supercomputer","authors":"Junqi Yin, Shubhankar Gahlot, N. Laanait, K. Maheshwari, Jack Morrison, Sajal Dash, M. Shankar","doi":"10.1109/DLS49591.2019.00016","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00016","url":null,"abstract":"The rapid growth and wide applicability of Deep Learning (DL) frameworks poses challenges to computing centers which need to deploy and support the software, and also to domain scientists who have to keep up with the system environment and scale up scientific exploration through DL. We offer recommendations for deploying and scaling DL frameworks on the Summit supercomputer, currently atop the Top500 list, at the Oak Ridge National Laboratory Leadership Computing Facility (OLCF). We discuss DL software deployment in the form of containers, and compare performance of native-built frameworks and containerized deployment. Software containers show no noticeable negative performance impact and exhibit faster Python loading times and promise easier maintenance. To explore strategies for scaling up DL model training campaigns, we assess DL compute kernel performance, discuss and recommend I/O data formats and staging, and identify communication needs for scalable message exchange for DL runs at scale. We recommend that users take a step-wise tuning approach beginning with algorithmic kernel choice, node I/O configuration, and communications tuning as best-practice. We present baseline examples of scaling efficiency 87% for a DL run of ResNet50 running on 1024 nodes (6144 V100 GPUs).","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"503 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121930971","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Evolving Larger Convolutional Layer Kernel Sizes for a Settlement Detection Deep-Learner on Summit 基于峰顶的沉降检测深度学习算法的更大卷积层核演化
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/DLS49591.2019.00010
M. Coletti, D. Lunga, Jeffrey K. Bassett, A. Rose
{"title":"Evolving Larger Convolutional Layer Kernel Sizes for a Settlement Detection Deep-Learner on Summit","authors":"M. Coletti, D. Lunga, Jeffrey K. Bassett, A. Rose","doi":"10.1109/DLS49591.2019.00010","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00010","url":null,"abstract":"Deep-learner hyper-parameters, such as kernel sizes, batch sizes, and learning rates, can significantly influence the quality of trained models. The state of the art for finding optimal hyper-parameters generally uses a brute force, grid search approach, random search, or Bayesian-based optimization among other techniques. We applied an evolutionary algorithm to optimize kernel sizes for a convolutional neural network used to detect settlements in satellite imagery. Usually convolutional layer kernel sizes are small – typically one, three, or five – but we found that the system converged at, or near, kernel sizes of nine for the last convolutional layer, and that this occurred for multiple runs using two different datasets. Moreover, the larger kernel sizes had fewer false positives than the 3x3 kernel sizes found as optimal via a brute force uniform grid search. This suggests that this large kernel size may be leveraging patterns found in larger areal features in the source imagery, and that this may be generalized as possible guidance for similar remote sensing deep-learning tasks.","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"50 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127264769","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
DC-S3GD: Delay-Compensated Stale-Synchronous SGD for Large-Scale Decentralized Neural Network Training DC-S3GD:用于大规模分散神经网络训练的延迟补偿非同步SGD
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/DLS49591.2019.00013
Alessandro Rigazzi
{"title":"DC-S3GD: Delay-Compensated Stale-Synchronous SGD for Large-Scale Decentralized Neural Network Training","authors":"Alessandro Rigazzi","doi":"10.1109/DLS49591.2019.00013","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00013","url":null,"abstract":"Data parallelism has become the de facto standard for training Deep Neural Network on mul- tiple processing units. In this work we propose DC- S3GD, a decentralized (without Parameter Server) stale-synchronous version of the Delay-Compensated Asynchronous Stochastic Gradient Descent (DC- ASGD) algorithm. In our approach, we allow for the overlap of computation and communication, by averaging in parameter space and compensating the inherent error with a first-order correction of the locally computed gradients. We prove the effectiveness of our approach by training Convolutional Neural Network with large batches and achieving state-of- the-art results.","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126972166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Aggregating Local Storage for Scalable Deep Learning I/O 聚合本地存储用于可扩展深度学习I/O
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-11-01 DOI: 10.1109/DLS49591.2019.00014
Zhao Zhang, Lei Huang, J. G. Pauloski, Ian T Foster
{"title":"Aggregating Local Storage for Scalable Deep Learning I/O","authors":"Zhao Zhang, Lei Huang, J. G. Pauloski, Ian T Foster","doi":"10.1109/DLS49591.2019.00014","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00014","url":null,"abstract":"Deep learning applications introduce heavy I/O loads on computer systems. The inherently long-running, highly concurrent, and random file accesses can easily saturate traditional shared file systems and negatively impact other users. We investigate here a solution to these problems based on leveraging local storage and the interconnect to serve training datasets at scale. We present FanStore, a user-level transient object store that provides low-latency and scalable POSIX-compliant file access by integrating the function interception technique and various metadata/data placement strategies. On a single node, FanStore provides performance similar to that of the XFS journaling file system. On many nodes, our experiments with real applications show that FanStore achieves over 90% scaling efficiency.","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114331513","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Highly-Ccalable, Physics-Informed GANs for Learning Solutions of Stochastic PDEs 用于随机偏微分方程解学习的高可扩展性、物理信息的gan
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS) Pub Date : 2019-10-29 DOI: 10.1109/DLS49591.2019.00006
Liu Yang, Sean Treichler, T. Kurth, Keno Fischer, D. Barajas-Solano, J. Romero, Valentin Churavy, A. Tartakovsky, Michael Houston, Prabhat, G. Karniadakis
{"title":"Highly-Ccalable, Physics-Informed GANs for Learning Solutions of Stochastic PDEs","authors":"Liu Yang, Sean Treichler, T. Kurth, Keno Fischer, D. Barajas-Solano, J. Romero, Valentin Churavy, A. Tartakovsky, Michael Houston, Prabhat, G. Karniadakis","doi":"10.1109/DLS49591.2019.00006","DOIUrl":"https://doi.org/10.1109/DLS49591.2019.00006","url":null,"abstract":"Uncertainty quantification for forward and inverse problems is a central challenge across physical and biomedical disciplines. We address this challenge for the problem of modeling subsurface flow at the Hanford Site by combining stochastic computational models with observational data using physics-informed GAN models. The geographic extent, spatial heterogeneity, and multiple correlation length scales of the Hanford Site require training a computationally intensive GAN model to thousands of dimensions. We develop a highly optimized implementation that scales to 27,500 NVIDIA Volta GPUs. We develop a hierarchical scheme based on a multi-player game-theoretic approach for exploiting domain parallelism, map discriminators and generators to multiple GPUs, and employ efficient communication schemes to ensure training stability and convergence. Our implementation scales to 4584 nodes on the Summit supercomputer with a 93.1%scaling efficiency, achieving peak and sustained half-precision . rates of 1228 PF/s and 1207 PF/s.","PeriodicalId":363604,"journal":{"name":"2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121167100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 32
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书