Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments最新文献

筛选
英文 中文
Optimizing deep learning hyper-parameters through an evolutionary algorithm 通过进化算法优化深度学习超参数
Steven R. Young, Derek C. Rose, T. Karnowski, Seung-Hwan Lim, R. Patton
{"title":"Optimizing deep learning hyper-parameters through an evolutionary algorithm","authors":"Steven R. Young, Derek C. Rose, T. Karnowski, Seung-Hwan Lim, R. Patton","doi":"10.1145/2834892.2834896","DOIUrl":"https://doi.org/10.1145/2834892.2834896","url":null,"abstract":"There has been a recent surge of success in utilizing Deep Learning (DL) in imaging and speech applications for its relatively automatic feature generation and, in particular for convolutional neural networks (CNNs), high accuracy classification abilities. While these models learn their parameters through data-driven methods, model selection (as architecture construction) through hyper-parameter choices remains a tedious and highly intuition driven task. To address this, Multi-node Evolutionary Neural Networks for Deep Learning (MENNDL) is proposed as a method for automating network selection on computational clusters through hyper-parameter optimization performed via genetic algorithms.","PeriodicalId":307997,"journal":{"name":"Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments","volume":" 11","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132074608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 339
LBANN: livermore big artificial neural network HPC toolkit LBANN: livermore大型人工神经网络HPC工具包
B. V. Essen, Hyojin Kim, R. Pearce, K. Boakye, Barry Y. Chen
{"title":"LBANN: livermore big artificial neural network HPC toolkit","authors":"B. V. Essen, Hyojin Kim, R. Pearce, K. Boakye, Barry Y. Chen","doi":"10.1145/2834892.2834897","DOIUrl":"https://doi.org/10.1145/2834892.2834897","url":null,"abstract":"Recent successes of deep learning have been largely driven by the ability to train large models on vast amounts of data. We believe that High Performance Computing (HPC) will play an increasingly important role in helping deep learning achieve the next level of innovation fueled by neural network models that are orders of magnitude larger and trained on commensurately more training data. We are targeting the unique capabilities of both current and upcoming HPC systems to train massive neural networks and are developing the Livermore Big Artificial Neural Network (LBANN) toolkit to exploit both model and data parallelism optimized for large scale HPC resources. This paper presents our preliminary results in scaling the size of model that can be trained with the LBANN toolkit.","PeriodicalId":307997,"journal":{"name":"Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133092417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 68
HPDBSCAN: highly parallel DBSCAN HPDBSCAN:高度并行的DBSCAN
Markus Goetz, Christian Bodenstein, M. Riedel
{"title":"HPDBSCAN: highly parallel DBSCAN","authors":"Markus Goetz, Christian Bodenstein, M. Riedel","doi":"10.1145/2834892.2834894","DOIUrl":"https://doi.org/10.1145/2834892.2834894","url":null,"abstract":"Clustering algorithms in the field of data-mining are used to aggregate similar objects into common groups. One of the best-known of these algorithms is called DBSCAN. Its distinct design enables the search for an apriori unknown number of arbitrarily shaped clusters, and at the same time allows to filter out noise. Due to its sequential formulation, the parallelization of DBSCAN renders a challenge. In this paper we present a new parallel approach which we call HPDBSCAN. It employs three major techniques in order to break the sequentiality, empower workload-balancing as well as speed up neighborhood searches in distributed parallel processing environments i) a computation split heuristic for domain decomposition, ii) a data index preprocessing step and iii) a rule-based cluster merging scheme. As a proof-of-concept we implemented HPDBSCAN as an OpenMP/MPI hybrid application. Using real-world data sets, such as a point cloud from the old town of Bremen, Germany, we demonstrate that our implementation is able to achieve a significant speed-up and scale-up in common HPC setups. Moreover, we compare our approach with previous attempts to parallelize DBSCAN showing an order of magnitude improvement in terms of computation time and memory consumption.","PeriodicalId":307997,"journal":{"name":"Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments","volume":"62 6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116252715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 53
Dynamic adaptive neural network arrays: a neuromorphic architecture 动态自适应神经网络阵列:一种神经形态结构
Catherine D. Schuman, A. Disney, J. Reynolds
{"title":"Dynamic adaptive neural network arrays: a neuromorphic architecture","authors":"Catherine D. Schuman, A. Disney, J. Reynolds","doi":"10.1145/2834892.2834895","DOIUrl":"https://doi.org/10.1145/2834892.2834895","url":null,"abstract":"Dynamic Adaptive Neural Network Array (DANNA) is a neuromorphic hardware implementation. It differs from most other neuromorphic projects in that it allows for programmability of structure, and it is trained or designed using evolutionary optimization. This paper describes the DANNA structure, how DANNA is trained using evolutionary optimization, and an application of DANNA to a very simple classification task.","PeriodicalId":307997,"journal":{"name":"Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114818702","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Asynchronous parallel stochastic gradient descent: a numeric core for scalable distributed machine learning algorithms 异步并行随机梯度下降:可扩展分布式机器学习算法的数值核心
J. Keuper, F. Pfreundt
{"title":"Asynchronous parallel stochastic gradient descent: a numeric core for scalable distributed machine learning algorithms","authors":"J. Keuper, F. Pfreundt","doi":"10.1145/2834892.2834893","DOIUrl":"https://doi.org/10.1145/2834892.2834893","url":null,"abstract":"The implementation of a vast majority of machine learning (ML) algorithms boils down to solving a numerical optimization problem. In this context, Stochastic Gradient Descent (SGD) methods have long proven to provide good results, both in terms of convergence and accuracy. Recently, several parallelization approaches have been proposed in order to scale SGD to solve very large ML problems. At their core, most of these approaches are following a MapReduce scheme. This paper presents a novel parallel updating algorithm for SGD, which utilizes the asynchronous single-sided communication paradigm. Compared to existing methods, Asynchronous Parallel Stochastic Gradient Descent (ASGD) provides faster convergence, at linear scalability and stable accuracy.","PeriodicalId":307997,"journal":{"name":"Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125552293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 38
Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments 高性能计算环境中的机器学习研讨会论文集
{"title":"Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments","authors":"","doi":"10.1145/2834892","DOIUrl":"https://doi.org/10.1145/2834892","url":null,"abstract":"","PeriodicalId":307997,"journal":{"name":"Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments","volume":"44 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127035874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 20
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书