Proceedings of the 8th Workshop on General Purpose Processing using GPUs最新文献

筛选

英文中文

Stochastic gradient descent on GPUs gpu的随机梯度下降

Proceedings of the 8th Workshop on General Purpose Processing using GPUs Pub Date : 2015-02-07 DOI: 10.1145/2716282.2716289

R. Kaleem, Sreepathi Pai, K. Pingali

{"title":"Stochastic gradient descent on GPUs","authors":"R. Kaleem, Sreepathi Pai, K. Pingali","doi":"10.1145/2716282.2716289","DOIUrl":"https://doi.org/10.1145/2716282.2716289","url":null,"abstract":"Irregular algorithms such as Stochastic Gradient Descent (SGD) can benefit from the massive parallelism available on GPUs. However, unlike in data-parallel algorithms, synchronization patterns in SGD are quite complex. Furthermore, scheduling for scale-free graphs is challenging. This work examines several synchronization strategies for SGD, ranging from simple locking to conflict-free scheduling. We observe that static schedules do not yield better performance despite eliminating the need to perform conflict detection and resolution at runtime. We identify the source of the performance degradation to be the structure of certain parts of the graph (dense vs sparse). This classification can be used to devise hybrid scheduling strategies which exploit different schedules for different regions of the graph to obtain better performance. We found that the best schedule for some problems can be up to two orders of magnitude faster than the worst one. To evaluate the performance of our GPU implementation, we also compare against a CPU implementation of SGD. Dynamic schedules perform comparably to a 14-thread CPU implementation, while a static schedule performs comparably to a 6-thread CPU implementation.","PeriodicalId":432610,"journal":{"name":"Proceedings of the 8th Workshop on General Purpose Processing using GPUs","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2015-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133598307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 33

Proceedings of the 8th Workshop on General Purpose Processing using GPUs 第八届gpu通用处理研讨会论文集

Proceedings of the 8th Workshop on General Purpose Processing using GPUs Pub Date : 1900-01-01 DOI: 10.1145/2716282

D. Kaeli, John Cavazos

引用次数: 1

首页上一页