{"title":"Accelerating Neural Network Training using Arbitrary Precision Approximating Matrix Multiplication Algorithms","authors":"Grey Ballard, J. Weissenberger, Luoping Zhang","doi":"10.1145/3458744.3474050","DOIUrl":"https://doi.org/10.1145/3458744.3474050","url":null,"abstract":"Matrix multiplication is one of the bottleneck computations for training the weights within deep neural networks. To speed up the training phase, we propose to use faster algorithms for matrix multiplication known as Arbitrary Precision Approximating (APA) algorithms. APA algorithms perform asymptotically fewer arithmetic operations than the classical algorithm, but they compute an approximate result with an error that can be made arbitrarily small in exact arithmetic. Practical APA algorithms provide significant reduction in computation time and still provide enough accuracy for many applications like neural network training. We demonstrate that APA algorithms can be efficiently implemented and parallelized for multicore CPUs to obtain up to 28% and 21% speedups over the fastest implementation of the classical algorithm using one core and 12 cores, respectively. Furthermore, using these algorithms to train a Multi-Layer Perceptron (MLP) network yields no significant reduction in the training or testing error. Our performance results on a large MLP network show overall sequential and multithreaded performance improvements of up to 25% and 13%, respectively. We also demonstrate up to 15% improvement when training the fully connected layers of the VGG-19 image classification network.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115626624","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Automated Arrhythmia Detection using Hilbert-Huang Transform based Convolutional Neural Network","authors":"Tzu-Chia Lin, J. Zhang, Min Sun","doi":"10.1145/3458744.3473345","DOIUrl":"https://doi.org/10.1145/3458744.3473345","url":null,"abstract":"In this paper, a novel approach to arrhythmia-based signal classification is introduced. The objective is to properly identify three classes of patients exhibiting normal sinus rhythm, atrial fibrillation, and other rhythm. The proposed method apply Hilbert-Huang transform on raw signal to generate noise-free reconstruction of the original containing temporal variations as input for classification mechanism to learn representative features. The features are directly learned by Convolutional Neural Network, thus replacing traditional methods of relying on experts to handcraft features. To summarize, this paper contains two major processes: utilize a nonlinear and nonstationary signal processing technique to produce input, and to feed reconstructed signal containing representative features to CNN for multi-classification task. The experimental results indicate the effectiveness of this method, removing the need of human involvement in the process of feature selection. Through analyses and stimulations, the effectiveness of the proposed ECG-classification method is evaluated.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115548598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
T. Wada, Takuya Ikeda, Yutaka Kanayama, K. Ohtsuki
{"title":"New Evacuation Guidance Using Augmented Reality for Emergency Rescue Evacuation Support System (ERESS)","authors":"T. Wada, Takuya Ikeda, Yutaka Kanayama, K. Ohtsuki","doi":"10.1145/3458744.3473347","DOIUrl":"https://doi.org/10.1145/3458744.3473347","url":null,"abstract":"We are developing an Emergency Rescue Evacuation Support System (ERESS) for the purpose of evacuation support in the event of a local disaster. This system uses only terminals equipped with multiple sensors such as smartphones. It is a system that automatically detects disasters at an early stage and provides disaster evacuees with highly realtime disaster information. In this paper, we propose an evacuation guidance method using augmented reality (AR) in order to convey the evacuation route in the event of a disaster to the evacuees in an easy-to-understand manner. By using AR, many people can intuitively confirm the direction in which they should evacuate, leading to quick and accurate evacuation. It is versatile because it properly synthesizes the 3D model prepared in advance. The performance of the proposed method is evaluated by an indoor demonstration experiment, and its effectiveness is shown.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128667328","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jiashu Wang, Xun Deng, Kai-Ting Amy Wang, ZiChun Ye
{"title":"Adapting SYCL’s SIMT Programming Paradigm for Accelerators via Program Reconstruction","authors":"Jiashu Wang, Xun Deng, Kai-Ting Amy Wang, ZiChun Ye","doi":"10.1145/3458744.3473354","DOIUrl":"https://doi.org/10.1145/3458744.3473354","url":null,"abstract":"We present an IR-to-IR Converter that is capable of converting from LLVM IR to Halide IR and MLIR’s Affine Dialect to support generation of high performance SYCL kernel code [10] targeting accelerators with explicit memory hierarchy design. The converter performs program reconstruction to raise abstraction of the IR. Once the IR is raised to the level of Halide IR, AKG [2] can be leveraged to generate performant DaVinci code [2]. Alternatively, when the IR is raised to MLIR’s Affine Dialect, existing MLIR Affine passes with minor modifications can be readily used. Subsequently, the IR is lowered back to LLVM IR through progressive lowering. LLVM’s LLC is used to create the final binary for both cases. We extend upon SYCL’s buffer, accessor and parallel_for abstractions to facilitate the IR raising process.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122258687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Impact of AVX-512 Instructions on Graph Partitioning Problems.","authors":"M. Hossain, Erik Saule","doi":"10.1145/3458744.3473362","DOIUrl":"https://doi.org/10.1145/3458744.3473362","url":null,"abstract":"Graph analysis now percolates society with applications ranging from advertising and transportation to medical research. The structure of graphs is becoming more complex every day while they are getting larger. The increasing size of graph networks has made many of the classical algorithms reasonably slow. Fortunately, CPU architectures have evolved to adjust to new and more complex problems in terms of core-level parallelism and vector-level parallelism (SIMD-level). In this paper, we are exploring how the modern vector architecture of CPUs can help with community detection, partitioning, and coloring kernels by studying two representatives algorithms. We consider the Intel SkylakeX and Cascade Lake architectures, which support gather and scatter instructions on 512-bit vectors. The existing vectorized graph algorithms of classic graph problems, such as BFS and PageRank, do not apply well to community detection; we show the support of gather and scatter are necessary. In particular for the implementation of the reduce-scatter patterns. We evaluate the performances achieved on the two architectures and conclude that good hardware support for scatter instructions is necessary to fully leverage the vector processing for graph partitioning problems.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127606334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sunidhi Dhandhania, A. Deodhar, Konstantin Pogorelov, Swarnendu Biswas, J. Langguth
{"title":"Explaining the Performance of Supervised and Semi-Supervised Methods for Automated Sparse Matrix Format Selection","authors":"Sunidhi Dhandhania, A. Deodhar, Konstantin Pogorelov, Swarnendu Biswas, J. Langguth","doi":"10.1145/3458744.3474049","DOIUrl":"https://doi.org/10.1145/3458744.3474049","url":null,"abstract":"The performance of sparse matrix-vector multiplication kernels (SpMV) depends on the sparse matrix storage format and the architecture and the memory hierarchy of the target processor. Many sparse matrix storage formats along with corresponding SpMV algorithms have been proposed for improved SpMV performance. Given a sparse matrix and a target architecture, supervised Machine Learning techniques automate selecting the best formats. However, existing supervised approaches suffer from several drawbacks. They depend on large representative datasets and are expensive to train. In addition, retraining to incorporate new classes of matrices or different processor architectures is just as costly since new training data must be generated by benchmarking many instances. Furthermore, it is hard to understand the results of many supervised systems. We propose using semi-supervised machine learning techniques for format selection. We highlight the challenges in using the K-Means clustering for the sparse format selection problem and show how to adapt the algorithm to improve its performance. An empirical evaluation of our technique shows that the performance of our proposed semi-supervised learning approach is competitive with supervised methods, in addition to providing flexibility and explainability.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"69 6","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131922497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ArchViMP – a Framework for Automatic Extraction of Concurrency-related Software Architectural Properties","authors":"Monireh Pourjafarian, J. Jahic","doi":"10.1145/3458744.3473349","DOIUrl":"https://doi.org/10.1145/3458744.3473349","url":null,"abstract":"Concurrent multithreaded programs are more complex than sequential ones due to inter-dependencies of threads over shared memory. Because of these, software architects and developers quickly become overwhelmed when trying to design and manage concurrent software. Existing approaches that try to support architecture efforts in this domain rely on the visualization of concurrency-related properties of software to ease its understanding, but they fail because i) the abstractions they use do not capture information of architectural significance, and because ii) raw visualization of the interdependencies does not scale. In this paper, we suggest a scalable solution that focuses on the abstraction of concurrency properties and their visualization using architectural views. Our framework for automatic extraction of concurrency-related architectural properties (ArchViMP) proposes i) a set of logical rules that abstract concurrency-related architecturally significant software properties and ii) a set of architectural views suitable for showing these concurrency properties.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133341299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Maldonado, Michel Schanen, F. Pacaud, M. Anitescu
{"title":"Domain Decomposition Preconditioners for Unstructured Network Problems in Parallel Vector Architectures","authors":"D. Maldonado, Michel Schanen, F. Pacaud, M. Anitescu","doi":"10.1145/3458744.3473363","DOIUrl":"https://doi.org/10.1145/3458744.3473363","url":null,"abstract":"In this paper we present our experience implementing domain decomposition preconditioners on vector architectures. In particular, we will focus on the solution of unstructured network equations arising from electrical power systems by preconditioning iterative algorithms with the Additive Schwarz Method (ASM). The implementation will be carried out using the Julia programming language, which allows for easy prototyping and interfacing with GPU architectures thanks to its multiple dispatch features. In our experiments, we will show the trade-off between device throughput and convergence of the iterative algorithm as the size of the domain varies, and determine optimal fronts of computational performance.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"62 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127998127","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Shared Memory Remote Procedure Calls","authors":"John L. Chesterfield","doi":"10.1145/3458744.3473357","DOIUrl":"https://doi.org/10.1145/3458744.3473357","url":null,"abstract":"The remote procedure call (RPC) is a simple interface for executing code on a different machine. Almost none of the well known problems inherent to RPC apply on a shared memory system. Further, a shared memory system is sufficient to implement a RPC library, so that said simple interface can be more widely available. This paper includes a minimal implementation of the proposed algorithm, with a real world implementation tested on x86-64, AMDGPU and NVPTX architectures. This can bring host capabilities to the GPU or offload code without using kernel launch APIs. The client and server both compile and run on each architecture.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125658462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Dual-KV: Improving Performance of Key-value Caches on Multilevel Cell Non-volatile Memory","authors":"Zong-Ming Ke, Yun-Ze Li, Da-Wei Chang","doi":"10.1145/3458744.3473350","DOIUrl":"https://doi.org/10.1145/3458744.3473350","url":null,"abstract":"∗In-memory key-value caches are widely used by web applications to reduce latencies of database queries. In recent years, multilevel cell (MLC) technology has been used in various emerging nonvolatile memories (NVM), allowing the building of large-scale main memory in a computing system. Building key-value caches on NVM, instead of DRAM, not only prevents cache data loss due to power outage but also improves the cache hit rate. However, it could lead to degraded performance due to the inferior write performance of NVM, compared to that of DRAM. To address this problem, previous studies focus on reducing the amount of NVM writes. In this paper, we propose Dual-KV, a method to improve performance of MLC NVM key-value caches. Different from previous studies, Dual-KV achieves the performance improvement by exploiting the write-latency/retention-time trade-off of MLC NVM. In MLC NVM, write latency increases with the retention time. The basic idea of Dual-KV is to adopt short-latency-and-short-retention writes (i.e., fast writes) for frequently updated data items while using longlatency-and-long-retention writes (i.e., slow writes) for the other data items. The experiment results show that Dual-KV improves throughput by up to 43% and 83% for single-thread and 16-thread YCSB workloads, respectively. Moreover, Dual-KV is designed to be integrated into existing key-value caches easily. Integrating DualKV into Memcached, a widely-used key-value cache, requires less than 30 lines of code modifications.","PeriodicalId":204366,"journal":{"name":"50th International Conference on Parallel Processing Workshop","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126061840","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}