Proceedings of the 47th International Conference on Parallel Processing最新文献_第7页

Unveiling Thread Communication Bottlenecks Using Hardware-Independent Metrics 使用与硬件无关的度量揭示线程通信瓶颈

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225142

Arya Mazaheri, F. Wolf, A. Jannesari

{"title":"Unveiling Thread Communication Bottlenecks Using Hardware-Independent Metrics","authors":"Arya Mazaheri, F. Wolf, A. Jannesari","doi":"10.1145/3225058.3225142","DOIUrl":"https://doi.org/10.1145/3225058.3225142","url":null,"abstract":"A critical factor for developing robust shared-memory applications is the efficient use of the cache and the communication between threads. Inappropriate data structures, algorithm design, and inefficient thread affinity may result in superfluous communication between threads/cores and severe performance problems. For this reason, state-of-the-art profiling tools focus on thread communication and behavior to present different metrics that enable programmers to write cache-friendly programs. The data shared between a pair of threads should be reused with a reasonable distance to preserve data locality. However, existing tools do not take into account the locality of communication events and mainly focus on analyzing the amount of communication instead. In this paper, we introduce a new method to analyze performance and communication bottlenecks that arise from data-access patterns and thread interactions of each code region. We propose new hardware-independent metrics to characterize thread communication and provide suggestions for applying appropriate optimizations on a specific code region. We evaluated our approach on the SPLASH and Rodinia benchmark suites. Experimental results validate the effectiveness of our approach by finding communication locality issues due to inefficient data structures and/or poor algorithm implementations. By applying the suggested optimizations, we improved the performance in Rodinia benchmarks by up to 56%. Furthermore, by varying the input size we demonstrated the ability of our method to assess the cache usage and scalability of a given application in terms of its inherent communication.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114070976","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

A Fast Sparse Triangular Solver for Structured-grid Problems on Sunway Many-core Processor SW26010 基于双威多核处理器SW26010的结构网格问题快速稀疏三角求解器

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225071

Xinliang Wang, Ping Xu, Wei Xue, Yulong Ao, Chao Yang, H. Fu, L. Gan, Guangwen Yang, Weimin Zheng

{"title":"A Fast Sparse Triangular Solver for Structured-grid Problems on Sunway Many-core Processor SW26010","authors":"Xinliang Wang, Ping Xu, Wei Xue, Yulong Ao, Chao Yang, H. Fu, L. Gan, Guangwen Yang, Weimin Zheng","doi":"10.1145/3225058.3225071","DOIUrl":"https://doi.org/10.1145/3225058.3225071","url":null,"abstract":"The sparse triangular solver (SpTRSV) is one of the most essential kernels in many scientific and engineering applications. Efficiently parallelizing the SpTRSV on modern many-core architectures is considerably difficult due to inherent dependency of computation and discontinuous memory accesses. Achieving high performance of SpTRSV is even more challenging for SW26010, the new-generation customized heterogeneous many-core processor equipped in the top-rank Sunway TaihuLight supercomputer. Owing to regular sparse pattern, structured-grid triangular problems show much different computing characteristics with general ones as well as new opportunities to algorithm design on many-core architectures, which ever lacks attention. In this work, we focus on how to design and implement fast SpTRSV for structured-grid problems on SW26010. A generalized algorithm framework of parallel SpTRSV is proposed for best utilization of the features and flexibilities of SW26010 many-core architecture according to the fine-grained Producer-Consumer model. Moreover, a novel parallel structured-grid SpTRSV is presented by using direct data transfers across registers of the computing elements of SW26010. Experiments on four typical structured-grid triangular problems with different problem sizes demonstrate that our SpTRSV can achieve an average momory bandwidth utilization of 79.7% according to the stream benchmark, which leads to a speedup of 17.7 over serial version on SW26010. Furthermore, experiments with real world sparse linear problems show that our proposed SpTRSV can achieve superior preconditioning performance over the Intel Xeon E5-2670 v3 CPU and Intel Xeon Phi 7210 KNL over DDR4 memory.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114117316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 7

Scalable Behavioral Emulation of Extreme-Scale Systems Using Structural Simulation Toolkit 使用结构仿真工具包的极端尺度系统的可扩展行为仿真

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225124

Ajay Ramaswamy, Nalini Kumar, Aravind Neelakantan, H. Lam, G. Stitt

{"title":"Scalable Behavioral Emulation of Extreme-Scale Systems Using Structural Simulation Toolkit","authors":"Ajay Ramaswamy, Nalini Kumar, Aravind Neelakantan, H. Lam, G. Stitt","doi":"10.1145/3225058.3225124","DOIUrl":"https://doi.org/10.1145/3225058.3225124","url":null,"abstract":"With extremely large design spaces for algorithm and architecture to be explored, there is a need for fast and scalable performance modeling tools for preparing HPC application codes. Behavioral Emulation (BE) is a recent coarse-grained modeling and simulation methodology that has been proposed to solve this co-design problem. In this paper, we introduce a distributed parallel simulation library for Behavioral Emulation called BE-SST, integrated into the Structural Simulation Toolkit (SST). BE-SST provides simple interfaces and framework for development of coarse-grained BE models which can be extended to model new notional architectures. BE-SST also supports Monte Carlo simulations to generate meaningful distributions and summary statistics rather than a single datum for performance. In this paper, we present BE-SST simulations of two existing large DOE machines (Vulcan and Titan), which have been validated against actual testbed measurements and showed 5-10% error. These validated system models (up to 128k cores) are used to make blind predictions of application performance on systems larger than the current machines (up to 512k cores) - a crucial simulator feature for design-space exploration of notional systems. We further studied BE-SST in terms of scalability and performance, simulating up to a million cores, with BE-SST running on more than 2k parallel processes. BE-SST shows good scalability with a linear increase in memory usage and simulation time with increase in simulated system size, and a peak speedup of 7x over single process simulation. With ease of use and good scaling, we assert that BE-SST can significantly speed up design-space exploration.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116593703","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 11

UHCL-Darknet: An OpenCL-based Deep Neural Network Framework for Heterogeneous Multi-/Many-core Clusters UHCL-Darknet:一个基于opencl的异构多核/多核集群深度神经网络框架

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225107

Longlong Liao, Kenli Li, Kuan-Ching Li, Canqun Yang, Qi Tian

{"title":"UHCL-Darknet: An OpenCL-based Deep Neural Network Framework for Heterogeneous Multi-/Many-core Clusters","authors":"Longlong Liao, Kenli Li, Kuan-Ching Li, Canqun Yang, Qi Tian","doi":"10.1145/3225058.3225107","DOIUrl":"https://doi.org/10.1145/3225058.3225107","url":null,"abstract":"As the majority of popular deep neural network (DNN) frameworks focus on a closed format CUDA implementations based on one or more NVIDIA GPUs, they cannot efficiently leverage other devices in cluster mode to accelerate the training and inference of DNNs except NVIDIA GPUs. To accelerate DNNs using heterogeneous multi-/many-core clusters, we propose an OpenCL-based DNN framework called UHCL-Darknet. First, we design a unified OpenCL platform model for the heterogeneous cluster called UHCL, and an adaptive runtime system with the affinity-based dynamic scheduler for UHCL, enabling transparent utilization of a wide variety of vendor-specific OpenCL devices in the heterogeneous cluster. Then, we extend Darknet to UHCL by introducing the parallel optimization of DNNs, such as paralleling Winogrand-based convolutions and auto-tuning parameterized OpenCL kernels. The training and inference of art-of-data DNN models (e.g., YOLOv2, ResNet-50, and DenseNet-201) are executed on an experimental heterogeneous cluster. Results show that UHCL-Darknet is a scalable and portable DNN framework for heterogeneous clusters, and achieves 1.9x and 2.2x speedups on average respectively for the image throughput of data-parallel training and inference on the experimental heterogeneous cluster.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124190818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 11

Reducing Communication in Proximal Newton Methods for Sparse Least Squares Problems 稀疏最小二乘问题的邻牛顿法减少通信

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225131

Saeed Soori, Aditya Devarakonda, Zachary Blanco, J. Demmel, M. Gürbüzbalaban, M. Dehnavi

引用次数: 3

Integrating Low-latency Analysis into HPC System Monitoring 将低延迟分析集成到HPC系统监控中

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225086

Ramin Izadpanah, Nichamon Naksinehaboon, J. Brandt, A. Gentile, D. Dechev

{"title":"Integrating Low-latency Analysis into HPC System Monitoring","authors":"Ramin Izadpanah, Nichamon Naksinehaboon, J. Brandt, A. Gentile, D. Dechev","doi":"10.1145/3225058.3225086","DOIUrl":"https://doi.org/10.1145/3225058.3225086","url":null,"abstract":"The growth of High Performance Computer (HPC) systems increases the complexity with respect to understanding resource utilization, system management, and performance issues. While raw performance data is increasingly exposed at the component level, the usefulness of the data is dependent on the ability to do meaningful analysis on actionable timescales. However, current system monitoring infrastructures largely focus on data collection, with analysis performed off-system in post-processing mode. This increases the time required to provide analysis and feedback to a variety of consumers. In this work, we enhance the architecture of a monitoring system used on large-scale computational platforms, to integrate streaming analysis capabilities at arbitrary locations within its data collection, transport, and aggregation facilities. We leverage the flexible communication topology of the monitoring system to enable placement of transformations based on overhead concerns, while still enabling low-latency exposure on node. Our design internally supports and exposes the raw and transformed data uniformly for both node level and off-system consumers. We show the viability of our implementation for a case with production-relevance: run-time determination of the relative per-node files system demands.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133779749","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 12

ran-GJS: Orchestrating Data Analytics for Heterogeneous Geo-distributed Edges rang - gjs:异构地理分布边缘的编排数据分析

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225059

Yibo Jin, Zhuzhong Qian, Song Guo, Sheng Zhang, Xiaoliang Wang, Sanglu Lu

{"title":"ran-GJS: Orchestrating Data Analytics for Heterogeneous Geo-distributed Edges","authors":"Yibo Jin, Zhuzhong Qian, Song Guo, Sheng Zhang, Xiaoliang Wang, Sanglu Lu","doi":"10.1145/3225058.3225059","DOIUrl":"https://doi.org/10.1145/3225058.3225059","url":null,"abstract":"Many organizations and companies have deployed not only datacenters but also large number of geo-distributed heterogeneous edges to provide fast data analytics services. Since large volume of data transmission across WAN can be costly, existing works mainly focus on pre-processing data in-place to avoid transmission. However, the heterogeneity of edges on either local computing capacity or network bandwidth limits the efficient use on scarce resource, which may result in long task completion time. To cope with dynamic demands on scarce resource, we take the heterogeneity of both computing capacity and network bandwidth of geo-distributed edges into consideration when assigning data analytical tasks and their associated data between the central datacenter and edges such that the overall latency can be reduced. We formulate the geo-distributed data-task joint scheduling problem (GJS), show its NP-hardness, and propose a near-optimal randomized scheduling algorithm (ran-GJS). ran-GJS can be proved concentrated around its optimum value with high probability, i.e., 1--O(e--t2) where t is the concentration bound by using Martingale Analysis. The experimental results obtained form both extensive simulations and Yarn-based prototype show that ran-GJS significantly speeds up the geo-distributed analytics with a gain on average completion time of at least 28% over state-of-the-art baseline algorithms.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"647 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133634764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

Dual-Paradigm Stream Processing 双范式流处理

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225120

Song Wu, Zhiyi Liu, Shadi Ibrahim, Lin Gu, Hai Jin, Fei Chen

{"title":"Dual-Paradigm Stream Processing","authors":"Song Wu, Zhiyi Liu, Shadi Ibrahim, Lin Gu, Hai Jin, Fei Chen","doi":"10.1145/3225058.3225120","DOIUrl":"https://doi.org/10.1145/3225058.3225120","url":null,"abstract":"Existing stream processing frameworks operate either under data stream paradigm processing data record by record to favor low latency, or under operation stream paradigm processing data in micro-batches to desire high throughput. For complex and mutable data processing requirements, this dilemma brings the selection and deployment of stream processing frameworks into an embarrassing situation. Moreover, current data stream or operation stream paradigms cannot handle data burst efficiently, which probably results in noticeable performance degradation. This paper introduces a dual-paradigm stream processing, called DO (Data and Operation) that can adapt to stream data volatility. It enables data to be processed in micro-batches (i.e., operation stream) when data burst occurs to achieve high throughput, while data is processed record by record (i.e., data stream) in the remaining time to sustain low latency. DO embraces a method to detect data bursts, identify the main operations affected by the data burst and switch paradigms accordingly. Our insight behind DO's design is that the trade-off between latency and throughput of stream processing frameworks can be dynamically achieved according to data communication among operations in a fine-grained manner (i.e., operation level) instead of framework level. We implement a prototype stream processing framework that adopts DO. Our experimental results show that our framework with DO can achieve 5x speedup over operation stream under low data stream sizes, and outperforms data stream on throughput by 2.1x to 3.2x under data burst.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130530296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Vectorized Parallel Sparse Matrix-Vector Multiplication in PETSc Using AVX-512 基于AVX-512的PETSc矢量化并行稀疏矩阵-矢量乘法

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225100

Hong Zhang, R. Mills, K. Rupp, Barry F. Smith

{"title":"Vectorized Parallel Sparse Matrix-Vector Multiplication in PETSc Using AVX-512","authors":"Hong Zhang, R. Mills, K. Rupp, Barry F. Smith","doi":"10.1145/3225058.3225100","DOIUrl":"https://doi.org/10.1145/3225058.3225100","url":null,"abstract":"Emerging many-core CPU architectures with high degrees of single-instruction, multiple data (SIMD) parallelism promise to enable increasingly ambitious simulations based on partial differential equations (PDEs) via extreme-scale computing. However, such architectures present several challenges to their efficient use. Here, we explore the efficient implementation of sparse matrix-vector (SpMV) multiplications---a critical kernel for the workhorse iterative linear solvers used in most PDE-based simulations---on recent CPU architectures from Intel as well as the second-generation Knights Landing Intel Xeon Phi, which features many CPU cores, wide SIMD lanes, and on-package high-bandwidth memory. Traditional SpMV algorithms use compressed sparse row storage format, which is a hindrance to exploiting wide SIMD lanes. We study alternative matrix formats and present an efficient optimized SpMV kernel, based on a sliced ELLPACK representation, which we have implemented in the PETSc library. In addition, we demonstrate the benefit of using this representation to accelerate preconditioned iterative solvers in realistic PDE-based simulations in parallel.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116263032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 17

Click-Based Asynchronous Mesh Network with Bounded Bundled Data 带有绑定数据的基于点击的异步网状网络

Proceedings of the 47th International Conference on Parallel Processing Pub Date : 2018-08-13 DOI: 10.1145/3225058.3225118

Anping He, G. Feng, Jilin Zhang, Pengfei Li, Yong Hei, Hong Chen

{"title":"Click-Based Asynchronous Mesh Network with Bounded Bundled Data","authors":"Anping He, G. Feng, Jilin Zhang, Pengfei Li, Yong Hei, Hong Chen","doi":"10.1145/3225058.3225118","DOIUrl":"https://doi.org/10.1145/3225058.3225118","url":null,"abstract":"We have implemented an asynchronous mesh network. This paper describes our innovative design using a Click controller. Compared to designs that use other asynchronous circuit families with C-elements and four-phase bundled data, our two-phase Click-based Bounded Bundled Data design is faster, but introduces phase skews when handling concurrent traffic at a single node. Instead of eliminating the phase skews, we use them as computation slots. Our network uses a novel asynchronous arbiter with a queue that can accept data from both the four cardinal directions as well as from a local source, five directions in all. We have implemented our network design in 1 × 1, 2 × 2 and 4 × 4 sizes, larger network could be implemented easier since the isomorphism and modularity of the routing nodes. Our experiments show that an initial data item passes through a node in 157ns v.s. 81ns for non-delay-branch and delay-branch designs separately. Following items take about 65% as long. But for a network, the average latency of a node keeps almost same for different paths. We believe that with the non-delay-branch designs, our asynchronous mesh network could offer 10.1M routes per second for a 1 × 1 network and 5.33M routes per second for 2 × 2 or 5.06M for 4 × 4 networks, and work at the rate of 17.3M, 10.1M and 11.7M with the enhanced delay-branch way. For both cases, its latency is approximately linear with scale.","PeriodicalId":307809,"journal":{"name":"Proceedings of the 47th International Conference on Parallel Processing","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125100391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2