{"title":"Adaptive Code Refinement: A Compiler Technique and Extensions to Generate Self-Tuning Applications","authors":"M. Schmitt, P. Helluy, C. Bastoul","doi":"10.1109/HiPC.2017.00028","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00028","url":null,"abstract":"Compiler high-level automatic optimization and parallelization techniques are well suited for some classes of simulation or signal processing applications, however they usually don't take into account domain-specific knowledge nor the possibility to change or to remove some computations to achieve \"good enough\" results. Differently, production simulation and signal processing codes have adaptive capabilities: they are designed to compute precise results only where it matters if the complete problem is not tractable or if computation time must be short. In this paper, we present a new way to provide adaptive capabilities to compute-intensive codes automatically. It relies on domain-specific knowledge provided through special pragmas by the programmer in the input code and on polyhedral compilation techniques to continuously regenerate at runtime a code that performs heavy computations only where it matters. We present experimental results on several applications where our strategy enables significant computation savings and speedup while maintaining a good precision, with a minimal effort from the programmer.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133598159","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Lifting Barriers Using Parallel Polyhedral Regions","authors":"Harenome Razanajato, C. Bastoul, Vincent Loechner","doi":"10.1109/HiPC.2017.00046","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00046","url":null,"abstract":"Nowadays best performing automatic parallelizers and data locality optimizers for static control programs rely on the polyhedral model. Polyhedral compilation consists of three phases: (1) abstracting the input code into a mathematical view; (2) analyzing and transforming this representation into an optimized alternative; (3) generating the corresponding code while ensuring it is semantically equivalent to the input code. During this last phase, state-of-the-art polyhedral compilers generate only one type of parallelism when targeting multicore shared memory architectures: parallel loops via the OpenMP omp parallel for directive. In this work, we propose to explore how a polyhedral compiler could exploit parallel region constructs. Instead of initializing a new set of threads each time the code enters a parallel loop and synchronizing them when exiting it, the threads are initialized once for all at the entrance of the region of interest, and synchronized only when it is necessary. Technically, we propose to embed the whole region containing parallel loops in an omp parallel construct. Inside the parallel region, the single construct is used when some code needs to be executed sequentially; the for construct is used to distribute loop iterations between threads. Thanks to the power of the polyhedral dependence analysis, we compute when it is valid to add the optional nowait clause, to omit the implicit barrier at the end of a worksharing construct and thus to reduce even more control overhead. Through a set of experiments on the PolyBench benchmarks, we show that resulting codes can overwhelm the performance obtained by the Pluto polyhedral compiler.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"153 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117015087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mulya Agung, Muhammad Alfian Amrizal, K. Komatsu, Ryusuke Egawa, H. Takizawa
{"title":"A Memory Congestion-Aware MPI Process Placement for Modern NUMA Systems","authors":"Mulya Agung, Muhammad Alfian Amrizal, K. Komatsu, Ryusuke Egawa, H. Takizawa","doi":"10.1109/HiPC.2017.00026","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00026","url":null,"abstract":"MPI process placement is an important step to achieve scalable performance on modern non-uniform memory access (NUMA) systems. A recent study on NUMA architectures has shown that, on modern NUMA systems, the memory congestion problem could cause more severe performance degradation than the data locality problem because heavy congestion on memory controllers could cause long latencies. However, conventional work on MPI process placement has focused on locality to minimize the remote-access communication. Moreover, maximizing the locality may actually degrade performance because the load imbalance among nodes in a modern NUMA system may increase. Thus, a process placement algorithm must be designed to consider memory congestion. In this paper, a method to reconcile both the locality and the memory congestion on modern NUMA systems is proposed. This method statically analyzes the application communication pattern to optimize the process placement. A data clustering method is applied to the time-series data of the MPI communications in order to identify data traffics that potentially cause memory congestion. The proposed method has been evaluated with the NPB kernels on a real NUMA system and a simulation environment. Experimental results show that the proposed method can achieve 1.6x performance improvement compared with the current state-of-the-art strategy.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126934720","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Parallel Exact Dynamic Bayesian Network Structure Learning with Application to Gene Networks","authors":"Md. Vasimuddin, S. Aluru","doi":"10.1109/HiPC.2017.00015","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00015","url":null,"abstract":"Learning the structure of Bayesian networks, even in the static case, is NP-hard, compelling much of the research to focus on heuristic-based approaches. However, there are instances where exact solutions are desirable especially for small network sizes. In this work, we present a dynamic programming based exact solution to learn dynamic Bayesian network structure. Our method simultaneously learns intra- as well as higher order inter-time-slice interactions in the network. For n variables, our exact solution requires O(n^2.2^n(M+1)) computations to learn M-th order network. To handle such high computational requirements, we present a parallel exact solution to push the limit on the size of the networks that can be learned. Given p=2^k processors, the parallel algorithm runs in O(n^2.2^nM.(2^n-k + k)) time and achieves optimal parallel efficiency when 2^n-k > k. Using MPI+X parallel programming model, the parallel algorithm linearly scales to 1,024 cores of a 64-node Intel Xeon InfiniBand cluster, sustaining >99% of parallel efficiency. We also show that the learned networks on gene network datasets are of high fidelity compared to heuristic-based techniques.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127931990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Potluri, Anshuman Goswami, D. Rossetti, C. Newburn, Manjunath Gorentla Venkata, N. Imam
{"title":"GPU-Centric Communication on NVIDIA GPU Clusters with InfiniBand: A Case Study with OpenSHMEM","authors":"S. Potluri, Anshuman Goswami, D. Rossetti, C. Newburn, Manjunath Gorentla Venkata, N. Imam","doi":"10.1109/HiPC.2017.00037","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00037","url":null,"abstract":"GPUs have become an essential component for building compute clusters with high compute density and high performance per watt. As such clusters scale to have 1000s of GPUs, efficiently moving data between the GPUs becomes imperative to get maximum performance. NVSHMEM is an implementation of the OpenSHMEM standard for NVIDIA GPU clusters which allows communication to be issued from inside GPU kernels. In earlier work, we have shown how NVSHMEM can be used to achieve better application performance on GPUs connected through PCIe or NVLink. As part of this effort, we implement IB verbs for Mellanox InfiniBand adapters in CUDA. We evaluate different design alternatives, taking into consideration the relaxed memory model, automatic memory access coalescing and thread hierarchy on the GPU. We also consider correctness issues that arise in these designs. We take advantage of these designs transparently or through API extensions in NVSHMEM. With micro-benchmarks, we show that a Nvidia Pascal P100 GPU is able saturate the network bandwidth using only one or two of its 56 available streaming multiprocessors (SM). On a single GPU using a single IB EDR adapter, we achieve a throughput of around 90 million messages per second. In addition, we implement a 2dstencil application kernel using NVSHMEM and compare its performance with a CUDA-aware MPI-based implementation that uses GPUDirect RDMA. Speedups in the range of 23% to 42% are seen for input sizes large enough to fill the occupancy of Nvidia Pascal P100 GPUs on 2 to 4 nodes indicating that there are gains to be had by eliminating the CPU from the communication path when all computation runs on the GPU.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127784745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ConvLight: A Convolutional Accelerator with Memristor Integrated Photonic Computing","authors":"Dharanidhar Dang, Jyotikrishna Dass, R. Mahapatra","doi":"10.1109/HiPC.2017.00022","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00022","url":null,"abstract":"Neuromorphic computing is a promising candidate to accelerate big data processing. Recently, several attempts have been made to design neuromorphic accelerators for popular machine learning algorithms, such as reservoir computing, deep learning, spiking neurons etc. Deep learning accelerator which involves convolutional neural networks (CNNs) have received widespread attention for their accuracy and efficiency. This paper proposes ConvLight, a novel deep learning accelerator based on memristor integrated photonic computing framework. While the use of on-chip photonic circuits for analog computing is well known, no prior work has demonstrated a full-fledged accelerator based on photonic components. In particular, this paper makes the following novel contributions: (i) A multilayer deep learning architecture design is proposed using compute efficient memristors and photonic components for the first time. (ii) A pipelined design for each CNN layer is presented for maximizing throughput and enabling parallelism across the layers. (iii) Simulation of ConvLight architecture with standard photonic tools for demonstrating the execution of DNN and CNN workloads yielding 25X, 60X, and 40X improvements in computational efficiency, throughput, and energy efficiency (respectively) compared to state-of-the-art design.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125864825","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Changsu Kim, Ju-Han Kim, Juwon Kang, Jae W. Lee, Hanjun Kim
{"title":"Context-Aware Memory Profiling for Speculative Parallelism","authors":"Changsu Kim, Ju-Han Kim, Juwon Kang, Jae W. Lee, Hanjun Kim","doi":"10.1109/HiPC.2017.00045","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00045","url":null,"abstract":"To expose hidden parallelism from programs with complex dependences, modern compilers employ memory profilers to augment imprecise static analyses. Since dynamic dependence patterns among instructions can vary widely depending on the context, such as function call site stack and loop nest level, context-aware memory profiling is of great value for precise memory profiling. However, recording memory dependences with full context information causes huge overheads in terms of CPU cycles and memory space. Existing profilers mitigate this problem by compromising precision, coverage, or both. This paper proposes a new precise Context-Aware Memory Profiling (CAMP) framework that efficiently traces all the memory dependences with full context information. CAMP statically analyzes a context tree of a program that illustrates all the possible dynamic contexts, and simplifies context management during profiling. For 14 programs from SPEC CINT2000 and CINT2006 benchmark suites, CAMP increases speculative parallelism opportunities by 12.6% on average and by up to 63.0% compared to the baseline context-oblivious, loop-aware memory profiler.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128391930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Redundant Arithmetic Based High Speed Carry Free Hybrid Adders with Built-In Scan Chain on FPGAs","authors":"Ayan Palchaudhuri, A. Dhar","doi":"10.1109/HiPC.2017.00021","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00021","url":null,"abstract":"Scan based error detection architectures for hybrid, carry-free radix-2 and radix-4 addition operations using redundant arithmetic are presented in this paper. Such addition operations have been chosen as representative examples as they are free from carry propagation delay and are ideal from the viewpoint of technology mapping of the logic elements onto the FPGA slices. The architectures have been conceived following the design paradigm of target FPGA specific primitive instantiation coupled with location constraints, without any degradation in the speed of circuit operation as compared to the original circuit implementation without the scan operation. Our architectures also comfortably outperform the existing state-of-the-art error detection architectures in terms of speed and consumes less area.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130989339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Esteban Rangel, N. Frontiere, S. Habib, K. Heitmann, W. Liao, Ankit Agrawal, A. Choudhary
{"title":"Building Halo Merger Trees from the Q Continuum Simulation","authors":"Esteban Rangel, N. Frontiere, S. Habib, K. Heitmann, W. Liao, Ankit Agrawal, A. Choudhary","doi":"10.1109/HiPC.2017.00052","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00052","url":null,"abstract":"Cosmological N-body simulations rank among the most computationally intensive efforts today. A key challenge is the analysis of structure, substructure, and the merger history for many billions of compact particle clusters, called halos. Effectively representing the merging history of halos is essential for many galaxy formation models used to generate synthetic sky catalogs, an important application of modern cosmological simulations. Generating realistic mock catalogs requires computing the halo formation history from simulations with large volumes and billions of halos over many time steps, taking hundreds of terabytes of analysis data. We present fast parallel algorithms for producing halo merger trees and tracking halo substructure from a single-level, density-based clustering algorithm. Merger trees are created from analyzing the halo-particle membership function in adjacent snapshots, and substructure is identified by tracking the \"cores\" of merging halos – sets of particles near the halo center. Core tracking is performed after creating merger trees and uses the relationships found during tree construction to associate substructures with hosts. The algorithms are implemented with MPI and evaluated on a Cray XK7 supercomputer using up to 16,384 processes on data from HACC, a modern cosmological simulation framework. We present results for creating merger trees from 101 analysis snapshots taken from the Q Continuum, a large volume, high mass resolution, cosmological simulation evolving half a trillion particles.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126607678","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sunwoo Lee, Dipendra Jha, Ankit Agrawal, A. Choudhary, W. Liao
{"title":"Parallel Deep Convolutional Neural Network Training by Exploiting the Overlapping of Computation and Communication","authors":"Sunwoo Lee, Dipendra Jha, Ankit Agrawal, A. Choudhary, W. Liao","doi":"10.1109/HiPC.2017.00030","DOIUrl":"https://doi.org/10.1109/HiPC.2017.00030","url":null,"abstract":"Training Convolutional Neural Network (CNN) is a computationally intensive task whose parallelization has become critical in order to complete the training in an acceptable time. However, there are two obstacles to developing a scalable parallel CNN in a distributed-memory computing environment. One is the high degree of data dependency exhibited in the model parameters across every two adjacent minibatches and the other is the large amount of data to be transferred across the communication channel. In this paper, we present a parallelization strategy that maximizes the overlap of inter-process communication with the computation. The overlapping is achieved by using a thread per compute node to initiate communication after the gradients are available. The output data of backpropagation stage is generated at each model layer, and the communication for the data can run concurrently with the computation of other layers. To study the effectiveness of the overlapping and its impact on the scalability, we evaluated various model architectures and hyperparameter settings. When training VGG-A model using ImageNet data sets, we achieve speedups of 62.97× and 77.97× on 128 compute nodes using mini-batch sizes of 256 and 512, respectively.","PeriodicalId":183270,"journal":{"name":"2017 IEEE 24th International Conference on High Performance Computing (HiPC)","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2017-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116622409","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}