2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)最新文献_第3页

Designing Effective Sparse Expert Models 设计有效的稀疏专家模型

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00171

Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, J. Dean, Noam M. Shazeer, W. Fedus

引用次数: 58

Optimal Triangulation on the High Bandwidth Memory Model 高带宽内存模型的最优三角剖分

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00089

K. Nakano, V. Poupet

{"title":"Optimal Triangulation on the High Bandwidth Memory Model","authors":"K. Nakano, V. Poupet","doi":"10.1109/IPDPSW55747.2022.00089","DOIUrl":"https://doi.org/10.1109/IPDPSW55747.2022.00089","url":null,"abstract":"The High Bandwidth Memory (HBM) model is a theoretical computing model consisting of a logic circuit with a large external memory. Each address of the external memory can store $p$ elements which can be read or written at the same time. Access to $p$ elements stored at a given address in the external memory has a latency of $l$ clock cycles. However, access to any $k$ consecutive addresses can be done only in $(k+l-1)$ clock cycles in a pipeline fashion by burst mode. A hardware algorithm is implemented in a logic circuit of the HBM to solve a particular problem. In this paper, we present an optimal implementation of the $O(n^{3})$ -time dynamic programming algorithm for solving the optimal polygon triangulation (OPT) problem which is a problem to find a triangulation with minimum total weight of an input convex n-gon with weighted cords. We assume that the input weight matrix of a convex n-gon is stored in the external memory of the HBM model. Our hardware algorithm implemented in the logic circuit of size $O(s^{2})$ operates on it and computes the optimal polygon triangulation of the input polygon in $O(frac{n^{3}}{sp}+frac{n^{3}}{s^{2}}+frac{n^{3}}{s^{3}}l)$ time. We also provide a theoretical proof showing that any hardware algorithm in a logic circuit of size $O(s^{2})$ takes at least $Omega(frac{n^{3}}{sp}+frac{n^{3}}{s^{2}})$ time to solve the OPT problem. Thus, our implementation is optimal whenever $s^{2}geq lp$ or $sgeq l$, and this optimality condition is always satisfied from a practical point of view.","PeriodicalId":286968,"journal":{"name":"2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)","volume":"20 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115015687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An Architecture- Independent CGRA Compiler enabling OpenMP Applications 一个支持OpenMP应用程序的体系结构独立的CGRA编译器

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00112

Takuya Kojima, B. Adhi, Carlos Cortes, Y. Tan, K. Sano

{"title":"An Architecture- Independent CGRA Compiler enabling OpenMP Applications","authors":"Takuya Kojima, B. Adhi, Carlos Cortes, Y. Tan, K. Sano","doi":"10.1109/IPDPSW55747.2022.00112","DOIUrl":"https://doi.org/10.1109/IPDPSW55747.2022.00112","url":null,"abstract":"Coarse-Grained reconfigurable architecture (CGRA) is a promising platform for HPC systems in the post-Moore's era. A single-source programming model is essential for practical heterogeneous computing. However, we do not have a canonical programming model and a frontend compiler for it. Existing versatile CGRAs, in respect to their execution model, computational capability, and system structure, magnify the difficulty of orchestrating the compiler techniques. It consequently forces designers of the CGRAs to develop the compiler from scratch, working only for their architectures. Such an approach is outdated, given other successful accelerators like GPU and FPGAs. This paper presents a new CGRA compiler framework in order to reduce development efforts of CG RA applications. OpenMP annotated codes are fed into the proposed compiler, as recent OpenMP support device offloading to the accelerators. This property improves the reusability of the existing source code for HPC workloads. The design of the compiler is inspired by LLVM, which is the most famous compiler framework so that the frontend is built to be architecture-independent. In this work, we demonstrate that the proposed compiler can handle different types of CG RAs without changing the source codes. In addition, we discuss the effect of architecture-independent optimization algorithms. We also provide an open-source implementation of the compiler framework at https://github.com/hal-lab-u-tokyo/CGRAOmp.","PeriodicalId":286968,"journal":{"name":"2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)","volume":"257 8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115456396","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Keynote Talk 1: Efficient DNN Training at Scale: from Algorithms to Hardware 主题演讲1:大规模高效深度神经网络训练:从算法到硬件

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00219

Gennady Pekhimenko

引用次数: 0

Optimizing Non-commutative Allreduce Over Virtualized, Migratable MPI Ranks 优化非交换的Allreduce虚拟化，可迁移的MPI排名

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00085

Sam White, L. Kalé

{"title":"Optimizing Non-commutative Allreduce Over Virtualized, Migratable MPI Ranks","authors":"Sam White, L. Kalé","doi":"10.1109/IPDPSW55747.2022.00085","DOIUrl":"https://doi.org/10.1109/IPDPSW55747.2022.00085","url":null,"abstract":"Dynamic load balancing can be difficult for MPI-based applications. Application logic and algorithms are often rewritten to enable dynamic repartitioning of the domain. An alternative approach is to virtualize the MPI ranks as threads-instead of operating system processes- and to migrate threads around the system to balance the computational load. Adaptive MPI is one such implementation. It supports virtualization of MPI ranks as migratable user-level threads. However, this migratability itself can introduce new performance overheads to applications. In this paper, we identify non-commutative reduction operations as problematic for any runtime supporting either user-defined initial mapping of ranks or dynamic migration of ranks among the cores or nodes of a machine. We investigate the challenges associated with supporting efficient non-commutative reduction operations, and explore algorithmic alternatives such as recursive doubling and halving in combination with a novel adaptive message combining technique. We explore tradeoffs in the different algorithms for various message sizes and mappings of ranks to cores, demonstrating our performance improvements using microbenchmarks.","PeriodicalId":286968,"journal":{"name":"2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)","volume":"283 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122958668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Teaching High-Performance Computing in Developing Countries: A Case Study in Mexican Universities 发展中国家的高性能计算教学:以墨西哥大学为例

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00066

J. Trejo-Sánchez, F. Hernández-López, Miguel Ángel Uh Zapata, J. López-Martínez, Daniel Fajardo-Delgado, J. Pacheco

{"title":"Teaching High-Performance Computing in Developing Countries: A Case Study in Mexican Universities","authors":"J. Trejo-Sánchez, F. Hernández-López, Miguel Ángel Uh Zapata, J. López-Martínez, Daniel Fajardo-Delgado, J. Pacheco","doi":"10.1109/IPDPSW55747.2022.00066","DOIUrl":"https://doi.org/10.1109/IPDPSW55747.2022.00066","url":null,"abstract":"Teaching High-Performance Computing (HPC) to undergraduate programs represents a significant challenge in most universities in developing countries like Mexico. Deficien-cies in the required infrastructure and equipment, inadequate curricula in computer engineering programs (and resistance to change them), students' lack of interest, motivation, or knowledge of this area are the main difficulties to overcome. The COVID-19 pandemic represents an additional challenge to these difficulties in teaching HPC in these programs. Despite the detriments, some strategies have been developed to incorporate the HPC concepts to Mexican students without necessarily modifying the traditional curricula. This paper presents a case study over four public universities in Mexico based on our experience as instructors. We also propose a course that introduces the HPC principles considering the heterogeneous background of the students in such universities. The results are about the number of students enrolling in related classes and participating in extra-curricular projects.","PeriodicalId":286968,"journal":{"name":"2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122106345","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Optimal Schedules for High-Level Programming Environments on FPGAs with Constraint Programming 基于约束规划的fpga高级编程环境的最优调度

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00025

Pascal Jungblut, D. Kranzlmüller

引用次数: 0

Combined Application of Approximate Computing Techniques in DNN Hardware Accelerators 近似计算技术在DNN硬件加速器中的联合应用

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00013

Enrico Russo, M. Palesi, Davide Patti, Habiba Lahdhiri, Salvatore Monteleone, G. Ascia, V. Catania

引用次数: 1

Benchmarking Quantum Processor Performance through Quantum Distance Metrics Over An Algorithm Suite 通过算法套件上的量子距离度量对量子处理器性能进行基准测试

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00106

S. Stein, N. Wiebe, James Ang, A. Li

引用次数: 0

ESSA 2022 Invited Speaker: The Curious Incident of the Data in the Scientific Workflow ESSA 2022特邀演讲者:科学工作流程中数据的奇怪事件

2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) Pub Date : 2022-05-01 DOI: 10.1109/IPDPSW55747.2022.00181

L. Ramakrishnan

引用次数: 0