GPGPU-based heterogeneous parallel implementation of direct discontinuous Galerkin methods

IF 5.4 3区材料科学 Q2 CHEMISTRY, PHYSICAL

ACS Applied Energy Materials Pub Date : 2024-10-11 DOI:10.1016/j.matcom.2024.09.034

Jiaxin Wang , Kun Wang , Zhen-Guo Yan , Xiaofeng He , Tiegang Liu

{"title":"GPGPU-based heterogeneous parallel implementation of direct discontinuous Galerkin methods","authors":"Jiaxin Wang , Kun Wang , Zhen-Guo Yan , Xiaofeng He , Tiegang Liu","doi":"10.1016/j.matcom.2024.09.034","DOIUrl":null,"url":null,"abstract":"<div><div>This paper implements the CUDA and hybrid CUDA/MPI parallel computation based on GPGPU heterogeneous parallel strategies for the direct discontinuous method (DDG) on 3D unstructured grids. The direct discontinuous Galerkin method inherits the compactness of the discontinuous Galerkin (DG) method, making it well-suited for large-scale parallelization. Firstly, we present the full single-GPU implementation of the three-dimensional (3D) DDG method with cell-level parallelism and face-level parallelism. Herein, all the numerical operators including volume integration, face integration (numerical fluxes), conservation variables calculation, and time iteration, are implemented by designing the corresponding kernel functions. Especially, we implement several key memory access optimization strategies, which are crucial for performance improvement. Operators merging and shared memory utilizing reduces the number of global access. Such memory Coalescing and data structure reconstruction apparently enhances the efficiency of global memory access. To align with data access pattern, we employ atomic operations to eliminate data race conditions. Furthermore, we propose a full hybrid GPU/CPU heterogeneous parallel strategy to implement multi-GPU parallelization of the DDG method, where asynchronization optimization is introduced to fully overlap communication and computation and basically eliminates the communication overhead. Finally, several numerical tests are conducted on Tesla V100 Cards to show performance of the parallelization. In addition, we utilize the NVIDIA performance testing tool, <span><math><mrow><mi>n</mi><mi>v</mi><mi>p</mi><mi>r</mi><mi>o</mi><mi>f</mi></mrow></math></span>, to evaluate multiple metrics of the kernel functions and conduct a detailed analysis of the results. In the tests of parallel scalability, the weak scaling efficiency achieves 97% from 4 to 32 GPU cards, and the strong scaling efficiency is 90% from 1 to 8 GPU cards.</div></div>","PeriodicalId":4,"journal":{"name":"ACS Applied Energy Materials","volume":null,"pages":null},"PeriodicalIF":5.4000,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ACS Applied Energy Materials","FirstCategoryId":"100","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0378475424003896","RegionNum":3,"RegionCategory":"材料科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CHEMISTRY, PHYSICAL","Score":null,"Total":0}

引用次数: 0

Abstract

This paper implements the CUDA and hybrid CUDA/MPI parallel computation based on GPGPU heterogeneous parallel strategies for the direct discontinuous method (DDG) on 3D unstructured grids. The direct discontinuous Galerkin method inherits the compactness of the discontinuous Galerkin (DG) method, making it well-suited for large-scale parallelization. Firstly, we present the full single-GPU implementation of the three-dimensional (3D) DDG method with cell-level parallelism and face-level parallelism. Herein, all the numerical operators including volume integration, face integration (numerical fluxes), conservation variables calculation, and time iteration, are implemented by designing the corresponding kernel functions. Especially, we implement several key memory access optimization strategies, which are crucial for performance improvement. Operators merging and shared memory utilizing reduces the number of global access. Such memory Coalescing and data structure reconstruction apparently enhances the efficiency of global memory access. To align with data access pattern, we employ atomic operations to eliminate data race conditions. Furthermore, we propose a full hybrid GPU/CPU heterogeneous parallel strategy to implement multi-GPU parallelization of the DDG method, where asynchronization optimization is introduced to fully overlap communication and computation and basically eliminates the communication overhead. Finally, several numerical tests are conducted on Tesla V100 Cards to show performance of the parallelization. In addition, we utilize the NVIDIA performance testing tool,

n v p r o f

, to evaluate multiple metrics of the kernel functions and conduct a detailed analysis of the results. In the tests of parallel scalability, the weak scaling efficiency achieves 97% from 4 to 32 GPU cards, and the strong scaling efficiency is 90% from 1 to 8 GPU cards.

查看原文本刊更多论文

基于 GPGPU 的直接非连续伽勒金方法的异构并行实施

本文基于 GPGPU 异构并行策略，针对三维非结构网格上的直接非连续方法（DDG）实现了 CUDA 和 CUDA/MPI 混合并行计算。直接非连续伽勒金方法继承了非连续伽勒金（DG）方法的紧凑性，因此非常适合大规模并行化。首先，我们介绍了具有单元级并行性和面级并行性的三维（3D）DDG 方法的完整单 GPU 实现。在这里，所有数值运算符，包括体积积分、面积分（数值通量）、守恒变量计算和时间迭代，都是通过设计相应的内核函数实现的。特别是，我们实施了几个关键的内存访问优化策略，这对提高性能至关重要。运算符合并和共享内存利用减少了全局访问次数。内存凝聚和数据结构重构明显提高了全局内存访问的效率。为了与数据访问模式保持一致，我们采用了原子操作来消除数据竞赛条件。此外，我们还提出了一种完全混合的 GPU/CPU 异构并行策略，以实现 DDG 方法的多 GPU 并行化，其中引入了异步优化，使通信和计算完全重叠，基本消除了通信开销。最后，我们在 Tesla V100 卡上进行了几次数值测试，以显示并行化的性能。此外，我们还利用英伟达™（NVIDIA®）性能测试工具 nvprof 评估了内核函数的多个指标，并对结果进行了详细分析。在并行可扩展性测试中，4 到 32 个 GPU 卡的弱扩展效率达到 97%，1 到 8 个 GPU 卡的强扩展效率为 90%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

ACS Applied Energy Materials Materials Science-Materials Chemistry

CiteScore

10.30

自引率

6.20%

发文量

1368

期刊介绍： ACS Applied Energy Materials is an interdisciplinary journal publishing original research covering all aspects of materials, engineering, chemistry, physics and biology relevant to energy conversion and storage. The journal is devoted to reports of new and original experimental and theoretical research of an applied nature that integrate knowledge in the areas of materials, engineering, physics, bioscience, and chemistry into important energy applications.