An architecture-adaptive optimization strategy for high-performance SYMV on a heterogeneous AI accelerator

Abstract

Emerging AI accelerators offer strong compute density for HPC workloads, but decoupled execution engines and software-managed memory systems complicate performance portability. This paper studies the memory-bound SYmmetric Matrix–Vector multiplication (SYMV) kernel on Huawei Ascend A2, a heterogeneous architecture with disjoint Cube (AIC) and Vector (AIV) engines. We propose an architecture-adaptive mapping that (i) assigns off-diagonal dense tiles to AIC while keeping diagonal/finalization on AIV, (ii) orchestrates cross-engine execution with a three-stage software pipeline to overlap DMA, compute, and synchronization, and (iii) reduces off-chip matrix-read traffic via symmetry-aware traversal under triangular storage, together with a transpose-free diagonal-tile strategy on AIV. On Ascend A2, the proposed kernel achieves a consistent 1.3$\times$–1.6$\times$ speedup over the vendor matmul_gemv baseline, and we provide cross-platform context against cuBLAS (A100) and rocBLAS (MI210).