IOPS: A Unified SpMM Accelerator Based on Inner-Outer-Hybrid Product

Abstract

Sparse matrix multiplication (SpMM) is widely applied to numerous domains, such as graph processing and machine learning. However, inner product (IP) induces redundant zero-element computing for mismatched nonzero operands, while outer product (OP) lacks input reuse across Process Elements (PEs). Besides, current accelerators only focus on sparse-sparse matrix multiplication (SSMM) or sparse-dense matrix multiplication (SDMM), rarely performing efficiently for both. To compensate for the shortcomings of IP and OP, we propose an inner-outer-hybrid product (IOHP) method, which reuses the input matrix among PEs with IP and removes zero-element calculations with OP in each PE. Based on IOHP, we co-design a accelerator with a unified computing flow, called IOPS, to efficiently process both SSMM and SDMM. It divides the SpMM into three stages: encoding, partial sum (psum) calculation, and address mapping, where the input matrices can be reused among PEs after encoding (IP) and the zero element can be skipped in the latter two stages (OP). Furthermore, an adaptive partition strategy is proposed to tile the input matrices based on their sparsity ratios, effectively utilizing the on-chip storage and reducing DRAM access. Compared with SpArch, we achieve $1.2\boldsymbol{\times}$~$4.3\boldsymbol{\times}$ performance and $1.3\boldsymbol{\times}$~$4.8\boldsymbol{\times}$ energy efficiency, with $1.4\boldsymbol{\times}$~$2.1\boldsymbol{\times}$ DRAM access saving.