Enabling Computation and Communication Overlap in PIMs for On-Device LLM Inference

IF 2 3区计算机科学 Q4 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-03-23 DOI:10.1109/LCA.2026.3676795

Siu Jeong;Suhwan Kim;Changyong Eom;Jaehyuk Huh

引用次数: 0

Abstract

DRAM-based processing-in-memory (PIM) exploits all-bank execution to accelerate memory-bound workloads, but limited PIM capacity makes runtime data movement unavoidable and often performance-critical. Existing PIM architectures tightly couple computation and memory access across all banks, serializing computation and data transfer and limiting opportunities to hide communication latency. This paper proposes a bank-level computation–communication overlap mechanism that selectively assigns a subset of bank resources for communication, enabling concurrent data transfer without disrupting ongoing computation while preserving the unified control model of all-bank PIM. Evaluations on mixture-of-experts (MoE) and multi–small language model (multi-SLM) inference workloads using a simulated LPDDR5-based PIM system show execution time reductions of up to 19% and 16%, respectively, along with consistently improved channel bandwidth utilization compared to all-bank and channel-level overlap baselines.

查看原文本刊更多论文

基于设备上LLM推理的PIMs计算和通信重叠实现

基于dram的内存处理（PIM）利用全银行执行来加速内存受限的工作负载，但是有限的PIM容量使得运行时数据移动不可避免，并且通常对性能至关重要。现有的PIM体系结构将所有银行的计算和内存访问紧密耦合，序列化计算和数据传输，并限制隐藏通信延迟的机会。本文提出了一种银行级计算-通信重叠机制，该机制有选择地分配银行资源子集进行通信，在保持全银行PIM统一控制模型的同时，在不中断正在进行的计算的情况下实现数据的并发传输。使用模拟的基于lpddr5的PIM系统对混合专家（MoE）和多小语言模型（multi-SLM）推理工作负载的评估显示，与全银行和通道级重叠基线相比，执行时间分别减少了19%和16%，并且通道带宽利用率持续提高。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Computer Architecture Letters COMPUTER SCIENCE, HARDWARE & ARCHITECTURE-

CiteScore

4.60

自引率

4.30%

发文量

期刊介绍： IEEE Computer Architecture Letters is a rigorously peer-reviewed forum for publishing early, high-impact results in the areas of uni- and multiprocessor computer systems, computer architecture, microarchitecture, workload characterization, performance evaluation and simulation techniques, and power-aware computing. Submissions are welcomed on any topic in computer architecture, especially but not limited to: microprocessor and multiprocessor systems, microarchitecture and ILP processors, workload characterization, performance evaluation and simulation techniques, compiler-hardware and operating system-hardware interactions, interconnect architectures, memory and cache systems, power and thermal issues at the architecture level, I/O architectures and techniques, independent validation of previously published results, analysis of unsuccessful techniques, domain-specific processor architectures (e.g., embedded, graphics, network, etc.), real-time and high-availability architectures, reconfigurable systems.