IEEE Computer Architecture Letters最新文献

3DLS: A 3D Logic-Stacked Architecture for Disaggregated LLM Serving 3DLS：一种用于分解LLM服务的3D逻辑堆叠架构

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 DOI: 10.1109/LCA.2026.3709108

Jaehun Lee;In-Jun Jung;Joo-Young Kim

引用次数: 0

PLUG: Processing-in-Memory LUT Gathering for Graph ANNS PLUG：用于图形神经网络的内存处理LUT收集

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-07-13 DOI: 10.1109/LCA.2026.3712764

Seonah Yoo;Jinwook Yang;Hyeonu Kim;Yongwon Shin;Dookyung Kang;Changmin Shin;Jinho Lee;Hyojin Sung

{"title":"PLUG: Processing-in-Memory LUT Gathering for Graph ANNS","authors":"Seonah Yoo;Jinwook Yang;Hyeonu Kim;Yongwon Shin;Dookyung Kang;Changmin Shin;Jinho Lee;Hyojin Sung","doi":"10.1109/LCA.2026.3712764","DOIUrl":"https://doi.org/10.1109/LCA.2026.3712764","url":null,"abstract":"Approximate nearest neighbor search (ANNS) is a key primitive in vector databases and graph-based indexes. In practice, HNSW with PQ compression is widely adopted because it offers a strong recall-latency trade-off with a compact memory footprint. However, HNSW-PQ is memory-bound, with PQ distance computation requiring repeated fine-grained lookup table (LUT) reads and PQ code fetches for a large candidate set. We present PLUG, an HBM-based PIM design that accelerates PQ-based graph ANNS by offloading LUT gathering and distance accumulation. PLUG extends in-DRAM gather primitives with a PQ-friendly memory layout and all-bank execution support. By pinning the per-query LUT in memory and gathering LUT entries in parallel across banks, PLUG reduces command and activation overhead while exploiting bank-level parallelism. Evaluated on four standard ANN workloads, PLUG accelerates PQ distance computation by up to <inline-formula><tex-math>$3.74times$</tex-math></inline-formula> (<inline-formula><tex-math>$2.70times$</tex-math></inline-formula> on average) over the HBM2 baseline and improves end-to-end performance by up to <inline-formula><tex-math>$1.53times$</tex-math></inline-formula> (<inline-formula><tex-math>$1.30times$</tex-math></inline-formula> on average) over CPU-only execution.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"303-306"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148747380","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Agentic LLMs for Microarchitecture Research: The Role of the Human Architect 微建筑研究代理法学硕士：人类建筑师的角色

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-07-30 DOI: 10.1109/LCA.2026.3718795

Georgios Vavouliotis

引用次数: 0

Hardware-Managed Heterogeneous High-Bandwidth Memory and Flash in LLM Inference Systems LLM推理系统中硬件管理的异构高带宽存储器和闪存

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-08-12 DOI: 10.1109/LCA.2026.3723326

Hakam Atassi;Noa Zilberman;Amro Awad

引用次数: 0

NDPool: Correctness-Preserving Shared Execution for Efficient LLM Inference on CXL-NDP Systems NDPool：在CXL-NDP系统上有效LLM推理的保持正确性的共享执行

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-06-12 DOI: 10.1109/LCA.2026.3702973

Yeji Jung;Hwanjun Lee;Sungju Kim;Seulki Kim;Yunhyeong Jeon;Daehoon Kim

{"title":"NDPool: Correctness-Preserving Shared Execution for Efficient LLM Inference on CXL-NDP Systems","authors":"Yeji Jung;Hwanjun Lee;Sungju Kim;Seulki Kim;Yunhyeong Jeon;Daehoon Kim","doi":"10.1109/LCA.2026.3702973","DOIUrl":"https://doi.org/10.1109/LCA.2026.3702973","url":null,"abstract":"The inference performance of Large Language Models (LLMs) is increasingly constrained by memory capacity and bandwidth rather than raw computation, particularly for long-context workloads. CXL-based Near-Data Processing (CXL-NDP) has emerged as a promising solution for expanding memory capacity, but existing systems suffer from severe resource under-utilization due to exclusive device allocation under hybrid parallel execution. To minimize per-request latency, prior designs dedicate multiple NDP devices to tensor parallelism for FC layers, while non-FC layers are executed only on a single master device. As a result, most auxiliary devices remain idle for a large fraction of execution time, fundamentally limiting overall system throughput. Safe device sharing is challenging because out-of-order peer-to-peer communication can break request isolation and computational correctness at the shared CXL receive path. To address this limitation, we propose <monospace>NDPool</monospace>, a correctness-preserving shared execution framework that dynamically reuses idle TP devices across concurrent requests for FC-layer execution. <monospace>NDPool</monospace> combines ordered receive control for safe data delivery with adaptive scheduling to reduce execution bubbles and maximize shared-device utilization. Experimental results show that, under equivalent service-level objective (SLO) constraints, <monospace>NDPool</monospace> improves system throughput by up to 3.7× and by 1.5× on a geometric mean basis over the state-of-the-art CXL-NDP hybrid parallel design.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"243-246"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148442768","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SoftmaxPIM: An HBM-Based PIM Architecture for Accelerating GeMV–Softmax Execution Pipeline SoftmaxPIM：一种基于hbm的加速GeMV-Softmax执行管道的PIM架构

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-07-23 DOI: 10.1109/LCA.2026.3716671

Taewoon Kang;Namhun Kim;Jihun Lee;Murali Annavaram;Gunjae Koo

{"title":"SoftmaxPIM: An HBM-Based PIM Architecture for Accelerating GeMV–Softmax Execution Pipeline","authors":"Taewoon Kang;Namhun Kim;Jihun Lee;Murali Annavaram;Gunjae Koo","doi":"10.1109/LCA.2026.3716671","DOIUrl":"https://doi.org/10.1109/LCA.2026.3716671","url":null,"abstract":"LLM inference is dominated by GeMV and softmax computations during attention. These computations impose substantial compute and memory bandwidth demands. Prior works have demonstrated that HBM-based PIM architectures effectively accelerate GeMV. However, softmax remains a critical bottleneck due to frequent data movement. In this work, we make the case for computing softmax operations in memory and integrate them with in-memory GeMV operations to provide a comprehensive in-memory LLM accelerator engine. We propose SoftmaxPIM, an in-HBM compute architecture that tightly integrates GeMV with a lightweight near-bank softmax unit. The proposed softmax unit employs an approximation-based softmax formulation that removes reduction operations, enabling the entire GeMV–softmax pipeline to execute within the HBM memory stack. By eliminating host-memory transfers, the proposed design significantly reduces softmax latency. Our simulation results demonstrate up to 42.7× speedup over a CPU and 2.64× over an RTX3090 GPU, as well as up to 1.45× improvement over the prior in-memory attention accelerator AttAcc.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"287-290"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148748729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Memory Fabric: HBM Die-to-Die Interconnect for GPU Bandwidth Expansion 内存结构：用于GPU带宽扩展的HBM Die-to-Die互连

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-08-10 DOI: 10.1109/LCA.2026.3721758

Hyowon An;Gyeongjae Lee;Zhixian Jin;Joungho Kim;John Kim

引用次数: 0

An Autonomous Hazard-Resolving MAC Unit With Conditional Reduction for DRAM-PIM 一种具有条件约简的自动危险分辨MAC单元

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-08-25 DOI: 10.1109/LCA.2026.3727232

Wontak Han;Taeyang Jeong;Yoonah Paik;Byeongho Kim;Jaewan Choi;Sukhan Lee;Hongju Kal;Kyomin Sohn

{"title":"An Autonomous Hazard-Resolving MAC Unit With Conditional Reduction for DRAM-PIM","authors":"Wontak Han;Taeyang Jeong;Yoonah Paik;Byeongho Kim;Jaewan Choi;Sukhan Lee;Hongju Kal;Kyomin Sohn","doi":"10.1109/LCA.2026.3727232","DOIUrl":"https://doi.org/10.1109/LCA.2026.3727232","url":null,"abstract":"Processing-in-Memory (PIM) is a pivotal solution for accelerating memory-intensive AI workloads like General Matrix-Vector Multiplication (GEMV). However, the inherent speed limitations of DRAM process technology necessitate multi-cycle pipelined MAC units, which introduce critical accumulation hazards. Current industry solutions resolve these hazards by either sacrificing computational precision, compromising memory layout flexibility, or requiring invasive modifications to the host-side memory controller. This paper proposes Conditional Reduction, an autonomous hardware-based scheme that resolves accumulation hazards entirely within the PIM unit without host or controller intervention. By bridging the information asymmetry between the memory controller and internal DRAM pipeline states, our scheme triggers reduction only when data dependencies are detected. Utilizing a register-renaming mechanism with only five additional entries, the proposed architecture minimizes silicon overhead while maintaining strict JEDEC-standard compatibility. Experimental evaluations on LPDDR5X-9600 demonstrate that Conditional Reduction achieves up to a 1.504× speedup over conventional double-register approaches by eliminating explicit software fences and redundant reduction phases. Furthermore, it reduces the area overhead by 6.94% for a 2 kb output register configuration, offering a scalable and future-proof solution for high-performance DRAM-PIM architectures.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"339-342"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148985808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Address Matters: Modeling Boundaries of Bandwidth–Latency Memory Simulation 地址问题：带宽延迟内存模拟的建模边界

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-09-10 DOI: 10.1109/LCA.2026.3733002

Seungmin Baek;Seungmin Ahn;Jumin Kim;Hwayong Nam;Jung Ho Ahn

{"title":"Address Matters: Modeling Boundaries of Bandwidth–Latency Memory Simulation","authors":"Seungmin Baek;Seungmin Ahn;Jumin Kim;Hwayong Nam;Jung Ho Ahn","doi":"10.1109/LCA.2026.3733002","DOIUrl":"https://doi.org/10.1109/LCA.2026.3733002","url":null,"abstract":"Analytical memory simulators that assign latency from bandwidth–latency curves profiled on real hardware promise real-system fidelity without microarchitectural detail. We measure where this abstraction breaks, taking the Mess simulator as an exemplar. Its runtime state—aggregate bandwidth and read/write ratio—conceals the access-pattern effects that govern real memory behavior. Controlled real-system measurements show that channel skewness causes up to 4.7 × latency variation at equal aggregate bandwidth and a 4.2 × saturation-bandwidth gap, while row-buffer locality and read/write ordering induce 36.4% and 21.3% latency variation. Its fixed window size and convergence factor further produce up to 194.5% parameter sensitivity across SPEC CPU workloads with different memory intensities. Reducing these discrepancies requires address- and phase-aware runtime signals. Our proof-of-concept (<inline-formula><tex-math>$text{Mess}^+$</tex-math></inline-formula>) tracks per-channel traffic using physical addresses, reducing the IPC divergence with Ramulator2 by 49.1% under channel-skewed traffic. The open challenge is incorporating additional signals without collapsing the lightweight abstraction into detailed timing simulation.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"379-382"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149033845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FlexNM: A Fully Flexible SpMM Accelerator for N:M Structured Sparsity FlexNM: N:M结构稀疏的全柔性SpMM加速器

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-09-16 DOI: 10.1109/LCA.2026.3735038

Jaehun Han;Wan Choi

{"title":"FlexNM: A Fully Flexible SpMM Accelerator for N:M Structured Sparsity","authors":"Jaehun Han;Wan Choi","doi":"10.1109/LCA.2026.3735038","DOIUrl":"https://doi.org/10.1109/LCA.2026.3735038","url":null,"abstract":"N:M structured pruning is widely adopted in modern transformer-based deep learning workloads to reduce model size and enable efficient Sparse-Dense Matrix Multiplication (SpMM). However, existing accelerators either support limited N:M configurations or require hardware regeneration for different sparsity patterns. This limitation prevents efficient acceleration under optimal sparsity patterns, forcing suboptimal N:M configurations or hardware redesign, and resulting in degraded model accuracy. To address this issue, we propose FlexNM, a fully flexible N:M structured SpMM accelerator. FlexNM employs a unified hardware architecture that supports a diverse range of N:M values, enabling direct acceleration of workloads at their optimal N:M sparsity configurations. To achieve this flexibility, FlexNM employs a novel asymmetric adder-tree (AAT) mapping scheme that efficiently supports diverse values of N. It also incorporates a bitmap-based compression scheme to reduce memory overhead and enhance N:M flexibility. FPGA experimental results show that FlexNM achieves up to a 15.94× speedup over the dense systolic-array baseline while requiring 1.47× LUT and 1.54× FF usage relative to the same baseline.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"391-394"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149130643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0