{"title":"Precision-Flexible Processing Units for Fully Utilizing Internal Bandwidth in PIM","authors":"Suhwan Kim;Changyong Eom;Sieun Moon;Siu Jeong;Yelim Han;Jaehyuk Huh","doi":"10.1109/LCA.2026.3737969","DOIUrl":"https://doi.org/10.1109/LCA.2026.3737969","url":null,"abstract":"Recent commercial DRAM-based PIM (Processing-in-Memory) accelerates memory-bound GEMV but supports only fixed FP16/BF16 arithmetic, while modern workloads require diverse low-bit formats. Expanding low-bit operands to FP16 keeps a 16-lane processing unit (PU) consuming a fixed number of operands per command and re-reading the same column. Spatially scaling the datapath recovers throughput but costs up to 4.44 × the area of a fixed FP16 PU. Since a column need not be consumed in a single DRAM cycle, we instead temporally reuse a single narrow multiplier and one shared accumulation path across the phases of one column interval. Our PU supports 15 operand-format combinations and achieves up to 3.75 × speedup and 2.7 × lower energy for memory-bound GEMV over an FP16 PIM baseline with 16% area overhead.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"403-406"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149188547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"NeuroMTA: Programmable Simulation Framework for Multi-Tile NPU Architectures","authors":"Seongwook Kim;Seokin Hong","doi":"10.1109/LCA.2026.3716090","DOIUrl":"https://doi.org/10.1109/LCA.2026.3716090","url":null,"abstract":"Multi-Tile Accelerators (MTAs) have emerged as a promising paradigm to scale the computational throughput of NPUs for modern deep learning workloads. However, existing NPU simulation frameworks are insufficient for evaluating modern MTAs, as they fail to capture the inter-core data transactions that serve as the primary spatial data reuse mechanism in MTAs. To bridge this gap, we introduce NeuroMTA, a cycle-accurate simulation framework that couples a reconfigurable hardware template with a compiler stack. The hardware template models both NUMA-based and FIFO-driven data sharing, and the compiler stack exposes fine-grained control over data reuse patterns across multiple compute tiles. Our experimental validation against a real MTA device shows that NeuroMTA achieves a Pearson correlation of 0.9.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"283-286"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148746861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs","authors":"Juyeon Lee;Jiayue Wang;Soojung Ryu;Hyuk-Jae Lee","doi":"10.1109/LCA.2026.3703423","DOIUrl":"https://doi.org/10.1109/LCA.2026.3703423","url":null,"abstract":"LLM inference on unified-memory (UM) edge SoCs is commonly assumed to follow a GPU-centric execution model once nominal memory capacity appears sufficient. Our experiments on an NVIDIA Jetson Orin Nano 8 GB reveal a prefill feasibility boundary in the evaluated modified FlexGen-based runtime, where execution stability becomes sensitive to runtime memory-management behavior. Under GPU-centric execution, collapse is first observed at batch size 8 with zram enabled, and the boundary shifts to batch size 16 under a modified memory configuration. At these boundary points, completion becomes probabilistic, with only 23%–33% of runs succeeding; all failures occur before decode. Notably, collapse can occur despite apparent physical-memory headroom, showing that nominal capacity alone does not determine execution stability. These findings show that, in the evaluated setup, prefill completion acts as an effective feasibility gate: latency and throughput become meaningful only after execution feasibility is established.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"267-270"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148506191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiangqun Zhang;Juhyung Park;Seonggyun Oh;Kyeyoon Park;Jinki Han;Eunji Lee;Sungjin Lee;Yeseong Kim;Bryan S. Kim
{"title":"Packet-Level CXL.cache Analysis With ${sf OpenCIS}$OpenCIS","authors":"Xiangqun Zhang;Juhyung Park;Seonggyun Oh;Kyeyoon Park;Jinki Han;Eunji Lee;Sungjin Lee;Yeseong Kim;Bryan S. Kim","doi":"10.1109/LCA.2026.3728300","DOIUrl":"https://doi.org/10.1109/LCA.2026.3728300","url":null,"abstract":"CXL.cache enables devices to coherently access host memory, but its behavior is difficult to understand without packet-level visibility into ownership transfers, cache line invalidations, and host-device request flows. Packet-level visibility into CXL.cache is thus crucial for CXL research, but it is largely ignored by existing CXL tools. Therefore, we present <inline-formula><tex-math>${sf OpenCIS}$</tex-math></inline-formula>, an open-source CXL simulation platform on the transaction layer that supports CXL.cache and flexible CXL topologies. We show that <inline-formula><tex-math>${sf OpenCIS}$</tex-math></inline-formula>’s transaction-layer visibility is useful for both CXL.cache workload-side optimization and protocol-side exploration by implementing two workloads using <inline-formula><tex-math>${sf OpenCIS}$</tex-math></inline-formula>, namely PageRank and FlushData. For PageRank, we explore the importance of CXL-aware workload-side optimization by showing the differences between two workload-sharding approaches under a multi-accelerator scenario. For FlushData, we reveal the benefits of a new CXL.cache opcode that actively pushes needed data to accelerators from the host.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"351-354"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11668657","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149033444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Back-Invalidate Timing: Coherence Channels in CXL 3.x Device-Coherent Memory","authors":"Oğuz Ergin","doi":"10.1109/LCA.2026.3738639","DOIUrl":"https://doi.org/10.1109/LCA.2026.3738639","url":null,"abstract":"A cache line held exclusively by one core costs more to read than a shared one, and that difference carries a channel. CXL 3.x permits device-coherent memories on which that asymmetry reappears across a trust boundary, in two forms. A state channel follows from ordinary coherence resolution. An occupancy channel extends the specification’s own worked example, where a read stalls behind a back-invalidate snoop to an <italic>unrelated</i> address until a snoop-filter entry can be allocated, so a requester reads pressure on a device-side structure without touching a line its peers touch. We name the assumptions each rests on, and the occupancy-channel construction runs between two hosts on an unmodified third-party model of a device-side filter, separating the two symbols with one error in 288 on a deterministic clock. The confidential-computing protocol declares the occupancy channel’s deployment in scope, then assumes away the resource it reads.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"395-398"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149188545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"SPIRE-HBM: A Sub-Pseudo-Channel Architecture for HBM4 Bandwidth Scaling","authors":"Jungsul Lee;Eojin Lee","doi":"10.1109/LCA.2026.3705360","DOIUrl":"https://doi.org/10.1109/LCA.2026.3705360","url":null,"abstract":"High-frequency HBM4 suffers from bandwidth underutilization because each pseudo channel (pCH) has only two bank groups within a selected Stack ID (SID), making BG interleaving insufficient to hide long column-to-column timing constraints. Existing industrial approaches mitigate this problem by adding DQ TSV resources, increasing implementation cost. We propose SPIRE-HBM, a sub-pseudo-channel-based HBM4 architecture that restores seamless BG-interleaved reads without increasing DQ TSVs. SPIRE-HBM partitions each 32-DQ pCH into two 16-DQ sub-pCH data paths across Stack IDs and doubles the burst length to preserve conventional access granularity. The longer burst provides sufficient column-command separation for two-BG interleaving, shifting the limiting column timing from <inline-formula><tex-math>$text{tCCD}_{text{L}}$</tex-math></inline-formula> to <inline-formula><tex-math>$text{tCCD}_{text{S}}$</tex-math></inline-formula>. Our evaluation shows that SPIRE-HBM improves performance by up to 1.10× for Llama3-8B and 1.15× for synthetic workloads over HBM4. In addition, at 12.8 Gbps, SPIRE-HBM achieves performance comparable to a TSV-doubled HBM4 baseline while retaining the DQ TSV count.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"263-266"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148504705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"DYCA-HBM: Accelerating Long-Context LLM on HBM-PIM With Dynamic Channel-Allocation","authors":"Sangkyu Jeon;Jungsul Lee;Eojin Lee;Sungju Ryu","doi":"10.1109/LCA.2026.3725787","DOIUrl":"https://doi.org/10.1109/LCA.2026.3725787","url":null,"abstract":"In long-context LLMs, attention is costly due to repeated accesses to a Key–Value (KV) cache. HBM-PIM can accelerate this memory-bound attention by leveraging its massive internal bandwidth. To effectively exploit this parallelism, prior PIM accelerators have proposed request-wise, head-wise, and block-wise KV-cache mappings to distribute attention workload across channels. However, these mapping schemes do not fully exploit the available internal bandwidth because they do not account for load changes caused by asynchronous request completion. To address this limitation, we propose DYCA-HBM, a dynamic channel allocation scheme that maintains balanced channel utilization across the entire decoding phase. DYCA-HBM tracks how each request’s KV is distributed across all channels. It maps new KV blocks to the least-loaded channel for load balancing, and prevents a single request’s blocks from excessively accumulating in specific channels. Experimental results show that DYCA-HBM improves throughput by up to 1.08× and 1.07× over AttAcc and BlockPIM, respectively, while reducing energy consumption by up to 7% with less than 0.07% logic-die area overhead.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"359-362"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149033754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Taxonomy of SIMDe Emulation Inefficiencies for ARM NEON Porting of VVC Encoders","authors":"Jeongkeun Kim","doi":"10.1109/LCA.2026.3725622","DOIUrl":"https://doi.org/10.1109/LCA.2026.3725622","url":null,"abstract":"Compile-time x86<inline-formula><tex-math>$to$</tex-math></inline-formula>ARM SIMD translation via SIMDe is widely used to port intrinsic-heavy codebases to Neoverse-class cores, but its per-instruction translation scope leaves systematic architectural-asymmetry overhead uncharacterized in the architecture literature. We present a six-type taxonomy of SIMDe emulation inefficiencies— Redundant (R), Suboptimal (S), Widening (W), Fusion-miss (F), Memory (M), and Pipeline (P)—with assembly-level evidence for each type on AWS Graviton3 (Neoverse V1), organised under two underlying principles (per-instruction translation scope vs. architectural asymmetry). Using VVenC—the dominant open-source VVC encoder—as a case study, the taxonomy identifies 84 pattern instances across five SIMDe-dependent modules and guides native NEON implementations for three of them (DepQuant, Quant, TrQuant), achieving 1.22–1.40× encoder speedup (SIMDe-only to ours, Build A <inline-formula><tex-math>$to$</tex-math></inline-formula> C) and +0.4–+1.9% incremental gain over the upstream NEON port (Build B <inline-formula><tex-math>$to$</tex-math></inline-formula> C) on AWS Graviton3 and Graviton4 (medium preset, three sequences), with bit-exact output verified by MD5. The taxonomy is the primary contribution: a reusable diagnostic framework applicable to other SIMDe-using codebases targeting ARM.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"335-338"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148985786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Junho Park;Hyowon An;Haeseok Suh;Youngsu Yoon;Hyuni Lee;Joungho Kim
{"title":"HBM-HBF-Centric Memory Pooling Architecture With Custom Base Die for Terabyte-Scale LLM Inference","authors":"Junho Park;Hyowon An;Haeseok Suh;Youngsu Yoon;Hyuni Lee;Joungho Kim","doi":"10.1109/LCA.2026.3703982","DOIUrl":"https://doi.org/10.1109/LCA.2026.3703982","url":null,"abstract":"This paper proposes a memory pooling architecture that integrates High Bandwidth Memory (HBM) and High Bandwidth Flash (HBF) through a Custom Base Die (CBD). As LLMs continue to scale, the combined memory footprint of model weights and KV cache increasingly exceeds HBM capacity, creating a critical bottleneck during inference. The proposed architecture extends memory capacity to the terabyte scale at the package level by stacking HBF behind HBM, with deterministic layer-wise prefetching and Buffered Write-Back (BWB) offloaded into CBD hardware without host-side intervention. This masks HBF read and write latencies entirely within the CBD, enabling the HBM-HBF pool to operate as a unified memory tier. Simulation results using LLaMA-4 Maverick demonstrate a maximum throughput gain of 87.4× over conventional GPU-HBM systems under extreme memory pressure, confirming that logic-integrated base dies are essential for sustaining next-generation LLM inference.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"259-262"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148442724","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dowon Son;Yonggon Park;Hyunuk Cho;Hyungkyu Ham;Onur Mutlu;Sungjin Lee;Gwangsun Kim;Jisung Park
{"title":"Exploring High-Bandwidth Flash for Modern LLM Inference: Opportunities and Challenges","authors":"Dowon Son;Yonggon Park;Hyunuk Cho;Hyungkyu Ham;Onur Mutlu;Sungjin Lee;Gwangsun Kim;Jisung Park","doi":"10.1109/LCA.2026.3705817","DOIUrl":"https://doi.org/10.1109/LCA.2026.3705817","url":null,"abstract":"This work investigates the potential benefits and technical challenges of using high-bandwidth flash (HBF) for large language model (LLM) inference. HBF has gained increasing attention as a promising solution to mitigate memory-capacity bottlenecks in modern LLM-serving systems, but its benefits and challenges remain largely uninvestigated. To address this gap, we thoroughly analyze HBF-based LLM-serving systems under diverse system configurations and operating scenarios in which HBF serves as a main GPU-memory component to handle both reads and writes. Our analysis shows that, despite its limited write performance, HBF can significantly improve the batch size, throughput, and flexibility of LLM-serving systems while reducing the minimum GPU requirements, but realizing these benefits critically depends on sustaining HBM-comparable read bandwidth and requires significant endurance improvements.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"251-254"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148442842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}