IEEE Computer Architecture Letters最新文献_第3页

Privilege Level Dynamics and Their Impact on Conditional Branch Prediction Performance in FaaS 权限级别动态及其对FaaS中条件分支预测性能的影响

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-06-12 DOI: 10.1109/LCA.2026.3703184

Louka Yerly;Arthur Perais

引用次数: 0

Toppy: A Global Offset-Based Data Prefetcher for Instruction Heavy Server Workloads Toppy：用于指令重服务器工作负载的基于偏移量的全局数据预取器

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-07-13 DOI: 10.1109/LCA.2026.3713016

Naman Sharma;Biswabandan Panda

{"title":"Toppy: A Global Offset-Based Data Prefetcher for Instruction Heavy Server Workloads","authors":"Naman Sharma;Biswabandan Panda","doi":"10.1109/LCA.2026.3713016","DOIUrl":"https://doi.org/10.1109/LCA.2026.3713016","url":null,"abstract":"Hardware data prefetching is a well-established technique for mitigating the long latency of off-chip memory accesses. However, there is little focus on data prefetching techniques specifically designed for server workloads with huge instruction footprints (10 s of MBs). As the instruction footprint of server workloads is higher than that of workloads like SPEC CPU2017, even to achieve decent performance gains, substantial metadata storage is needed, especially for prefetchers that use the program counter (PC) to understand memory access patterns. On top of that, server workloads exhibit irregular memory access patterns as control flow jumps between user and kernel level accesses. To address these issues, we propose Toppy, a global offset-based hardware prefetching technique tailored for server workloads that does not use PC for data prefetching. Toppy uses the Misra-Gries algorithm to select the top-k persistent global offsets that persist for the entire program and prefetch based on those offsets. Compared to an L1D IP-stride prefetcher, Toppy provides a performance improvement of 9.22%, whereas the next-best Bingo prefetcher delivers a performance boost of 6.28%. Toppy’s performance comes with a total storage budget of 274 bytes, whereas Bingo requires 119 KB.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"279-282"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148747055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Establishing Confidence in Average Metrics 建立对平均指标的信心

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-09-02 DOI: 10.1109/LCA.2026.3730505

Lieven Eeckhout

引用次数: 0

Quantum-Modulated Poisson Encoders for Hybrid QPU Workloads: A Closed-Form Hardware-Selection Frontier for Edge Inference 用于混合QPU工作负载的量子调制泊松编码器：边缘推断的封闭形式硬件选择前沿

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-07-29 DOI: 10.1109/LCA.2026.3718294

Magnus H. Strømme

{"title":"Quantum-Modulated Poisson Encoders for Hybrid QPU Workloads: A Closed-Form Hardware-Selection Frontier for Edge Inference","authors":"Magnus H. Strømme","doi":"10.1109/LCA.2026.3718294","DOIUrl":"https://doi.org/10.1109/LCA.2026.3718294","url":null,"abstract":"QPU interfaces add macroscopic I/O latency to hybrid edge inference that saturated-batch reporting can understate. We propose a deterministic proxy workload—the Quantum-Modulated Poisson Encoder—to isolate QPU-classical synchronisation overhead when an edge core issues low-rate queries to a remote or co-located coprocessor. Driving a non-homogeneous Poisson process with a Parametrised Quantum Circuit (PQC), we validate an additive overhead model end-to-end on local simulators and superconducting hardware and evaluate five interface-latency scenarios. Under a favourable illustrative 50 ms cloud-API round-trip, the penalty is 50.9% at saturated batch (<inline-formula><tex-math>$B=256$</tex-math></inline-formula>) and 98.3% at <inline-formula><tex-math>$B=1$</tex-math></inline-formula>. We derive the selection frontier <inline-formula><tex-math>$tau ^{*}_{text{QPU}}(B;p)=(p/(1-p))T_{text{core}}(B)$</tex-math></inline-formula>. A controlled ablation verifies that the tested PQC is non-contributory to accuracy, isolating interface effects; first-order thermal analysis shows that monolithic Through-Silicon Via (TSV) integration—the only paradigm clearing a 1% budget at <inline-formula><tex-math>$B=1$</tex-math></inline-formula>—violates the cryogenic cooling budget by orders of magnitude. Thus batch amortisation with thermally admissible co-packaging, rather than monolithic integration, is the viable near-term path.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"347-350"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149033505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SCALE-Sim TPU: Validating and Extending SCALE-Sim for TPUs SCALE-Sim TPU：验证和扩展用于TPU的SCALE-Sim

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-08-14 DOI: 10.1109/LCA.2026.3724136

Jingtian Dang;Ritik Raj;Changhai Man;Jianming Tong;Tushar Krishna

引用次数: 0

Hot–Cold Tiering of HBM and High Bandwidth Flash for Agentic LLM Serving 代理LLM服务中HBM的冷热分层和高带宽闪存

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-08-31 DOI: 10.1109/LCA.2026.3729099

Jongjin Baek;Won Ji;Seungjae Yoo;Joo-Young Kim

{"title":"Hot–Cold Tiering of HBM and High Bandwidth Flash for Agentic LLM Serving","authors":"Jongjin Baek;Won Ji;Seungjae Yoo;Joo-Young Kim","doi":"10.1109/LCA.2026.3729099","DOIUrl":"https://doi.org/10.1109/LCA.2026.3729099","url":null,"abstract":"Large language model (LLM) serving is increasingly agentic, with multi-turn sessions that idle between actions yet must retain their full context. Limited GPU memory capacity forces inactive KV states to be evicted, so resuming a session incurs either costly recomputation or slow interconnect transfers. To address this, high bandwidth flash (HBF)—an on-package 3D-NAND memory offering orders-of-magnitude greater capacity than high bandwidth memory (HBM) at comparable read bandwidth—has emerged as a strong candidate. However, its high read energy and limited write endurance make it impractical to serve <i>all</i> KV traffic. Fortunately, our analysis shows that agentic KV states exhibit distinct access patterns: a small hot set is read for every decoding step, while a large cold pool is read only when a paused session resumes. Exploiting this, we place the hot set in HBM and the cold pool in HBF, forming a hot–cold KV hierarchy within the GPU memory tier. On agentic workloads with Qwen3-Coder-30B-A3B, our design delivers 14 ms time-between-tokens (TBT) and adds only <inline-formula><tex-math>$approx !0.1$</tex-math></inline-formula> ms of resume latency on top of prefill, while hosting <inline-formula><tex-math>$24times$</tex-math></inline-formula> more concurrent sessions per GPU. By confining HBM to the hot set, our design also cuts read power by 7.6 kW per 8-GPU node relative to serving all KV from flash—establishing HBF as a cold-tier complement to HBM rather than its replacement.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"355-358"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149033810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Prompt Caching on Flash: Achieving Memory-Class Latency for LLM Inference Flash上的提示缓存：实现LLM推理的内存类延迟

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-07-09 DOI: 10.1109/LCA.2026.3711675

Joohee Lee;Kwangjin Ko;Taeho Hwang;Woosuk Chung;Hoshik Kim;Donguk Moon

{"title":"Prompt Caching on Flash: Achieving Memory-Class Latency for LLM Inference","authors":"Joohee Lee;Kwangjin Ko;Taeho Hwang;Woosuk Chung;Hoshik Kim;Donguk Moon","doi":"10.1109/LCA.2026.3711675","DOIUrl":"https://doi.org/10.1109/LCA.2026.3711675","url":null,"abstract":"Large Language Model (LLM) inference increasingly faces latency and cost challenges as model sizes and user demand grow. Prompt caching reduces redundant computation by reusing previously computed Key-Value (KV) data, but the reusable KV working set in online services can rapidly exceed the capacity of GPU memory and host DRAM. As a result, recent LLM serving systems increasingly adopt hierarchical KV cache architectures spanning HBM, DRAM, and SSD/storage tiers. This work focuses on optimizing the SSD-hit retrieval path within such multi-tier prompt cache systems. Under prompt caching with high cache hit rates, preparing KV data becomes dominated by their movement rather than computation, so achieving low latency depends on how effectively interconnect bandwidth is utilized. We enhance this path by shaping I/O behavior to activate SSD internal parallelism and by exploiting external parallelism through multiple NVMe devices. With these optimizations, SSD-backed prompt caching delivers Time-To-First-Token (TTFT) latency comparable to DRAM-backed prompt caching while providing far greater capacity at lower cost. We further analyze SSD endurance under realistic LLM serving workloads and quantify the capacity and Drive Writes Per Day (DWPD) requirements for sustainable deployment. Our results show that scaling from one to four SSDs reduces TTFT by 42% through external parallelism, while internal parallelism optimizations provide an additional 45% reduction—together enabling SSD-backed prompt caching to approach memory-class latency. These findings demonstrate that, with endurance-aware system design, modern SSDs can serve as a practical, low cost, high performance foundation for scalable prompt caching in LLM serving systems.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"295-298"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148747064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Perceptron-Based Data Prefetching Architecture 一种基于感知器的数据预取架构

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-08-31 DOI: 10.1109/LCA.2026.3729543

Sang Wook Stephen Do;Guangyuan Hu

引用次数: 0

BlockPIM: Enabling Sparse LLM Inference on Dense PIM Architectures BlockPIM：在密集PIM架构上启用稀疏LLM推理

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-07-21 DOI: 10.1109/LCA.2026.3715204

Seulki Kim;Yewon Jang;Yunhyeong Jeon;Kyeonghyeon Ryu;Daehoon Kim

{"title":"BlockPIM: Enabling Sparse LLM Inference on Dense PIM Architectures","authors":"Seulki Kim;Yewon Jang;Yunhyeong Jeon;Kyeonghyeon Ryu;Daehoon Kim","doi":"10.1109/LCA.2026.3715204","DOIUrl":"https://doi.org/10.1109/LCA.2026.3715204","url":null,"abstract":"Large Language Models (LLMs) have intensified the memory wall during inference, where performance is increasingly limited by memory bandwidth rather than computation. While Processing-In-Memory (PIM) and model compression techniques mitigate this issue, existing sparse PIM designs rely on CSR/COO formats that assume ultra-high sparsity (e.g., <inline-formula><tex-math>$> 90%$</tex-math></inline-formula>), which is difficult to achieve in practical LLMs without significant accuracy loss. Moreover, fine-grained indexing introduces substantial metadata overhead, diminishing the benefits of pruning, especially under low-bit quantization. To address this mismatch, we propose <monospace>BlockPIM</monospace>, a PIM architecture that employs hardware-aligned coarse-grained pruning to efficiently execute sparse LLMs on standard dense computation units. <monospace>BlockPIM</monospace> replaces per-element indices with compact bitmaps and contiguous non-zero blocks, enabling sparse workloads to be processed directly on dense execution pipelines without complex index decoding or irregular control flow. Experimental results show that <monospace>BlockPIM</monospace> achieves up to 8.0× and 3.6× speedups over the NPU and state-of-the-art sparse PIM baselines, respectively, while reducing energy consumption by up to 95.2% and providing superior memory efficiency under practical accuracy constraints.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 2","pages":"311-314"},"PeriodicalIF":2.0,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148836601","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Speculative Runahead Retrieval for Storage-Intensive Agentic RAG 存储密集型代理RAG的预见性提前检索

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-07-01 Epub Date: 2026-08-24 DOI: 10.1109/LCA.2026.3726677

Junsuk Lee;Dooik Kim;Woojin Son;Hunjun Lee

引用次数: 0