IEEE Computer Architecture Letters最新文献_第8页

Capacity-Latency Tradeoffs in CXL Memory Expander at Hyperscale 超大规模CXL内存扩展器的容量-延迟权衡

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-03-23 DOI: 10.1109/LCA.2026.3676988

Haneul Park;Grant Ayers;Nam Sung Kim;Philip Levis;Brian Morris

{"title":"Capacity-Latency Tradeoffs in CXL Memory Expander at Hyperscale","authors":"Haneul Park;Grant Ayers;Nam Sung Kim;Philip Levis;Brian Morris","doi":"10.1109/LCA.2026.3676988","DOIUrl":"https://doi.org/10.1109/LCA.2026.3676988","url":null,"abstract":"DRAM dominates the cost of hyperscale datacenter servers today and AI-driven memory supply shortages are pushing costs even higher. CXL memory expanders are a new technology to reduce server costs by reusing legacy DDR DRAM modules from older servers. Memory expanders further save cost through inline compression that increases the effective capacity. This paper investigates two critical design points of a memory expander: the compression algorithm and the compression block size. Both design points introduce tradeoffs between latency and capacity that involve complex interactions with the memory expander architecture, access patterns to the device, and the entropy of stored data. Current commercial devices use simple, fast compression and a 4 kB block as the default. Using a compressibility benchmark and the latest traces of cold memory accesses from a hyperscaler, we find that using a different compression algorithm and block size can simultaneously increase expander capacity <italic>and</i> reduce access latency. The key insight is counter-intuitive: using a slower, more complex compression algorithm improves latency because the reduction in DRAM traffic more than offsets the additional decompression time.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 1","pages":"122-125"},"PeriodicalIF":1.4,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11454555","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147665464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Simulator for LLM Inference Systems Exploiting CXL Memory Pools 利用CXL内存池的LLM推理系统模拟器

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-03-06 DOI: 10.1109/LCA.2026.3691007

Jinghan Huang;Hongkun Zeng;Mike Montano;Jaehong Cho;Hyunmin Choi;Jinin So;Junhyeok Im;Handeok Lee;Jongse Park;Nam Sung Kim

{"title":"A Simulator for LLM Inference Systems Exploiting CXL Memory Pools","authors":"Jinghan Huang;Hongkun Zeng;Mike Montano;Jaehong Cho;Hyunmin Choi;Jinin So;Junhyeok Im;Handeok Lee;Jongse Park;Nam Sung Kim","doi":"10.1109/LCA.2026.3691007","DOIUrl":"https://doi.org/10.1109/LCA.2026.3691007","url":null,"abstract":"The cost of large language model (LLM) inference is heavily tied to the immense memory capacity required to store model parameters—a challenge exacerbated by current systems that replicate these parameters across multiple servers to scale. To tackle this challenge, we may consider LLM inference systems exploiting a CXL memory pool to store and share model parameters across multiple servers, while keeping server-specific activations and KV-caches within each server’s local memory. Motivated by the lack of existing simulators for such systems, we first develop a simulator that extends LLMServingSim and integrates it with NS-3 and Mess to accurately model the CXL switch and the memory modules within the CXL pool, respectively. Second, we validate this simulator against a system connecting an industry CXL memory pool to two servers running two instances of the same LLM; across separate evaluations for OPT-1.6B and OPT-6.7B, the simulator demonstrates a <inline-formula><tex-math>$approx$</tex-math></inline-formula>3% error in inference latency. Lastly, using our simulator, we uncover a significant sensitivity between inference latency and the interconnect bandwidth of the CXL memory pool, demonstrating that adopting an emerging optical interconnect over the traditional electrical counterpart reduces inference latency by 67.9% and 64.4% for OPT-1.6B and OPT-6.7B, respectively.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 1","pages":"202-205"},"PeriodicalIF":1.4,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148171870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Near-HBM Tensor Core Acceleration for Fine-Grained Sparse Matrix-Matrix Multiplication 细粒度稀疏矩阵-矩阵乘法的近hbm张量核加速

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-03-04 DOI: 10.1109/LCA.2026.3690418

Lihan Hu;Shiju Li;Hoshik Kim;Jongryool Kim

引用次数: 0

Differential Privacy for Secure Cache Partitioning: Promises, Limits, and Fixes 安全缓存分区的差异隐私：承诺、限制和修复

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-03-27 DOI: 10.1109/LCA.2026.3697544

Prerna Priyadarshini;Abhijat Bharadwaj;Biswabandan Panda

引用次数: 0

InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference InfAMAX：弥合Intel AMX中高效LLM推理的计算与内存差距

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-03-14 DOI: 10.1109/LCA.2026.3693303

Kyeonghyeon Ryu;Yunhyeong Jeon;Gyeongseo Park;Minho Kim;Daehoon Kim

{"title":"InfAMAX: Bridging the Compute-Memory Gap in Intel AMX for Efficient LLM Inference","authors":"Kyeonghyeon Ryu;Yunhyeong Jeon;Gyeongseo Park;Minho Kim;Daehoon Kim","doi":"10.1109/LCA.2026.3693303","DOIUrl":"https://doi.org/10.1109/LCA.2026.3693303","url":null,"abstract":"Intel Advanced Matrix Extensions (AMX) offers a theoretical peak of over 90 TFLOPS for LLM inference, yet practical efficiency often plateaus at 5–40% of its potential. Our analysis reveals that this performance gap stems from a fundamental architectural mismatch: bursty AMX memory accesses frequently saturate Line Fill Buffers (LFB), and naive multi-socket scaling triggers prohibitive synchronization overhead. To bridge this gap, we propose <monospace>InfAMAX</monospace>, an LLM inference framework built upon two key ideas. First, we introduce decoupled tile-pipelining and multi-level adaptive prefetching to maximize data reuse and mask LFB-induced latencies across the cache hierarchy. Second, we employ a NUMA-aware resource management strategy that enforces strict thread affinity and shape-aware partitioning to eliminate cross-socket cache coherence traffic. Experimental results show that <monospace>InfAMAX</monospace> improves end-to-end inference throughput by up to 2× (reducing decoding latency by 2×) over state-of-the-art libraries, providing a scalable and efficient foundation for high-performance LLM serving on modern CPUs.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 1","pages":"198-201"},"PeriodicalIF":1.4,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148236238","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CMM-H PMEM: Performance Characterization of CXL Large Capacity Persistent Memory CMM-H PMEM: CXL大容量持久内存的性能表征

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-02-27 DOI: 10.1109/LCA.2026.3668463

Gongjin Sun;Mehdi Nik;Senthil Murugesapandian;Dongyang Li;Kwan Huen;Xuebin Yao;Caroline Kahn;Amir Beygi;Vitorio Cargnini;Ramdas Kachare

{"title":"CMM-H PMEM: Performance Characterization of CXL Large Capacity Persistent Memory","authors":"Gongjin Sun;Mehdi Nik;Senthil Murugesapandian;Dongyang Li;Kwan Huen;Xuebin Yao;Caroline Kahn;Amir Beygi;Vitorio Cargnini;Ramdas Kachare","doi":"10.1109/LCA.2026.3668463","DOIUrl":"https://doi.org/10.1109/LCA.2026.3668463","url":null,"abstract":"System architectures place memory in close proximity of CPU to achieve high performance. However, this directly and locally attached memory comes with constraints. Compute Express Link (CXL) enables robust memory disaggregation and thereby opens up new possibilities to address some of the direct-attached memory constraints. Samsung CMM-H is a CXL memory module that combines best aspects of DRAM and NAND in a novel architecture to provide a large capacity persistent memory (PMEM) solution. While NVMe SSDs offer capacity, they suffer from higher latency and inevitable write amplification when subjected to the fine-grained writes of data-intensive workloads. This paper presents a comprehensive characterization of the CMM-H PMEM solution. CMM-H device performance profile is compared to NVMe SSD. It achieves up to 100 M random read IOPS, and latencies an order of magnitude lower for small I/Os. The large built-in DRAM cache is the architectural mechanism that enables this performance. The same mechanism also achieves a significant reduction in data written to the backend NAND thereby reducing write amplification factor (WAF). Lower WAF in turn provides benefits such as higher media endurance, lower power consumption, and lower latency. Direct measurement of real-world case studies demonstrates this mechanism reduces WAF by over 200x. This unique combination of high performance and significant write reduction establishes CMM-H as a compelling device solution in the PMEM landscape.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 1","pages":"93-96"},"PeriodicalIF":2.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11415655","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148480444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Heterogeneous Mapping for Analog In-Memory Computing Accelerators: A Unified Workflow 模拟内存计算加速器的异构映射：一个统一的工作流

IF 2 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-06-04 DOI: 10.1109/LCA.2026.3699929

Corey Lammie

引用次数: 0

Accurate and High-Throughput Analog-Digital DNN Acceleration Using Sub-Network Scheduling 使用子网络调度的精确和高吞吐量模拟-数字DNN加速

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-02-27 DOI: 10.1109/LCA.2026.3668766

Jintae Kim;Byoungjun Jeong;Changdae Kim;Narae Ryu;Eunji Pak;Hunjun Lee

引用次数: 0

Characterizing System-Level Trade-Offs of Intel DSA for User-Space IPC Offloading 用户空间IPC卸载Intel DSA的系统级权衡特性

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-04-02 DOI: 10.1109/LCA.2026.3680453

Misun Park;Richi Dubey;Yifan Yuan;Nam Sung Kim;Ada Gavrilovska

{"title":"Characterizing System-Level Trade-Offs of Intel DSA for User-Space IPC Offloading","authors":"Misun Park;Richi Dubey;Yifan Yuan;Nam Sung Kim;Ada Gavrilovska","doi":"10.1109/LCA.2026.3680453","DOIUrl":"https://doi.org/10.1109/LCA.2026.3680453","url":null,"abstract":"Data movement accelerators such as Intel Data Streaming Accelerator (DSA) are increasingly proposed to address the rising cost of intra-node data movement in high-throughput IPC (Inter-Process Communication). Modern data-centric workloads transfer tens to hundreds of megabytes per request, making memory copies a primary bottleneck in latency and CPU utilization. Offloading appears attractive because it can reclaim CPU cycles and reduce cache pressure. However, without careful integration, it can degrade performance instead of improving it. This paper presents a system-level characterization of Intel DSA in user-space IPC and identifies three critical trade-offs: (1) synchronization overhead from polling-based completion, (2) sensitivity to memory state, especially page faults under shared virtual memory, and (3) cache locality and interference from cache injection. Although DSA can reduce latency by tens of microseconds for large transfers, inefficient polling, runtime page faults, or cache contention can eliminate these gains. Our results show that memory offload engines are not drop-in replacements for CPU <monospace>memcpy</monospace>. Their benefits depend on coordinated integration with synchronization, virtual memory, and cache subsystems. We therefore derive hardware-aware design guidelines for adaptive offloading, hybrid polling, pinned or pre-faulted memory management, and contention-aware cache control.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 1","pages":"146-149"},"PeriodicalIF":1.4,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147737043","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CXL-Tracer: Accurate, Full-System Memory Tracing on Commercial Hardware via CXL Memory CXL- tracer：通过CXL内存对商业硬件进行准确的全系统内存跟踪

IF 1.4 3区计算机科学

IEEE Computer Architecture Letters Pub Date : 2026-01-01 Epub Date: 2026-03-12 DOI: 10.1109/LCA.2026.3673181

Srikar Vanavasam;Austin Antony Cruz;Nam Sung Kim

{"title":"CXL-Tracer: Accurate, Full-System Memory Tracing on Commercial Hardware via CXL Memory","authors":"Srikar Vanavasam;Austin Antony Cruz;Nam Sung Kim","doi":"10.1109/LCA.2026.3673181","DOIUrl":"https://doi.org/10.1109/LCA.2026.3673181","url":null,"abstract":"Accurate, full-system physical memory traces are critical for computer architecture research, but existing collection methods based on software instrumentation or simulation suffer from a lack of fidelity, while prior hardware-based solutions are often intrusive or proprietary. We introduce CXL-Tracer, a non-intrusive tracing framework that uses a commodity FPGA as a CXL Type-3 memory device to capture complete, high-fidelity memory traces from a real system. Traces are offloaded over a dedicated PCIe link to a separate collection server, ensuring no performance interference with the system under test. Our evaluation demonstrates that CXL-Tracer captures the full spectrum of system activity, recording up to 70x more memory traffic than binary instrumentation for I/O-heavy workloads. Furthermore, we show that replaying these real-system traces in a cycle-accurate DRAM simulator yields significantly more realistic memory latency results compared to traces generated by a full-system simulator such as gem5. CXL-Tracer provides an accessible and accurate platform for designing and evaluating future memory subsystems.","PeriodicalId":51248,"journal":{"name":"IEEE Computer Architecture Letters","volume":"25 1","pages":"114-117"},"PeriodicalIF":1.4,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11433038","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147557744","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0