Journal of Systems Architecture最新文献_第4页

A visual–tactile fusion system for terrain perception under varying illumination conditions 不同光照条件下地形感知的视触觉融合系统

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-01-24 DOI: 10.1016/j.sysarc.2026.103698

Rui Wang , Shichun Yang , Yuyi Chen , Zhuoyang Li , Jiayi Lu , Zexiang Tong , Jianyi Xu , Bin Sun , Xinjie Feng , Yaoguang Cao

{"title":"A visual–tactile fusion system for terrain perception under varying illumination conditions","authors":"Rui Wang , Shichun Yang , Yuyi Chen , Zhuoyang Li , Jiayi Lu , Zexiang Tong , Jianyi Xu , Bin Sun , Xinjie Feng , Yaoguang Cao","doi":"10.1016/j.sysarc.2026.103698","DOIUrl":"10.1016/j.sysarc.2026.103698","url":null,"abstract":"<div><div>Road terrain conditions are vital for ensuring the driving safety of autonomous vehicles (AVs). However, traditional sensors like cameras and LiDARs are sensitive to changes in lighting and weather, posing challenges for real-time road condition perception. In this paper, we propose an illumination-aware visual–tactile fusion system (IVTF) for terrain perception, integrating visual and tactile data while optimizing the fusion process based on illumination characteristics. The system employs a camera and an intelligent tire to capture visual and tactile data across various lighting conditions and vehicle speeds. Additionally, we also design a visual–tactile fusion module that dynamically adjusts the weights of different modalities according to illumination features. Comparative results with single-modality perception methods demonstrate the superior ability of visual–tactile fusion to accurately perceive road terrains under diverse lighting conditions. This approach significantly advances the robustness and reliability of terrain perception in AVs, contributing to enhanced driving safety.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103698"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146192071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Formal design of safety-critical systems with MARS MARS安全关键系统的正式设计

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-01-30 DOI: 10.1016/j.sysarc.2026.103711

Yihao Yin , Hao Wu , Wan Liu , Shuling Wang , Xiong Xu , Wang Lin , Fanjiang Xu , Naijun Zhan

{"title":"Formal design of safety-critical systems with MARS","authors":"Yihao Yin , Hao Wu , Wan Liu , Shuling Wang , Xiong Xu , Wang Lin , Fanjiang Xu , Naijun Zhan","doi":"10.1016/j.sysarc.2026.103711","DOIUrl":"10.1016/j.sysarc.2026.103711","url":null,"abstract":"<div><div>MARS is a toolchain, supporting model-based design of cyber–physical systems (CPS), which integrates informal and formal design. With MARS, a system under development can be graphically modeled by the combination of AADL and Simulink/Stateflow, then the analysis of the graphical model can be conducted via simulation. Furthermore, the graphical model can be automatically transformed to Hybrid Communicating Sequential Processes (HCSP) for formal verification with HHLProver. Within HHLProver, both interactive and automated theorem proving of HCSP processes are supported, with the help of an invariant generator, which is used for automated synthesis of differential invariants of ordinary differential equations based on both numerical methods and neural networks. Finally, ANSI-C code or SystemC code can be generated from the verified HCSP formal model with the guarantee of correctness. As a case study of CPS, this paper applies the MARS toolchain to the design of an intelligent temperature control system, including its modeling, simulation, verification and code generation. This case study demonstrates the advantages of the design of CPS with MARS, including the integration of modeling, simulation, verification and code generation; the integration of informal and formal design, thus providing balance between efficiency and rigidity and enabling a <strong>correct-by-construction</strong> design flow.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103711"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146192076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Designing resilient IoT and Edge Computing with federated tinyML 使用federated tinyML设计弹性物联网和边缘计算

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-02-06 DOI: 10.1016/j.sysarc.2026.103709

Paul Laiu, Mingyan Li, Jeffrey A. Nichols, Mike Huettel, Isaac Sikkema, Mahim Mathur

{"title":"Designing resilient IoT and Edge Computing with federated tinyML","authors":"Paul Laiu, Mingyan Li, Jeffrey A. Nichols, Mike Huettel, Isaac Sikkema, Mahim Mathur","doi":"10.1016/j.sysarc.2026.103709","DOIUrl":"10.1016/j.sysarc.2026.103709","url":null,"abstract":"<div><div>The rapid growth of the Internet of Things (IoT) and Edge Computing (EC) has brought significant conveniences to modern society but has also greatly expanded the cyber attack surfaces, particularly as these technologies are being increasingly integrated into critical systems such as power grids, healthcare, and smart homes. To improve IoT/EC’s cybersecurity posture, we leveraged Artificial Intelligence (AI) and Machine Learning (ML) by employing tinyML to monitor voluminous IoT data for cyber threats while addressing devices’ resource constraints, and utilizing Federated Learning (FL) to share local detection knowledge across the system while preserving privacy. Building on our three-layer architecture combining tinyML and FL to enhance autonomous cyber attack detection, this paper demonstrated that the architecture improves detection accuracy, reduces resource consumption, and enables lightweight, secure IoT device monitoring. These results were validated using the public N-BaIoT dataset as well as real IoT network traffic data collected under multiple attack scenarios from our testbeds. Additionally, we introduced an enhanced FL methodology with a novel preprocessing stage, including federated feature selection and global preprocessor construction, to address IoT/EC data heterogeneity. We developed a physical IoT testbed for attack simulations and data collection, implemented a tinyML-powered detector for realistic model validation, and also built a virtual testbed for scalable evaluations of FL models across diverse network environments.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103709"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146191610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CADM: Content addressable commodity off-the-shelf DRAM-based genome classifier CADM：内容可寻址商品现成的基于dram的基因组分类器

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-02-04 DOI: 10.1016/j.sysarc.2026.103710

Esteban Garzón , Alexander Fish , Leonid Yavits

{"title":"CADM: Content addressable commodity off-the-shelf DRAM-based genome classifier","authors":"Esteban Garzón , Alexander Fish , Leonid Yavits","doi":"10.1016/j.sysarc.2026.103710","DOIUrl":"10.1016/j.sysarc.2026.103710","url":null,"abstract":"<div><div>Processing using memory (PuM) leverages analog properties of memory infrastructure to implement logic and arithmetic operations. Commodity Off-The-Shelf (COTS) DRAM is particularly attractive for PuM because it requires no device modification, thereby preserving the ubiquity, availability, and cost advantages of modern DRAM while enabling massive column-level parallelism. We propose CADM (Content-Addressable DRAM), that enables exact and approximate (similarity) search in- and using- unmodified COTS DRAM. CADM targets genome classification, which is one of the most important applications in bioinformatics. Specifically, rapid and accurate detection of bacterial pathogens is critical for effective clinical decision-making, particularly in life-threatening conditions such as sepsis, where early identification of the causative agent significantly improves patient outcomes. We implement CADM in commercial DDR4 and show that it can achieve up to 185<span><math><mo>×</mo></math></span> higher throughput and 73<span><math><mo>×</mo></math></span> energy savings compared to CPU-run state-of-the-art classifier Kraken2. Using approximate search, CADM can achieve 9<span><math><mo>×</mo></math></span> higher <span><math><msub><mrow><mi>F</mi></mrow><mrow><mn>1</mn></mrow></msub></math></span> score when matching relatively short (<span><math><mo><</mo></math></span>32 DNA bases) ambiguous and erroneous <span><math><mi>k</mi></math></span>-mers.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103710"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146191612","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CXL shared coherent memory simulation and cross-host synchronization mechanisms design for data sharing CXL共享一致内存模拟和跨主机同步机制设计，用于数据共享

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-02-03 DOI: 10.1016/j.sysarc.2026.103723

Ting Wu , Qingyuan Song , Xihong Huang , Linbo Long , Zhulin Ma , Weichen Liu

{"title":"CXL shared coherent memory simulation and cross-host synchronization mechanisms design for data sharing","authors":"Ting Wu , Qingyuan Song , Xihong Huang , Linbo Long , Zhulin Ma , Weichen Liu","doi":"10.1016/j.sysarc.2026.103723","DOIUrl":"10.1016/j.sysarc.2026.103723","url":null,"abstract":"<div><div>Compute Express Link (CXL) is an emerging interconnect standard for connecting CPU, memory, and devices. A key advancement in CXL 3.x is the introduction of shared coherent memory among multiple hosts, enabling low-latency data sharing and synchronization through load-store operations. However, CXL 3.x remains at an early stage and currently lacks commodity products. Existing simulation studies primarily focus on memory expansion devices without supporting shared coherent memory among multiple hosts. In this paper, we simulate the CXL shared coherent memory with cross-host synchronization mechanisms for data sharing among multiple hosts. The proposed simulation models the functional behavior of shared coherent memory by implementing shared memory among co-resident virtual machines, abstracting low-level hardware coherence transactions. Based on the simulated shared memory, two synchronization mechanisms, including shared spin_locks and semaphores, are designed to support concurrent access across hosts in both kernel and user space, respectively. Two use cases, data sharing and file sharing, are implemented to demonstrate the efficiency of inter-host communication via the simulated shared memory. Experimental results show that data sharing via the simulated shared memory is 202.8 and 41.5 times faster than LAN and VirtIO, respectively. The average file-sharing throughput of SPMFS-Remote is 108.1 and 22.4 times higher than that of Samba-LAN and Samba-VirtIO, respectively. Moreover, SPMFS-Remote achieves 82.8% of the average throughput of SPMFS-Local. While the simulation abstracts certain hardware-level coherence details and uses remote NUMA memory as the shared device, it captures the function and approximate latency of CXL shared coherent memory.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103723"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146191611","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Distilling knowledge for low-energy AIoT 为低能量AIoT提炼知识

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-02-02 DOI: 10.1016/j.sysarc.2026.103692

Franca Rocco di Torrepadula, Vincenzo Maisto, Alessandro Cilardo, Nicola Mazzocca

{"title":"Distilling knowledge for low-energy AIoT","authors":"Franca Rocco di Torrepadula, Vincenzo Maisto, Alessandro Cilardo, Nicola Mazzocca","doi":"10.1016/j.sysarc.2026.103692","DOIUrl":"10.1016/j.sysarc.2026.103692","url":null,"abstract":"<div><div>The Artificial Intelligence of Things (AIoT) empowers IoT devices to leverage the advantages of AI near data-sources, reducing data movement, latency, and mitigating privacy issues. However, AI workloads are notoriously energy-intensive, posing significant challenges for energy-constrained IoT devices. Since such devices are often deployed in thousands of instances, even minor inefficiencies can significantly increase carbon emissions and energy consumptions. Model compression techniques have been employed to enable AI inference in resource-constrained environments. For example, Knowledge Distillation (KD) is an elaborate approach targeting low-footprint and high-accuracy models, although introducing further complexity during training due to inefficient grid searches of additional hyperparameters. The emerging wave of AIoT, however, calls for prioritizing energy-awareness both in inference and training. To address this shortcoming, this work proposes a three-stage design workflow for low-energy AIoT applications, driven primarily by an input energy budget characterizing the target IoT scenario. Given a specific CNN architecture and IoT platform, our workflow identifies the most effective student under the imposed energy constrained and derives an efficient configuration of the KD hyperparameters that maximizes student accuracy, while avoiding inefficient and expensive grid-search. Hence, this approach enable energy-efficient CNN inference while substantially reducing overall training costs. We validate our workflow with a systematic experimental campaign using ResNets and DenseNets on CIFAR-10, CIFAR-100,and Tiny Imagenet datasets, on an AMD Xilinx Zynq Ultrascale+ ZCU102 MPSoC. Our proposal maintains high accuracy while lowering energy consumption by up to 80%, highlighting the potential of our flow for real-world AIoT applications.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103692"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146191613","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Formal semantics for hierarchical Simulink diagrams in Isabelle/HOL Isabelle/HOL中分层Simulink图的形式语义

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-02-04 DOI: 10.1016/j.sysarc.2026.103724

Yuzhen Qi , Shuling Wang , Xing Li , Bohua Zhan , Naijun Zhan

{"title":"Formal semantics for hierarchical Simulink diagrams in Isabelle/HOL","authors":"Yuzhen Qi , Shuling Wang , Xing Li , Bohua Zhan , Naijun Zhan","doi":"10.1016/j.sysarc.2026.103724","DOIUrl":"10.1016/j.sysarc.2026.103724","url":null,"abstract":"<div><div>Simulink is widely used in the design of safety-critical embedded systems, including avionics and automotive applications. While it offers simulation for model validation, formal verification remains essential to rigorously ensure system correctness. Existing approaches often translate Simulink diagrams into third-party formal models, however, the lack of a rigorously defined semantics for Simulink can lead to inconsistencies between the original diagrams and their translated formal counterparts. In this paper, we present a formal semantic foundation for a core subset of Simulink by defining both denotational and operational semantics. The denotational semantics offers a mathematical interpretation of the diagram’s input–output behavior, faithfully capturing its hierarchical structure. In contrast, the operational semantics specifies the concrete execution of Simulink diagrams, resolving block execution order, solving continuous dynamics, and coordinating hybrid discrete-continuous interactions. Both semantics have been fully formalized in Isabelle/HOL, and we have established their consistency by proving the existence and uniqueness of the timed state trajectories defined by the denotational semantics. Furthermore, to facilitate application, we developed a translator that automatically converts Simulink graphical diagrams into their Isabelle representation. Our formal semantics supports the rigorous analysis of Simulink diagram properties, as demonstrated through a PID control example. The semantics also establishes a foundation for validating simulation results and ensuring consistency between Simulink models and other formal models, thus enabling sound verification.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103724"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146192095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

RUnQuant: High-resolution weight quantization via unanchored weight decomposition in column-wise granularity for CIM accelerators RUnQuant：通过对CIM加速器按列粒度进行无锚定权重分解，实现高分辨率权重量化

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-05-01 Epub Date: 2026-02-04 DOI: 10.1016/j.sysarc.2026.103703

Jiyoon Kim , Kang Eun Jeon , Yulhwa Kim , Jong Hwan Ko

{"title":"RUnQuant: High-resolution weight quantization via unanchored weight decomposition in column-wise granularity for CIM accelerators","authors":"Jiyoon Kim , Kang Eun Jeon , Yulhwa Kim , Jong Hwan Ko","doi":"10.1016/j.sysarc.2026.103703","DOIUrl":"10.1016/j.sysarc.2026.103703","url":null,"abstract":"<div><div>Compute-in-memory (CIM) enables efficient deep neural network (DNN) implementation, but suffers from area and energy overhead from analog-to-digital converters (ADCs) and crossbar arrays of limited cell precisions. Low-precision ADCs mitigate these overheads, but introduce partial-sum quantization errors, degrading accuracy. Cell precision limitations impose low-bit weight constraints that further challenge network accuracy. Although prior work has focused on fine-grained partial-sum quantization to reduce ADC resolution, weight granularity remains underexplored, which is crucial for achieving high accuracy. Utilizing low-precision cells, weight decomposition is commonly employed to represent signed weights, but conventional zero-anchored schemes restrict resolution. We address these issues by integrating unanchored weight decomposition with column-wise alignment of weight and partial-sum quantization. Our method improves accuracy while maintaining dequantization overhead, simplifies training by removing two-stage processes, and maximizes representable weight quantization levels. We also introduce an open-source CIM-oriented convolution framework that manages fine-grained weights and partial-sums through novel tiling and group convolution. Experimental results demonstrate accuracy improvements up to 4.05% in comparison with state-of-the-art methods, highlighting the effectiveness of our quantization scheme in enhancing accuracy while maintaining hardware efficiency in CIM accelerators. Our code is available at <span><span>https://github.com/jiyoonkm/ColumnQuant</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"174 ","pages":"Article 103703"},"PeriodicalIF":4.1,"publicationDate":"2026-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146192074","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Precision boundary modeling for area-efficient Block Floating Point accumulation 区域高效块浮点积累的精确边界建模

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-04-01 Epub Date: 2026-01-13 DOI: 10.1016/j.sysarc.2026.103704

Jun He , Jing Feng , Xin Ju , Yasong Cao , Zhongdi Luo , Jianchao Yang , Jingkui Yang , Gang Li , Jian Cheng , Dong Chen , Mei Wen

{"title":"Precision boundary modeling for area-efficient Block Floating Point accumulation","authors":"Jun He , Jing Feng , Xin Ju , Yasong Cao , Zhongdi Luo , Jianchao Yang , Jingkui Yang , Gang Li , Jian Cheng , Dong Chen , Mei Wen","doi":"10.1016/j.sysarc.2026.103704","DOIUrl":"10.1016/j.sysarc.2026.103704","url":null,"abstract":"<div><div>Block Floating Point (BFP) is extensively employed for the low-precision quantization of deep network weights and activations to attain advantages in both hardware efficiency and performance. Nevertheless, when the precision of weights and activations is diminished to below 8 bits, the required high-precision floating-point accumulation becomes a dominant hardware bottleneck in the BFP processing element (PE). To address this challenge, we introduce a framework based on the Frobenius norm Retention Ratio (FnRR) to explore the precision boundaries for BFP accumulation, and extend it to a hierarchical chunk-based accumulation scheme. Comprehensive experiments across representative CNN and LLM models demonstrate that our predicted precision boundaries maintain performance closely matching FP32 baselines, while further precision reduction leads to substantial accuracy degradation, validating the effectiveness of our boundary determination. Guided by this analysis, we present a corresponding hardware for BFP computation. This design achieves 13.7%–25.2% improvements in area and power efficiency compared with FP32 accumulation under identical quantization settings, and delivers up to <span><math><mrow><mn>10</mn><mo>.</mo><mn>3</mn><mo>×</mo></mrow></math></span> area and <span><math><mrow><mn>11</mn><mo>.</mo><mn>0</mn><mo>×</mo></mrow></math></span> power reductions relative to conventional BFP implementations.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"173 ","pages":"Article 103704"},"PeriodicalIF":4.1,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145981643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Peak-memory-aware partitioning and scheduling for multi-tenant DNN model inference 多租户DNN模型推理的峰值内存感知分区和调度

IF 4.1 2区计算机科学

Journal of Systems Architecture Pub Date : 2026-04-01 Epub Date: 2026-01-09 DOI: 10.1016/j.sysarc.2026.103696

Jaeho Lee, Ju Min Lee, Haeeun Jeong, Hyunho Kwon, Youngsok Kim, Yongjun Park, Hanjun Kim

{"title":"Peak-memory-aware partitioning and scheduling for multi-tenant DNN model inference","authors":"Jaeho Lee, Ju Min Lee, Haeeun Jeong, Hyunho Kwon, Youngsok Kim, Yongjun Park, Hanjun Kim","doi":"10.1016/j.sysarc.2026.103696","DOIUrl":"10.1016/j.sysarc.2026.103696","url":null,"abstract":"<div><div>As Deep Neural Networks (DNNs) are widely used in various applications, multiple DNN inference models start to run on a single GPU. The simultaneous execution of multiple DNN models can overwhelm the GPU memory with increasing model size, leading to unexpected out-of-memory (OOM) errors. To avoid the OOM errors, existing systems attempt to schedule models either in model-level granularity or layer-level granularity. However, the model-level scheduling schemes inefficiently utilize memory spaces because they preallocate memory based on the model’s peak memory demand, and the layer-level scheduling schemes suffer from high scheduling overhead due to too fine-grained scheduling units. This work proposes a new peak-memory-aware DNN model partitioning compiler and scheduler, called Quilt. The Quilt compiler partitions a DNN model into multiple tasks based on their peak memory usage, and the Quilt scheduler orchestrates the tasks of multiple models without the OOM errors. Additionally, the compiler generates a memory pool for tensors shared between partitioned tasks, reducing CPU–GPU communication overhead when consecutively executing the tasks. Compared to the model-level and layer-level scheduling schemes, Quilt successfully reduces 25.4% and 37.7% of overall latency, respectively, while preventing the OOM errors. Moreover, Quilt achieves up to 10.8% faster overall inference latency than the state-of-the-art Triton inference server for 6 DNN models.</div></div>","PeriodicalId":50027,"journal":{"name":"Journal of Systems Architecture","volume":"173 ","pages":"Article 103696"},"PeriodicalIF":4.1,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145981644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0