加速ElasticTrainer与弹性层选择

IF 3.6 3区计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

IEEE Access Pub Date : 2025-07-23 DOI:10.1109/ACCESS.2025.3591772

Sangho Ha;Hyungshin Kim

{"title":"加速ElasticTrainer与弹性层选择","authors":"Sangho Ha;Hyungshin Kim","doi":"10.1109/ACCESS.2025.3591772","DOIUrl":null,"url":null,"abstract":"On-device training consumes a lot of training time due to the limited computing resources of edge devices. ElasticTrainer reduces training time by selecting important tensors from the model and then training them. However, selection at the tensor level leads to reduced arithmetic intensity, failing to fully utilize GPU resources. In this paper, we propose a layer-level selection method considering arithmetic intensity to further reduce training time. Compared to the existing tensor selection method, ElasticTrainer, our method reduces training time by up to 25% with less than 0.1% accuracy loss.","PeriodicalId":13079,"journal":{"name":"IEEE Access","volume":"13 ","pages":"133025-133034"},"PeriodicalIF":3.6000,"publicationDate":"2025-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11091304","citationCount":"0","resultStr":"{\"title\":\"Accelerated ElasticTrainer With Elastic Layer Selection\",\"authors\":\"Sangho Ha;Hyungshin Kim\",\"doi\":\"10.1109/ACCESS.2025.3591772\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"On-device training consumes a lot of training time due to the limited computing resources of edge devices. ElasticTrainer reduces training time by selecting important tensors from the model and then training them. However, selection at the tensor level leads to reduced arithmetic intensity, failing to fully utilize GPU resources. In this paper, we propose a layer-level selection method considering arithmetic intensity to further reduce training time. Compared to the existing tensor selection method, ElasticTrainer, our method reduces training time by up to 25% with less than 0.1% accuracy loss.\",\"PeriodicalId\":13079,\"journal\":{\"name\":\"IEEE Access\",\"volume\":\"13 \",\"pages\":\"133025-133034\"},\"PeriodicalIF\":3.6000,\"publicationDate\":\"2025-07-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11091304\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Access\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/11091304/\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Access","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/11091304/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

摘要

由于边缘设备的计算资源有限，设备上训练消耗了大量的训练时间。ElasticTrainer通过从模型中选择重要的张量然后训练它们来减少训练时间。然而，在张量级别上的选择导致算法强度降低，无法充分利用GPU资源。为了进一步减少训练时间，本文提出了一种考虑算法强度的分层选择方法。与现有的张量选择方法ElasticTrainer相比，我们的方法将训练时间减少了25%，精度损失小于0.1%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Accelerated ElasticTrainer With Elastic Layer Selection

On-device training consumes a lot of training time due to the limited computing resources of edge devices. ElasticTrainer reduces training time by selecting important tensors from the model and then training them. However, selection at the tensor level leads to reduced arithmetic intensity, failing to fully utilize GPU resources. In this paper, we propose a layer-level selection method considering arithmetic intensity to further reduce training time. Compared to the existing tensor selection method, ElasticTrainer, our method reduces training time by up to 25% with less than 0.1% accuracy loss.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

IEEE Access COMPUTER SCIENCE, INFORMATION SYSTEMSENGIN-ENGINEERING, ELECTRICAL & ELECTRONIC

CiteScore

9.80

自引率

7.70%

发文量

6673

审稿时长

6 weeks

期刊介绍： IEEE Access® is a multidisciplinary, open access (OA), applications-oriented, all-electronic archival journal that continuously presents the results of original research or development across all of IEEE''s fields of interest. IEEE Access will publish articles that are of high interest to readers, original, technically correct, and clearly presented. Supported by author publication charges (APC), its hallmarks are a rapid peer review and publication process with open access to all readers. Unlike IEEE''s traditional Transactions or Journals, reviews are "binary", in that reviewers will either Accept or Reject an article in the form it is submitted in order to achieve rapid turnaround. Especially encouraged are submissions on: Multidisciplinary topics, or applications-oriented articles and negative results that do not fit within the scope of IEEE''s traditional journals. Practical articles discussing new experiments or measurement techniques, interesting solutions to engineering. Development of new or improved fabrication or manufacturing techniques. Reviews or survey articles of new or evolving fields oriented to assist others in understanding the new area.