Enabling Resource-Efficient AIoT System With Cross-Level Optimization: A Survey

IF 34.4 1区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

IEEE Communications Surveys and Tutorials Pub Date : 2023-09-27 DOI:10.1109/COMST.2023.3319952

Sicong Liu;Bin Guo;Cheng Fang;Ziqi Wang;Shiyan Luo;Zimu Zhou;Zhiwen Yu

{"title":"Enabling Resource-Efficient AIoT System With Cross-Level Optimization: A Survey","authors":"Sicong Liu;Bin Guo;Cheng Fang;Ziqi Wang;Shiyan Luo;Zimu Zhou;Zhiwen Yu","doi":"10.1109/COMST.2023.3319952","DOIUrl":null,"url":null,"abstract":"The emerging field of artificial intelligence of things (AIoT, AI+IoT) is driven by the widespread use of intelligent infrastructures and the impressive success of deep learning (DL). With the deployment of DL on various intelligent infrastructures featuring rich sensors and weak DL computing capabilities, a diverse range of AIoT applications has become possible. However, DL models are notoriously resource-intensive. Existing research strives to realize near-/realtime inference of AIoT live data and low-cost training using AIoT datasets on resource-scare infrastructures. Accordingly, the accuracy and responsiveness of DL models are bounded by resource availability. To this end, the algorithm-system co-design that jointly optimizes the resource-friendly DL models and model-adaptive system scheduling improves the runtime resource availability and thus pushes the performance boundary set by the standalone level. Unlike previous surveys on resource-friendly DL models or hand-crafted DL compilers/frameworks with partially fine-tuned components, this survey aims to provide a broader optimization space for more free resource-performance tradeoffs. The cross-level optimization landscape involves various granularity, including the DL model, computation graph, operator, memory schedule, and hardware instructor in both on-device and distributed paradigms. Furthermore, due to the dynamic nature of AIoT context, which includes heterogeneous hardware, agnostic sensing data, varying user-specified performance demands, and resource constraints, this survey explores the context-aware inter-/intra-device controllers for automatic cross-level adaptation. Additionally, we identify some potential directions for resource-efficient AIoT systems. By consolidating problems and techniques scattered over diverse levels, we aim to help readers understand their connections and stimulate further discussions.","PeriodicalId":55029,"journal":{"name":"IEEE Communications Surveys and Tutorials","volume":"26 1","pages":"389-427"},"PeriodicalIF":34.4000,"publicationDate":"2023-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Communications Surveys and Tutorials","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10265028/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

The emerging field of artificial intelligence of things (AIoT, AI+IoT) is driven by the widespread use of intelligent infrastructures and the impressive success of deep learning (DL). With the deployment of DL on various intelligent infrastructures featuring rich sensors and weak DL computing capabilities, a diverse range of AIoT applications has become possible. However, DL models are notoriously resource-intensive. Existing research strives to realize near-/realtime inference of AIoT live data and low-cost training using AIoT datasets on resource-scare infrastructures. Accordingly, the accuracy and responsiveness of DL models are bounded by resource availability. To this end, the algorithm-system co-design that jointly optimizes the resource-friendly DL models and model-adaptive system scheduling improves the runtime resource availability and thus pushes the performance boundary set by the standalone level. Unlike previous surveys on resource-friendly DL models or hand-crafted DL compilers/frameworks with partially fine-tuned components, this survey aims to provide a broader optimization space for more free resource-performance tradeoffs. The cross-level optimization landscape involves various granularity, including the DL model, computation graph, operator, memory schedule, and hardware instructor in both on-device and distributed paradigms. Furthermore, due to the dynamic nature of AIoT context, which includes heterogeneous hardware, agnostic sensing data, varying user-specified performance demands, and resource constraints, this survey explores the context-aware inter-/intra-device controllers for automatic cross-level adaptation. Additionally, we identify some potential directions for resource-efficient AIoT systems. By consolidating problems and techniques scattered over diverse levels, we aim to help readers understand their connections and stimulate further discussions.

查看原文本刊更多论文

通过跨层优化实现资源高效的人工智能物联网系统：调查

智能基础设施的广泛应用和深度学习（DL）的巨大成功推动了新兴的人工智能物联网（AIoT，AI+IoT）领域。随着深度学习在具有丰富传感器和较弱深度学习计算能力的各种智能基础设施上的部署，各种 AIoT 应用成为可能。然而，DL 模型是众所周知的资源密集型模型。现有研究致力于在资源紧张的基础设施上使用人工智能物联网数据集实现近实时/实时推理和低成本训练。因此，DL 模型的准确性和响应速度受到资源可用性的限制。为此，联合优化资源友好型 DL 模型和模型自适应系统调度的算法-系统协同设计提高了运行时的资源可用性，从而突破了独立水平设定的性能边界。与以往关于资源友好型 DL 模型或带有部分微调组件的手工制作 DL 编译器/框架的调查不同，本调查旨在为更自由的资源-性能权衡提供更广阔的优化空间。跨级别优化涉及各种粒度，包括 DL 模型、计算图、算子、内存调度，以及设备上和分布式范例中的硬件指导员。此外，由于人工智能物联网上下文的动态性质，包括异构硬件、不可知觉的传感数据、不同的用户指定性能需求和资源限制，本调查探讨了自动跨级适应的上下文感知设备间/设备内控制器。此外，我们还确定了资源节约型人工智能物联网系统的一些潜在方向。通过整合分散在不同层面的问题和技术，我们旨在帮助读者了解它们之间的联系，并激发进一步的讨论。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Communications Surveys and Tutorials COMPUTER SCIENCE, INFORMATION SYSTEMS-TELECOMMUNICATIONS

CiteScore

80.20

自引率

2.50%

发文量

审稿时长

6 months

期刊介绍： IEEE Communications Surveys & Tutorials is an online journal published by the IEEE Communications Society for tutorials and surveys covering all aspects of the communications field. Telecommunications technology is progressing at a rapid pace, and the IEEE Communications Society is committed to providing researchers and other professionals the information and tools to stay abreast. IEEE Communications Surveys and Tutorials focuses on integrating and adding understanding to the existing literature on communications, putting results in context. Whether searching for in-depth information about a familiar area or an introduction into a new area, IEEE Communications Surveys & Tutorials aims to be the premier source of peer-reviewed, comprehensive tutorials and surveys, and pointers to further sources. IEEE Communications Surveys & Tutorials publishes only articles exclusively written for IEEE Communications Surveys & Tutorials and go through a rigorous review process before their publication in the quarterly issues. A tutorial article in the IEEE Communications Surveys & Tutorials should be designed to help the reader to become familiar with and learn something specific about a chosen topic. In contrast, the term survey, as applied here, is defined to mean a survey of the literature. A survey article in IEEE Communications Surveys & Tutorials should provide a comprehensive review of developments in a selected area, covering its development from its inception to its current state and beyond, and illustrating its development through liberal citations from the literature. Both tutorials and surveys should be tutorial in nature and should be written in a style comprehensible to readers outside the specialty of the article.