Energy Harvesting Aware Multi-Hop Routing Policy in Distributed IoT System Based on Multi-Agent Reinforcement Learning

2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC) Pub Date : 2022-01-17 DOI:10.1109/asp-dac52403.2022.9712528

Wen Zhang, Tao Liu, Mimi Xie, Longzhuang Li, Dulal C. Kar, Chen Pan

{"title":"Energy Harvesting Aware Multi-Hop Routing Policy in Distributed IoT System Based on Multi-Agent Reinforcement Learning","authors":"Wen Zhang, Tao Liu, Mimi Xie, Longzhuang Li, Dulal C. Kar, Chen Pan","doi":"10.1109/asp-dac52403.2022.9712528","DOIUrl":null,"url":null,"abstract":"Energy harvesting technologies offer a promising solution to sustainably power an ever-growing number of Internet of Things (IoT) devices. However, due to the weak and transient natures of energy harvesting, IoT devices have to work intermittently rendering conventional routing policies and energy allocation strategies impractical. To this end, this paper, for the very first time, developed a distributed multi-agent reinforcement algorithm known as global actor-critic policy (GAP) to address the problem of routing policy and energy allocation together for the energy harvesting powered IoT system. At the training stage, each IoT device is treated as an agent and one universal model is trained for all agents to save computing resources. At the inference stage, packet delivery rate can be maximized. The experimental results show that the proposed GAP algorithm achieves ~ 1.28× and ~ 1.24× data transmission rate than that of the Q-table and ESDSRAA algorithm, respectively.","PeriodicalId":239260,"journal":{"name":"2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2022-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"3","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/asp-dac52403.2022.9712528","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 3

Abstract

Energy harvesting technologies offer a promising solution to sustainably power an ever-growing number of Internet of Things (IoT) devices. However, due to the weak and transient natures of energy harvesting, IoT devices have to work intermittently rendering conventional routing policies and energy allocation strategies impractical. To this end, this paper, for the very first time, developed a distributed multi-agent reinforcement algorithm known as global actor-critic policy (GAP) to address the problem of routing policy and energy allocation together for the energy harvesting powered IoT system. At the training stage, each IoT device is treated as an agent and one universal model is trained for all agents to save computing resources. At the inference stage, packet delivery rate can be maximized. The experimental results show that the proposed GAP algorithm achieves ~ 1.28× and ~ 1.24× data transmission rate than that of the Q-table and ESDSRAA algorithm, respectively.

查看原文本刊更多论文

基于多智能体强化学习的分布式物联网系统能量收集感知多跳路由策略

能量收集技术为不断增长的物联网(IoT)设备的可持续供电提供了一个有前途的解决方案。然而，由于能量收集的弱和瞬态特性，物联网设备必须间歇性地工作，使得传统的路由策略和能量分配策略不切实际。为此，本文首次开发了一种分布式多智能体强化算法，称为全局行为者批评策略(GAP)，以解决能量收集驱动物联网系统的路由策略和能量分配问题。在训练阶段，将每个物联网设备视为一个智能体，并为所有智能体训练一个通用模型，以节省计算资源。在推理阶段，可以最大化数据包的传输速率。实验结果表明，GAP算法的数据传输率分别是q表和ESDSRAA算法的~ 1.28倍和~ 1.24倍。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC)

自引率

0.00%

发文量