Joint Throughput Maximization and Energy Management for Ultralow Power Ambient Backscatter Communication in WBANs by Distributed Deep Reinforcement Learning

IF 4.3 2区 综合性期刊 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC
Youze Yang;Sen Yan
{"title":"Joint Throughput Maximization and Energy Management for Ultralow Power Ambient Backscatter Communication in WBANs by Distributed Deep Reinforcement Learning","authors":"Youze Yang;Sen Yan","doi":"10.1109/JSEN.2024.3487354","DOIUrl":null,"url":null,"abstract":"In this article, we present an ultralow power ambient backscatter communications (AmBC) framework for wireless body area networks (WBANs). In these WBANs, the AmBC sensor nodes are energy harvesting (EH) powered and are able to transmit their own collected physiological information by backscattering the signals emitted by the Wi-Fi access point (AP). To achieve maximum throughput performance with sustainable operation for these ultralow power AmBC sensor nodes in WBANs, we propose a three-phase AmBC transmission model and formulate a joint throughput maximization and energy management (JTMEM) problem. The first-order Markov process channel model and random data collection are also developed in the system model to represent practical health monitoring application environments. Since the full real-time channel state information (CSI) is difficult to obtain at the beginning of each time slot, the optimal working mode selection (WMS) policy is not available for this problem. To overcome this issue, we propose a deep reinforcement learning (DRL)-based algorithm, which can use historical CSI to learn the potential channel correlation to infer the channel changing and decide the appropriate working mode for each AmBC sensor node. Moreover, we establish a distributed DRL structure to overcome huge action space issues and make the proposed algorithm flexible and scalable. Finally, extensive numerical simulation results demonstrate that our proposed algorithm can approach the average throughput performance and rewards of the ergodic policy, which has full real-time CSI in different scenarios, and represents favorable energy management performance after convergence.","PeriodicalId":447,"journal":{"name":"IEEE Sensors Journal","volume":"24 24","pages":"42484-42499"},"PeriodicalIF":4.3000,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Sensors Journal","FirstCategoryId":"103","ListUrlMain":"https://ieeexplore.ieee.org/document/10742315/","RegionNum":2,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

Abstract

In this article, we present an ultralow power ambient backscatter communications (AmBC) framework for wireless body area networks (WBANs). In these WBANs, the AmBC sensor nodes are energy harvesting (EH) powered and are able to transmit their own collected physiological information by backscattering the signals emitted by the Wi-Fi access point (AP). To achieve maximum throughput performance with sustainable operation for these ultralow power AmBC sensor nodes in WBANs, we propose a three-phase AmBC transmission model and formulate a joint throughput maximization and energy management (JTMEM) problem. The first-order Markov process channel model and random data collection are also developed in the system model to represent practical health monitoring application environments. Since the full real-time channel state information (CSI) is difficult to obtain at the beginning of each time slot, the optimal working mode selection (WMS) policy is not available for this problem. To overcome this issue, we propose a deep reinforcement learning (DRL)-based algorithm, which can use historical CSI to learn the potential channel correlation to infer the channel changing and decide the appropriate working mode for each AmBC sensor node. Moreover, we establish a distributed DRL structure to overcome huge action space issues and make the proposed algorithm flexible and scalable. Finally, extensive numerical simulation results demonstrate that our proposed algorithm can approach the average throughput performance and rewards of the ergodic policy, which has full real-time CSI in different scenarios, and represents favorable energy management performance after convergence.
基于分布式深度强化学习的wban超低功耗环境后向散射通信联合吞吐量最大化与能量管理
在本文中,我们提出了一种用于无线体域网络(wban)的超低功耗环境反向散射通信(AmBC)框架。在这些wban中,AmBC传感器节点是能量收集(EH)供电的,并且能够通过反向散射Wi-Fi接入点(AP)发出的信号来传输它们自己收集的生理信息。为了在wban中实现这些超低功耗AmBC传感器节点的最大吞吐量性能和可持续运行,我们提出了一种三相AmBC传输模型,并提出了吞吐量最大化和能量管理(JTMEM)联合问题。在系统模型中建立了一阶马尔可夫过程通道模型和随机数据采集模型,以表示实际的健康监测应用环境。由于在每个时隙开始时难以获得完整的实时信道状态信息,因此无法获得最优工作模式选择(WMS)策略。为了克服这一问题,我们提出了一种基于深度强化学习(DRL)的算法,该算法可以使用历史CSI来学习潜在的通道相关性,从而推断通道的变化,并确定每个AmBC传感器节点的合适工作模式。此外,我们建立了分布式DRL结构,克服了巨大的动作空间问题,使所提出的算法具有灵活性和可扩展性。最后,大量的数值仿真结果表明,本文提出的算法可以接近遍历策略的平均吞吐量性能和奖励,在不同场景下具有完全的实时CSI,收敛后具有良好的能量管理性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
IEEE Sensors Journal
IEEE Sensors Journal 工程技术-工程:电子与电气
CiteScore
7.70
自引率
14.00%
发文量
2058
审稿时长
5.2 months
期刊介绍: The fields of interest of the IEEE Sensors Journal are the theory, design , fabrication, manufacturing and applications of devices for sensing and transducing physical, chemical and biological phenomena, with emphasis on the electronics and physics aspect of sensors and integrated sensors-actuators. IEEE Sensors Journal deals with the following: -Sensor Phenomenology, Modelling, and Evaluation -Sensor Materials, Processing, and Fabrication -Chemical and Gas Sensors -Microfluidics and Biosensors -Optical Sensors -Physical Sensors: Temperature, Mechanical, Magnetic, and others -Acoustic and Ultrasonic Sensors -Sensor Packaging -Sensor Networks -Sensor Applications -Sensor Systems: Signals, Processing, and Interfaces -Actuators and Sensor Power Systems -Sensor Signal Processing for high precision and stability (amplification, filtering, linearization, modulation/demodulation) and under harsh conditions (EMC, radiation, humidity, temperature); energy consumption/harvesting -Sensor Data Processing (soft computing with sensor data, e.g., pattern recognition, machine learning, evolutionary computation; sensor data fusion, processing of wave e.g., electromagnetic and acoustic; and non-wave, e.g., chemical, gravity, particle, thermal, radiative and non-radiative sensor data, detection, estimation and classification based on sensor data) -Sensors in Industrial Practice
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信