Huicheol Shin, Sangki Jeong, Seungjae Baek, Yujae Song
{"title":"水下同时光波信息和电力传输的自适应控制:分层深度强化方法","authors":"Huicheol Shin, Sangki Jeong, Seungjae Baek, Yujae Song","doi":"10.3390/jmse12091647","DOIUrl":null,"url":null,"abstract":"In this work, we consider a point-to-point underwater optical wireless communication scenario where an underwater sensor (US) transmits its sensing data to a remotely operated vehicle (ROV). Before the US transmits its data to the ROV, the ROV performs simultaneous lightwave information and power transfer (SLIPT), delivering both control data and lightwave power to the US. Under the considered scenario, our objective is to maximize energy harvesting at the US while supporting predetermined communication performance between the two nodes. To achieve this objective, we develop a hierarchical deep Q-network (DQN)–deep deterministic policy gradient (DDPG)-based online algorithm. This algorithm involves two reinforcement learning agents: the ROV and US. The role of the ROV agent is to determine an optimal beam-divergence angle that maximizes the received optical signal power at the US while ensuring a seamless optical link. Meanwhile, the US agent, which is influenced by the decision of the ROV agent, is responsible for determining the time-switching and power-splitting ratios to maximize energy harvesting without compromising the required communication performance. Unlike existing studies that do not account for adaptive parameter control in underwater SLIPT, the proposed algorithm’s adaptive nature allows for the dynamic fine-tuning of optimization parameters in response to varying underwater environmental conditions and diverse user requirements.","PeriodicalId":16168,"journal":{"name":"Journal of Marine Science and Engineering","volume":"14 1","pages":""},"PeriodicalIF":2.7000,"publicationDate":"2024-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Adaptive Control for Underwater Simultaneous Lightwave Information and Power Transfer: A Hierarchical Deep-Reinforcement Approach\",\"authors\":\"Huicheol Shin, Sangki Jeong, Seungjae Baek, Yujae Song\",\"doi\":\"10.3390/jmse12091647\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this work, we consider a point-to-point underwater optical wireless communication scenario where an underwater sensor (US) transmits its sensing data to a remotely operated vehicle (ROV). Before the US transmits its data to the ROV, the ROV performs simultaneous lightwave information and power transfer (SLIPT), delivering both control data and lightwave power to the US. Under the considered scenario, our objective is to maximize energy harvesting at the US while supporting predetermined communication performance between the two nodes. To achieve this objective, we develop a hierarchical deep Q-network (DQN)–deep deterministic policy gradient (DDPG)-based online algorithm. This algorithm involves two reinforcement learning agents: the ROV and US. The role of the ROV agent is to determine an optimal beam-divergence angle that maximizes the received optical signal power at the US while ensuring a seamless optical link. Meanwhile, the US agent, which is influenced by the decision of the ROV agent, is responsible for determining the time-switching and power-splitting ratios to maximize energy harvesting without compromising the required communication performance. Unlike existing studies that do not account for adaptive parameter control in underwater SLIPT, the proposed algorithm’s adaptive nature allows for the dynamic fine-tuning of optimization parameters in response to varying underwater environmental conditions and diverse user requirements.\",\"PeriodicalId\":16168,\"journal\":{\"name\":\"Journal of Marine Science and Engineering\",\"volume\":\"14 1\",\"pages\":\"\"},\"PeriodicalIF\":2.7000,\"publicationDate\":\"2024-09-14\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Marine Science and Engineering\",\"FirstCategoryId\":\"89\",\"ListUrlMain\":\"https://doi.org/10.3390/jmse12091647\",\"RegionNum\":3,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, MARINE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Marine Science and Engineering","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.3390/jmse12091647","RegionNum":3,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MARINE","Score":null,"Total":0}
引用次数: 0
摘要
在这项工作中,我们考虑了一种点对点水下光无线通信方案,即水下传感器(US)向遥控潜水器(ROV)传输传感数据。在 US 将数据传输给 ROV 之前,ROV 会执行同步光波信息和功率传输(SLIPT),将控制数据和光波功率同时传输给 US。在所考虑的情况下,我们的目标是最大限度地收集 US 的能量,同时支持两个节点之间预定的通信性能。为实现这一目标,我们开发了一种基于分层深度 Q 网络(DQN)和深度确定性策略梯度(DDPG)的在线算法。该算法涉及两个强化学习代理:ROV 和 US。ROV 代理的作用是确定最佳光束发散角,使 US 接收到的光信号功率最大化,同时确保无缝光链路。同时,US 代理受 ROV 代理决策的影响,负责确定时间切换和功率分配比例,以便在不影响所需通信性能的情况下最大限度地收集能量。现有研究没有考虑到水下 SLIPT 的自适应参数控制,与之不同的是,所提出算法的自适应性质允许对优化参数进行动态微调,以应对不断变化的水下环境条件和不同的用户需求。
Adaptive Control for Underwater Simultaneous Lightwave Information and Power Transfer: A Hierarchical Deep-Reinforcement Approach
In this work, we consider a point-to-point underwater optical wireless communication scenario where an underwater sensor (US) transmits its sensing data to a remotely operated vehicle (ROV). Before the US transmits its data to the ROV, the ROV performs simultaneous lightwave information and power transfer (SLIPT), delivering both control data and lightwave power to the US. Under the considered scenario, our objective is to maximize energy harvesting at the US while supporting predetermined communication performance between the two nodes. To achieve this objective, we develop a hierarchical deep Q-network (DQN)–deep deterministic policy gradient (DDPG)-based online algorithm. This algorithm involves two reinforcement learning agents: the ROV and US. The role of the ROV agent is to determine an optimal beam-divergence angle that maximizes the received optical signal power at the US while ensuring a seamless optical link. Meanwhile, the US agent, which is influenced by the decision of the ROV agent, is responsible for determining the time-switching and power-splitting ratios to maximize energy harvesting without compromising the required communication performance. Unlike existing studies that do not account for adaptive parameter control in underwater SLIPT, the proposed algorithm’s adaptive nature allows for the dynamic fine-tuning of optimization parameters in response to varying underwater environmental conditions and diverse user requirements.
期刊介绍:
Journal of Marine Science and Engineering (JMSE; ISSN 2077-1312) is an international, peer-reviewed open access journal which provides an advanced forum for studies related to marine science and engineering. It publishes reviews, research papers and communications. Our aim is to encourage scientists to publish their experimental and theoretical results in as much detail as possible. There is no restriction on the length of the papers. The full experimental details must be provided so that the results can be reproduced. Electronic files and software regarding the full details of the calculation or experimental procedure, if unable to be published in a normal way, can be deposited as supplementary electronic material.