Energy consumption minimization for robotic systems in intelligent factories with the assistance of STAR-RIS: A reinforcement learning approach

IF 4.6 2区 计算机科学 Q1 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE
Nguyen Thi Thanh Van , Hoang Le Hung , Nguyen Cong Luong , Huy T. Nguyen , Nguyen Tien Hoa , Ngo Manh Duy , Ngo Manh Tien
{"title":"Energy consumption minimization for robotic systems in intelligent factories with the assistance of STAR-RIS: A reinforcement learning approach","authors":"Nguyen Thi Thanh Van ,&nbsp;Hoang Le Hung ,&nbsp;Nguyen Cong Luong ,&nbsp;Huy T. Nguyen ,&nbsp;Nguyen Tien Hoa ,&nbsp;Ngo Manh Duy ,&nbsp;Ngo Manh Tien","doi":"10.1016/j.comnet.2025.111441","DOIUrl":null,"url":null,"abstract":"<div><div>The integration of wireless communication to factory robots to form connected robots has become an encouraging technology for intelligent factories due to their cost-effectiveness and high flexibility. However, due to obstacles, guaranteeing stable communication links between an access point (AP) and a robot is challenging. To address this, we propose to deploy a Simultaneously Transmitting and Reflecting Reconfigurable Intelligent Surface (STAR-RIS) which allows incident signals to be reflected and transmitted on both sides of the RIS surface, enabling comprehensive <span><math><mrow><mn>36</mn><msup><mrow><mn>0</mn></mrow><mrow><mn>0</mn></mrow></msup></mrow></math></span> coverage to improve data rate between the AP and the robot. Then, we define an optimization problem that seeks to minimize the system’s overall energy consumption including the AP communication energy and the robot energy consumption while satisfying the requirements of the robot’s safety distance, maximum movement duration, and data rate threshold. The problem involves optimizing the robot’s trajectory, the transmitted power of the AP, the phase shifts, and the transmitting/reflecting coefficient of the STAR-RIS. The optimization problem is nonconvex due to the nonconvex objective function, the nonconvex obstacle-robot distance constraint, the phase shifts and transmitting/reflecting coefficient of STAR-RIS, and the data rate requirement constraint. In addition, there are many dynamic factors in the working environment, such as the robot’s location, the channel between the AP and robot. Therefore, we first approximate the original optimization problem by a Markov Decision Process (MDP) model, then propose to use a DRL algorithm based on Proximal Policy Optimization (PPO) which uses an actor and critic network policy reinforcement to solve the optimization problem. We conducted extensive simulations under various scenarios, and the results show that the case with the use of the STAR-RIS significantly reduces the travel distance of the robot and the system energy consumption compared with the cases with A2C based algorithm, the conventional RIS or without RIS.</div></div>","PeriodicalId":50637,"journal":{"name":"Computer Networks","volume":"270 ","pages":"Article 111441"},"PeriodicalIF":4.6000,"publicationDate":"2025-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1389128625004086","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}
引用次数: 0

Abstract

The integration of wireless communication to factory robots to form connected robots has become an encouraging technology for intelligent factories due to their cost-effectiveness and high flexibility. However, due to obstacles, guaranteeing stable communication links between an access point (AP) and a robot is challenging. To address this, we propose to deploy a Simultaneously Transmitting and Reflecting Reconfigurable Intelligent Surface (STAR-RIS) which allows incident signals to be reflected and transmitted on both sides of the RIS surface, enabling comprehensive 3600 coverage to improve data rate between the AP and the robot. Then, we define an optimization problem that seeks to minimize the system’s overall energy consumption including the AP communication energy and the robot energy consumption while satisfying the requirements of the robot’s safety distance, maximum movement duration, and data rate threshold. The problem involves optimizing the robot’s trajectory, the transmitted power of the AP, the phase shifts, and the transmitting/reflecting coefficient of the STAR-RIS. The optimization problem is nonconvex due to the nonconvex objective function, the nonconvex obstacle-robot distance constraint, the phase shifts and transmitting/reflecting coefficient of STAR-RIS, and the data rate requirement constraint. In addition, there are many dynamic factors in the working environment, such as the robot’s location, the channel between the AP and robot. Therefore, we first approximate the original optimization problem by a Markov Decision Process (MDP) model, then propose to use a DRL algorithm based on Proximal Policy Optimization (PPO) which uses an actor and critic network policy reinforcement to solve the optimization problem. We conducted extensive simulations under various scenarios, and the results show that the case with the use of the STAR-RIS significantly reduces the travel distance of the robot and the system energy consumption compared with the cases with A2C based algorithm, the conventional RIS or without RIS.
基于STAR-RIS的智能工厂机器人系统能耗最小化:一种强化学习方法
将无线通信集成到工厂机器人中,形成联网机器人,由于其成本效益和高灵活性,已成为智能工厂的一种令人鼓舞的技术。然而,由于障碍,保证接入点(AP)和机器人之间的稳定通信链路是一项挑战。为了解决这个问题,我们建议部署一个同步传输和反射可重构智能表面(STAR-RIS),它允许事件信号在RIS表面的两侧反射和传输,实现全面的3600覆盖,以提高AP和机器人之间的数据速率。然后,我们定义了一个优化问题,在满足机器人的安全距离、最大运动持续时间和数据速率阈值要求的同时,寻求最小化系统的总体能耗,包括AP通信能耗和机器人能耗。该问题涉及优化机器人的轨迹、AP的发射功率、相移和STAR-RIS的发射/反射系数。由于非凸目标函数、非凸障碍物-机器人距离约束、STAR-RIS的相移和传输/反射系数以及数据速率要求约束,优化问题是非凸的。此外,工作环境中还有许多动态因素,如机器人的位置、AP与机器人之间的通道等。因此,我们首先用马尔可夫决策过程(MDP)模型近似原始优化问题,然后提出了基于近邻策略优化(PPO)的DRL算法,该算法使用参与者和评论家网络策略强化来解决优化问题。我们在各种场景下进行了大量的仿真,结果表明,与基于A2C算法、常规RIS或不使用RIS的情况相比,使用STAR-RIS的情况下,机器人的行驶距离和系统能耗明显减少。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Computer Networks
Computer Networks 工程技术-电信学
CiteScore
10.80
自引率
3.60%
发文量
434
审稿时长
8.6 months
期刊介绍: Computer Networks is an international, archival journal providing a publication vehicle for complete coverage of all topics of interest to those involved in the computer communications networking area. The audience includes researchers, managers and operators of networks as well as designers and implementors. The Editorial Board will consider any material for publication that is of interest to those groups.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信