{"title":"Adaptive energy-efficient reinforcement learning for AUV 3D motion planning in complex underwater environments","authors":"Jiayi Wen , Anqing Wang , Jingwei Zhu , Fengbei Xia , Zhouhua Peng , Weidong Zhang","doi":"10.1016/j.oceaneng.2024.119111","DOIUrl":null,"url":null,"abstract":"<div><p>This paper addresses the problem of 3D motion planning for autonomous underwater vehicles (AUVs) in complex underwater environments where prior environmental information is unavailable. A policy-feature-based state-dependent-exploration soft actor-critic (PSDE-SAC) framework integrating prioritized experience relay (PER) mechanism is developed for energy-efficient AUV underwater navigation. Specifically, a generalized exponential-based energy consumption model is firstly constructed to enable accurate calculation of energy consumption between any two points in a 3D underwater environment regardless of environmental disturbances. Then, an adaptive reward function with adjustable weights is designed to balance energy consumption and travel distance. Based on the well-designed reward function, the PSDE-SAC motion planning framework is constructed such that the frequently encountered challenges of erratic motion and restricted exploration in reinforcement learning are addressed. In addition, with the introduction of PER and policy features, the convergence and exploration abilities of the PSDE-SAC framework are significantly enhanced. Simulation results illustrate the superiority of the proposed method against other reinforcement learning algorithms in terms of energy consumption, convergence, and stability.</p></div>","PeriodicalId":19403,"journal":{"name":"Ocean Engineering","volume":"312 ","pages":"Article 119111"},"PeriodicalIF":5.5000,"publicationDate":"2024-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ocean Engineering","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0029801824024491","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, CIVIL","Score":null,"Total":0}
引用次数: 0
Abstract
This paper addresses the problem of 3D motion planning for autonomous underwater vehicles (AUVs) in complex underwater environments where prior environmental information is unavailable. A policy-feature-based state-dependent-exploration soft actor-critic (PSDE-SAC) framework integrating prioritized experience relay (PER) mechanism is developed for energy-efficient AUV underwater navigation. Specifically, a generalized exponential-based energy consumption model is firstly constructed to enable accurate calculation of energy consumption between any two points in a 3D underwater environment regardless of environmental disturbances. Then, an adaptive reward function with adjustable weights is designed to balance energy consumption and travel distance. Based on the well-designed reward function, the PSDE-SAC motion planning framework is constructed such that the frequently encountered challenges of erratic motion and restricted exploration in reinforcement learning are addressed. In addition, with the introduction of PER and policy features, the convergence and exploration abilities of the PSDE-SAC framework are significantly enhanced. Simulation results illustrate the superiority of the proposed method against other reinforcement learning algorithms in terms of energy consumption, convergence, and stability.
期刊介绍:
Ocean Engineering provides a medium for the publication of original research and development work in the field of ocean engineering. Ocean Engineering seeks papers in the following topics.