Meiying Yang , Hai Zhu , Xiaozhou Zhu , Zhe Liu , Wen Yao , Xiaoqian Chen
{"title":"Prescribed performance UAV tracking control under disturbance using reinforcement learning-based backstepping","authors":"Meiying Yang , Hai Zhu , Xiaozhou Zhu , Zhe Liu , Wen Yao , Xiaoqian Chen","doi":"10.1016/j.conengprac.2025.106393","DOIUrl":null,"url":null,"abstract":"<div><div>For the tracking control problem of unmanned aerial vehicle (UAVs) with nonlinear and strongly coupled dynamics, a reinforcement learning (RL) optimization control method with prescribed performance under disturbances is proposed based on the backstepping framework. This method employs RL to solve the Hamilton–Jacobi–Bellman (HJB) equation in the optimization problem, which involves tracking errors and control inputs. Among them, the actor network is used in the controller to ensure system stability, while the critic network is employed to evaluate system performance through performance index functions. Additionally, a reduced-order extended state observer is designed to estimate external disturbances, and the estimated results are applied to the controller to compensate for the impact of external disturbances on the UAV. During controller design, first-order filtering helps resolve the complex differentiation issues inherent in the backstepping method. For the prescribed performance issues, particularly the tracking error constraint, a performance index function guarantees that the tracking error stays within the desired range. Next, the stability performance of the UAV system is proven using Lyapunov theory. Finally, the effectiveness of the proposed control method is further validated through numerical simulations and physical experiments.</div></div>","PeriodicalId":50615,"journal":{"name":"Control Engineering Practice","volume":"164 ","pages":"Article 106393"},"PeriodicalIF":4.6000,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Control Engineering Practice","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S096706612500156X","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
For the tracking control problem of unmanned aerial vehicle (UAVs) with nonlinear and strongly coupled dynamics, a reinforcement learning (RL) optimization control method with prescribed performance under disturbances is proposed based on the backstepping framework. This method employs RL to solve the Hamilton–Jacobi–Bellman (HJB) equation in the optimization problem, which involves tracking errors and control inputs. Among them, the actor network is used in the controller to ensure system stability, while the critic network is employed to evaluate system performance through performance index functions. Additionally, a reduced-order extended state observer is designed to estimate external disturbances, and the estimated results are applied to the controller to compensate for the impact of external disturbances on the UAV. During controller design, first-order filtering helps resolve the complex differentiation issues inherent in the backstepping method. For the prescribed performance issues, particularly the tracking error constraint, a performance index function guarantees that the tracking error stays within the desired range. Next, the stability performance of the UAV system is proven using Lyapunov theory. Finally, the effectiveness of the proposed control method is further validated through numerical simulations and physical experiments.
期刊介绍:
Control Engineering Practice strives to meet the needs of industrial practitioners and industrially related academics and researchers. It publishes papers which illustrate the direct application of control theory and its supporting tools in all possible areas of automation. As a result, the journal only contains papers which can be considered to have made significant contributions to the application of advanced control techniques. It is normally expected that practical results should be included, but where simulation only studies are available, it is necessary to demonstrate that the simulation model is representative of a genuine application. Strictly theoretical papers will find a more appropriate home in Control Engineering Practice''s sister publication, Automatica. It is also expected that papers are innovative with respect to the state of the art and are sufficiently detailed for a reader to be able to duplicate the main results of the paper (supplementary material, including datasets, tables, code and any relevant interactive material can be made available and downloaded from the website). The benefits of the presented methods must be made very clear and the new techniques must be compared and contrasted with results obtained using existing methods. Moreover, a thorough analysis of failures that may happen in the design process and implementation can also be part of the paper.
The scope of Control Engineering Practice matches the activities of IFAC.
Papers demonstrating the contribution of automation and control in improving the performance, quality, productivity, sustainability, resource and energy efficiency, and the manageability of systems and processes for the benefit of mankind and are relevant to industrial practitioners are most welcome.