Two-stage three-phase photovoltaic grid-connected inverter control method based on off-policy integral reinforcement learning

IF 5.4 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Control Engineering Practice Pub Date : 2025-03-12 DOI:10.1016/j.conengprac.2025.106313

He Ren , Min Zhuang , Lingfang Sun , Huichao Ji , Xiuyu Zhang , Chun-Yi Su

{"title":"Two-stage three-phase photovoltaic grid-connected inverter control method based on off-policy integral reinforcement learning","authors":"He Ren , Min Zhuang , Lingfang Sun , Huichao Ji , Xiuyu Zhang , Chun-Yi Su","doi":"10.1016/j.conengprac.2025.106313","DOIUrl":null,"url":null,"abstract":"<div><div>In this article, a novel control method of the grid-connected inverter (GCI) based on the off-policy integral reinforcement learning (IRL) method is presented to solve two-stage three-phase photovoltaic (TTP) power quality degradation problem of the grid connection caused by unknown system dynamics. Firstly, the voltage-current dual-loop control (VDC) structure is adopted, where the model of the current loop is restructured benefitting from the current tracking principle. Secondly, the grid-connected power quality problem is turned into a zero-sum game (ZSG) problem, which can be completed by solving the Hamilton–Jacobi-Bellman (HJB) equations using the off-policy IRL method only with the collected input and output (I/O) data. Thirdly, the off-policy IRL method is improved on the basis of the traditional policy iteration method and the equivalence between them is demonstrated. Finally, the presented method is verified by StarSim Modeling Tech power electronic simulation experiment platform to satisfy the standard of the grid connection and the total harmonic distortion of the current is less than 5%.</div></div>","PeriodicalId":50615,"journal":{"name":"Control Engineering Practice","volume":"159 ","pages":"Article 106313"},"PeriodicalIF":5.4000,"publicationDate":"2025-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Control Engineering Practice","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0967066125000760","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

In this article, a novel control method of the grid-connected inverter (GCI) based on the off-policy integral reinforcement learning (IRL) method is presented to solve two-stage three-phase photovoltaic (TTP) power quality degradation problem of the grid connection caused by unknown system dynamics. Firstly, the voltage-current dual-loop control (VDC) structure is adopted, where the model of the current loop is restructured benefitting from the current tracking principle. Secondly, the grid-connected power quality problem is turned into a zero-sum game (ZSG) problem, which can be completed by solving the Hamilton–Jacobi-Bellman (HJB) equations using the off-policy IRL method only with the collected input and output (I/O) data. Thirdly, the off-policy IRL method is improved on the basis of the traditional policy iteration method and the equivalence between them is demonstrated. Finally, the presented method is verified by StarSim Modeling Tech power electronic simulation experiment platform to satisfy the standard of the grid connection and the total harmonic distortion of the current is less than 5%.

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Control Engineering Practice 工程技术-工程：电子与电气

CiteScore

9.20

自引率

12.20%

发文量

183

审稿时长

44 days

期刊介绍： Control Engineering Practice strives to meet the needs of industrial practitioners and industrially related academics and researchers. It publishes papers which illustrate the direct application of control theory and its supporting tools in all possible areas of automation. As a result, the journal only contains papers which can be considered to have made significant contributions to the application of advanced control techniques. It is normally expected that practical results should be included, but where simulation only studies are available, it is necessary to demonstrate that the simulation model is representative of a genuine application. Strictly theoretical papers will find a more appropriate home in Control Engineering Practice''s sister publication, Automatica. It is also expected that papers are innovative with respect to the state of the art and are sufficiently detailed for a reader to be able to duplicate the main results of the paper (supplementary material, including datasets, tables, code and any relevant interactive material can be made available and downloaded from the website). The benefits of the presented methods must be made very clear and the new techniques must be compared and contrasted with results obtained using existing methods. Moreover, a thorough analysis of failures that may happen in the design process and implementation can also be part of the paper. The scope of Control Engineering Practice matches the activities of IFAC. Papers demonstrating the contribution of automation and control in improving the performance, quality, productivity, sustainability, resource and energy efficiency, and the manageability of systems and processes for the benefit of mankind and are relevant to industrial practitioners are most welcome.