Conference on Learning for Dynamics & Control最新文献

筛选

英文中文

Practical Reinforcement Learning For MPC: Learning from sparse objectives in under an hour on a real robot MPC的实用强化学习:在一个真实的机器人上在一个小时内从稀疏目标学习

Conference on Learning for Dynamics & Control Pub Date : 2020-03-06 DOI: 10.3929/ETHZ-B-000404690

Napat Karnchanachari, M. I. Valls, David Hoeller, M. Hutter

引用次数: 25

Policy Optimization for H2 Linear Control with H∞ Robustness Guarantee: Implicit Regularization and Global Convergence 具有H∞鲁棒性保证的H2线性控制策略优化:隐正则化和全局收敛

Conference on Learning for Dynamics & Control Pub Date : 2019-10-21 DOI: 10.1137/20m1347942

K. Zhang, Bin Hu, T. Başar

{"title":"Policy Optimization for H2 Linear Control with H∞ Robustness Guarantee: Implicit Regularization and Global Convergence","authors":"K. Zhang, Bin Hu, T. Başar","doi":"10.1137/20m1347942","DOIUrl":"https://doi.org/10.1137/20m1347942","url":null,"abstract":"Policy optimization (PO) is a key ingredient for reinforcement learning (RL). For control design, certain constraints are usually enforced on the policies to optimize, accounting for either the stability, robustness, or safety concerns on the system. Hence, PO is by nature a constrained (nonconvex) optimization in most cases, whose global convergence is challenging to analyze in general. More importantly, some constraints that are safety-critical, e.g., the $mathcal{H}_infty$-norm constraint that guarantees the system robustness, are difficult to enforce as the PO methods proceed. Recently, policy gradient methods have been shown to converge to the global optimum of linear quadratic regulator (LQR), a classical optimal control problem, without regularizing/projecting the control iterates onto the stabilizing set (Fazel et al., 2018), its (implicit) feasible set. This striking result is built upon the coercive property of the cost, ensuring that the iterates remain feasible as the cost decreases. In this paper, we study the convergence theory of PO for $mathcal{H}_2$ linear control with $mathcal{H}_infty$-norm robustness guarantee. One significant new feature of this problem is the lack of coercivity, i.e., the cost may have finite value around the feasible set boundary, breaking the existing analysis for LQR. Interestingly, we show that two PO methods enjoy the implicit regularization property, i.e., the iterates preserve the $mathcal{H}_infty$ robustness constraint as if they are regularized by the algorithms. Furthermore, convergence to the globally optimal policies with globally sublinear and locally (super-)linear rates are provided under certain conditions, despite the nonconvexity of the problem. To the best of our knowledge, our work offers the first results on the implicit regularization property and global convergence of PO methods for robust/risk-sensitive control.","PeriodicalId":268449,"journal":{"name":"Conference on Learning for Dynamics & Control","volume":"136 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133650190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 93

Frequency Domain Gaussian Process Models for H∞ Uncertainties H∞不确定性的频域高斯过程模型

Conference on Learning for Dynamics & Control Pub Date : 1900-01-01 DOI: 10.48550/arXiv.2211.15923

Alex Devonport, P. Seiler, M. Arcak

引用次数: 0

首页上一页