{"title":"Risk-Sensitive Average Markov Decision Processes in General Spaces","authors":"Xian Chen, Qingda Wei","doi":"10.1137/23m156118x","DOIUrl":"https://doi.org/10.1137/23m156118x","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 2115-2147, August 2024. <br/> Abstract. In this paper we study discrete-time Markov decision processes with Borel state and action spaces under the risk-sensitive average cost criterion. The cost function can be unbounded. We introduce a new kernel and prove the quasi-compactness of the kernel from which the multiplicative Poisson equation is derived. Moreover, we develop a new approach to show the existence of a solution to the risk-sensitive average cost optimality equation and obtain the existence of an optimal deterministic stationary policy. Furthermore, we give two examples to illustrate our results.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141608521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Existence of Optimal Pairs for Optimal Control Problems with States Constrained to Riemannian Manifolds","authors":"Li Deng, Xu Zhang","doi":"10.1137/23m1584095","DOIUrl":"https://doi.org/10.1137/23m1584095","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 2098-2114, August 2024. <br/> Abstract. In this paper, we investigate the existence of optimal pairs for optimal control problems with their states constrained pointwise to Riemannian manifolds. For this purpose, by means of the Riemannian geometric tool, we introduce a crucial Cesari-type property, which is an extension of the classical Cesari property (see Definition 3.3, p. 51 in [L. D. Berkovitz, Optimal Control Theory, Appl. Math. Sci. 12, Springer-Verlag, New York, Heidelberg, 1974]) from the setting of Euclidean spaces to that of Riemannian manifolds. Moreover, we show the efficiency of our result by a concrete example.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141608524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Thomas Berger, Dario Dennstädt, Lukas Lanza, Karl Worthmann
{"title":"Robust Funnel Model Predictive Control for Output Tracking with Prescribed Performance","authors":"Thomas Berger, Dario Dennstädt, Lukas Lanza, Karl Worthmann","doi":"10.1137/23m1551195","DOIUrl":"https://doi.org/10.1137/23m1551195","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 2071-2097, August 2024. <br/> Abstract. We propose a novel robust Model Predictive Control (MPC) scheme for nonlinear multi-input multi-output systems of relative degree one with stable internal dynamics. The proposed algorithm is a combination of funnel MPC, i.e., MPC with a particular stage cost, and the model-free adaptive funnel controller. The new robust funnel MPC scheme guarantees output tracking of reference signals within prescribed performance bounds—even in the presence of unknown disturbances and a structural model-plant mismatch. We show initial and recursive feasibility of the proposed control scheme without imposing terminal conditions or any requirements on the prediction horizon. Moreover, we allow for model updates at runtime. To this end, we propose a proper initialization strategy, which ensures that recursive feasibility is preserved. Finally, we validate the performance of the proposed robust MPC scheme by simulations.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141608522","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jan Bartsch, Patrik Knopf, Stefania Scheurer, Jörg Weber
{"title":"Controlling a Vlasov–Poisson Plasma by a Particle-in-Cell Method Based on a Monte Carlo Framework","authors":"Jan Bartsch, Patrik Knopf, Stefania Scheurer, Jörg Weber","doi":"10.1137/23m1563852","DOIUrl":"https://doi.org/10.1137/23m1563852","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 1977-2011, August 2024. <br/> Abstract. The Vlasov–Poisson system describes the time evolution of a plasma in the so-called collisionless regime. The investigation of a high-temperature plasma that is influenced by an exterior magnetic field is one of the most significant aspects of thermonuclear fusion research. In this paper, we formulate and analyze a kinetic optimal control problem for the Vlasov–Poisson system where the control is represented by an external magnetic field. The main goal of such optimal control problems is to confine the plasma to a certain region in phase space. We first investigate the optimal control problem in terms of mathematical analysis, i.e., we show the existence of at least one global minimizer and rigorously derive a first-order necessary optimality condition for local minimizers by the adjoint approach. Then we build a Monte Carlo framework to solve the state equations as well as the adjoint equations by means of a particle-in-cell method, and we apply a nonlinear conjugate gradient method to solve the optimization problem. Eventually, we present numerical experiments that successfully validate our optimization framework.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141587996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Gradient Flows for Regularized Stochastic Control Problems","authors":"David Šiška, Łukasz Szpruch","doi":"10.1137/20m1373645","DOIUrl":"https://doi.org/10.1137/20m1373645","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 2036-2070, August 2024. <br/> Abstract. This paper studies stochastic control problems with the action space taken to be probability measures, with the objective penalized by the relative entropy. We identify a suitable metric space on which we construct a gradient flow for the measure-valued control process, in the set of admissible controls, along which the cost functional is guaranteed to decrease. It is shown that any invariant measure of this gradient flow satisfies the Pontryagin optimality principle. If the problem we work with is sufficiently convex, the gradient flow converges exponentially fast. Furthermore, the optimal measure-valued control process admits a Bayesian interpretation, which means that one can incorporate prior knowledge when solving such stochastic control problems. This work is motivated by a desire to extend the theoretical underpinning for the convergence of stochastic gradient type algorithms widely employed in the reinforcement learning community to solve control problems.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141588585","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Optimal Impulsive Control for Time Delay Systems","authors":"Giovanni Fusco, Monica Motta, Richard Vinter","doi":"10.1137/24m1632450","DOIUrl":"https://doi.org/10.1137/24m1632450","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 2012-2035, August 2024. <br/> Abstract. We introduce discontinuous solutions to nonlinear impulsive control systems with state time delays in the dynamics and derive necessary optimality conditions in the form of a maximum principle for associated optimal control problems. In the case without delays, if the measure control is scalar valued, the corresponding discontinuous state trajectory, understood as a limit of classical state trajectories for absolutely continuous controls approximating the measure, is unique. For vector-valued measure controls, however, the limiting trajectory is not unique and a full description of the control must include additional “attached” controls affecting instantaneous state evolution at a discontinuity. For impulsive control systems with time delays we reveal a new phenomenon, namely, that the limiting state trajectory resulting from different approximations of a given measure control needs not to be unique, even in the scalar case. Correspondingly, our framework allows for additional attached controls, even though the measure control is scalar valued.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141587995","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Further on Pinning Synchronization of Dynamical Networks with Coupling Delay","authors":"Shuaibing Zhu, Jinhu Lü","doi":"10.1137/23m1578085","DOIUrl":"https://doi.org/10.1137/23m1578085","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 1933-1952, August 2024. <br/> Abstract. Though extensively studied, the longstanding pinning synchronization problem of dynamical networks with coupling delay has not been well solved until now. In this paper, we further investigate this problem. By proposing a system of functional differential inequalities, we derive synchronization criteria for dynamical networks with coupling delay under linear pinning control, where the threshold of the admissible delay and the control gain threshold are estimated. Since the estimated control gain threshold could be very large when the delay draws close to the delay threshold, we also use the adaptive pinning control scheme to avoid control gain estimation. Pinning synchronization criteria of networks under adaptive control are derived and the delay threshold is given. This is the first time that general coupling delay has been addressed in the field of pinning control. Finally, two numerical examples are presented to validate the theoretical results.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141548535","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the Exact Boundary Controllability of Semilinear Wave Equations","authors":"Sue Claret, Jérôme Lemoine, Arnaud Münch","doi":"10.1137/23m1586598","DOIUrl":"https://doi.org/10.1137/23m1586598","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 1953-1976, August 2024. <br/> Abstract. We address the exact boundary controllability of the semilinear wave equation [math] posed over a bounded domain [math] of [math]. Assuming that [math] is continuous and satisfies the condition [math] for some [math] small enough and some [math] in [math], we apply the Schauder fixed point theorem to prove the uniform controllability for initial data in [math]. Then, assuming that [math] is in [math] and satisfies the condition [math], we apply the Banach fixed point theorem and exhibit a strongly convergent sequence to a state-control pair for the semilinear equation.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141548471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"RBFNN Adaptive Sampled-Data Control for Nonlinear Plants: A Validity Analysis","authors":"Hao Yu, Tongwen Chen","doi":"10.1137/23m1595035","DOIUrl":"https://doi.org/10.1137/23m1595035","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 3, Page 1908-1932, June 2024. <br/> Abstract. This paper investigates adaptive sampled-data control for strict-feedback nonlinear plants with unmatched uncertainties by means of radial basis function neural networks (RBFNNs). First, the continuous-time plant is locally discretized as a disturbed strict-feedback model by using the approximate Euler model approach. Then, as a basis of rigorous stability analysis, the concept of validity is proposed, which, considering the locality of the universal approximation capacity in RBFNNs, requires that the argument of each RBFNN be inside the corresponding compact set all the time. Meanwhile, to address the noncausality issue, delayed signals are utilized in the backstepping method for discrete-time plants. Subsequently, the validity and stability are proved rigorously; meanwhile, a practical output tracking problem is solved under a time-varying reference signal, the order of whose continuous derivatives is the same as the plants. This is the first time the interdependence on the design of sampling periods and RBFNNs in different design steps has been shown. Finally, simulation results are provided to illustrate the efficiency and feasibility of the obtained results.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141508061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Average Cost Minimization Problems Subject to State Constraints","authors":"Piernicola Bettiol, Nathalie Khalil","doi":"10.1137/23m1558124","DOIUrl":"https://doi.org/10.1137/23m1558124","url":null,"abstract":"SIAM Journal on Control and Optimization, Volume 62, Issue 3, Page 1884-1907, June 2024. <br/> Abstract. In this paper we consider pathwise state constraint optimal control problems in which unknown parameters intervene in the dynamics, the cost, the endpoint constraint, and the state constraint. The cost criteria to minimize take an integral form of a given endpoint cost function with respect to a reference probability measure that is defined on the set of the unknown parameters. For this class of problems we derive necessary optimality conditions.","PeriodicalId":49531,"journal":{"name":"SIAM Journal on Control and Optimization","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141508064","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}