AutomaticaPub Date : 2024-10-30DOI: 10.1016/j.automatica.2024.111976
{"title":"Predict globally, correct locally: Parallel-in-time optimization of neural networks","authors":"","doi":"10.1016/j.automatica.2024.111976","DOIUrl":"10.1016/j.automatica.2024.111976","url":null,"abstract":"<div><div>The training of neural networks can be formulated as an optimal control problem of a dynamical system. The initial conditions of the dynamical system are given by the data. The objective of the control problem is to transform the initial conditions in a form that can be easily classified or regressed using linear methods. This link between optimal control of dynamical systems and neural networks has proved beneficial both from a theoretical and from a practical point of view. Several researchers have exploited this link to investigate the stability of different neural network architectures and develop memory efficient training algorithms. In this paper, we also adopt the dynamical systems view of neural networks, but our aim is different from earlier works. Instead, we develop a novel distributed optimization algorithm. The proposed algorithm addresses the most significant obstacle for distributed algorithms for neural network optimization: the network weights cannot be updated until the forward propagation of the data, and backward propagation of the gradients are complete. Using the dynamical systems point of view, we interpret the layers of a (residual) neural network as the discretized dynamics of a dynamical system and exploit the relationship between the co-states (adjoints) of the optimal control problem and backpropagation. We then develop a parallel-in-time method that updates the parameters of the network without waiting for the forward or back propagation algorithms to complete in full. We establish the convergence of the proposed algorithm. Preliminary numerical results suggest that the algorithm is competitive and more efficient than the state-of-the-art.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142552786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-28DOI: 10.1016/j.automatica.2024.111970
{"title":"Set-based value operators for non-stationary and uncertain Markov decision processes","authors":"","doi":"10.1016/j.automatica.2024.111970","DOIUrl":"10.1016/j.automatica.2024.111970","url":null,"abstract":"<div><div>This paper analyzes finite-state Markov Decision Processes (MDPs) with nonstationary and uncertain parameters via set-based fixed point theory. Given compact parameter ambiguity sets, we demonstrate that a family of contraction operators, including the Bellman operator and the policy evaluation operator, can be extended to set-based contraction operators with a unique fixed point—a compact value function set. For non-stationary MDPs, we show that while the value function trajectory diverges, its Hausdorff distance from this fixed point converges to zero. In parameter uncertain MDPs, the fixed point’s extremum value functions are equivalent to the min–max value function in robust dynamic programming under the rectangularity condition. Furthermore, we show that the rectangularity condition is a sufficient condition for the fixed point to contain its own extremum value functions. Finally, we derive novel guarantees for probabilistic path planning in capricious wind fields and stratospheric station-keeping.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142526131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-25DOI: 10.1016/j.automatica.2024.111977
{"title":"Nonuniqueness and convergence to equivalent solutions in observer-based inverse reinforcement learning","authors":"","doi":"10.1016/j.automatica.2024.111977","DOIUrl":"10.1016/j.automatica.2024.111977","url":null,"abstract":"<div><div>A key challenge in solving the deterministic inverse reinforcement learning (IRL) problem online and in real-time is the existence of multiple solutions. Nonuniqueness necessitates the study of the notion of equivalent solutions, <em>i.e.</em>, solutions that result in a different cost functional but same feedback matrix. While <em>offline</em> algorithms that result in convergence to equivalent solutions have been developed in the literature, online, real-time techniques that address nonuniqueness are not available. In this paper, a regularized history stack observer that converges to approximately equivalent solutions of the IRL problem is developed. Novel data-richness conditions are developed to facilitate the analysis and simulation results are provided to demonstrate the effectiveness of the developed technique.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142525967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-25DOI: 10.1016/j.automatica.2024.111919
{"title":"Successive over relaxation for model-free LQR control of discrete-time Markov jump systems","authors":"","doi":"10.1016/j.automatica.2024.111919","DOIUrl":"10.1016/j.automatica.2024.111919","url":null,"abstract":"<div><div>This paper aims to solve the model-free linear quadratic regulator problem for discrete-time Markov jump linear systems without requiring an initial stabilizing control policy. We propose both model-based and model-free successive over relaxation algorithms to learn the optimal control policy of discrete-time Markov jump linear systems. The model-free value iteration algorithm is a special case of our model-free algorithm when the relaxation factor equals one. A sufficient condition on the relaxation factor is provided to guarantee the convergence of our algorithms. Moreover, it is proved that our model-free algorithm can obtain an approximate optimal solution when the transition probability matrix is unknown. Finally, a numerical example is used to illustrate our results.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142525966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-22DOI: 10.1016/j.automatica.2024.111947
{"title":"Asymmetrical vulnerability of heterogeneous multi-agent systems under false-data injection attacks","authors":"","doi":"10.1016/j.automatica.2024.111947","DOIUrl":"10.1016/j.automatica.2024.111947","url":null,"abstract":"<div><div>This paper investigates the vulnerability of heterogeneous multi-agent systems (MASs) in face of perfect false-data injection (FDI) attacks that stealthily destabilize the synchronization processes of agents. In contrast to homogeneous dynamics, heterogeneous dynamics can be asymmetrically worsened by attackers, which is a greater challenge for MAS security. First of all, the existence conditions of perfect FDI attacks in different agents are established based on the spectral radius of heterogeneous system matrices. Then, it is proven that the attack targets against communication links are determined by the characteristic space of different agents. Since the attack properties including the attack existence and targets are different in each agent, heterogeneous MASs have the asymmetrical vulnerability under perfect FDI attacks. Finally, a sufficient condition and a necessary condition are obtained to achieve perfect FDI attacks with minimum attack targets. A numerical simulation of heterogeneous MASs is presented to demonstrate the effectiveness of perfect FDI attacks.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142525968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-21DOI: 10.1016/j.automatica.2024.111946
{"title":"Analysis of barrier function based adaptive sliding mode control in the presence of deterministic noise","authors":"","doi":"10.1016/j.automatica.2024.111946","DOIUrl":"10.1016/j.automatica.2024.111946","url":null,"abstract":"<div><div>Barrier function-based adaptive sliding mode control (BFASMC) is analyzed in presence of deterministic measurement noise. It is shown that, considering only boundedness of the measurement noise, it is impossible to select the controller parameters to track some perturbation with unknown bound. Nonetheless, under the assumption of continuity of the noise, the tracking of such a perturbation is possible; however, the barrier function width depends on the bound of the noise. If Lipschitz continuity of the noise is assumed, then it follows that the width of the barrier function can be chosen arbitrarily.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142525969","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-21DOI: 10.1016/j.automatica.2024.111981
{"title":"Stabilization for fast sampling discrete-time singularly perturbed singular Markovian systems","authors":"","doi":"10.1016/j.automatica.2024.111981","DOIUrl":"10.1016/j.automatica.2024.111981","url":null,"abstract":"<div><div>This paper considers the problems of stabilization and <span><math><msub><mrow><mi>H</mi></mrow><mrow><mi>∞</mi></mrow></msub></math></span> control for fast sampling discrete-time singularly perturbed singular Markovian systems (SPSMSs). The system equivalent approach is initially introduced to transform the discrete fast sampling SPSMS model into the augmented SPSMS for the convenience of designing system controller. Secondly, sufficient condition on stochastically mean square admissibility is established for the fast sampling SPSMS. By separating matrix variables and singularly perturbed parameter, a state feedback controller is also provided to ensure stochastically mean square admissibility of the fast sampling augmented SPSMS. Then, the results are extended to <span><math><msub><mrow><mi>H</mi></mrow><mrow><mi>∞</mi></mrow></msub></math></span> performance analysis and controller design in the presence of the external disturbances. The derived criteria can be converted to the feasible problems based on convex optimization, and the upper bound of singular perturbation parameter is also calculated. Besides, a discretized electrical circuit system is provided to verify the effectiveness and the superiority of the proposed approach.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142526134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-21DOI: 10.1016/j.automatica.2024.111978
{"title":"Approximate constrained stochastic optimal control via parameterized input inference","authors":"","doi":"10.1016/j.automatica.2024.111978","DOIUrl":"10.1016/j.automatica.2024.111978","url":null,"abstract":"<div><div>Approximate methods to solve stochastic optimal control (SOC) problems have received significant interest from researchers in the past decade. Probabilistic inference approaches to SOC have been developed to solve nonlinear quadratic Gaussian problems. In this work, we propose an Expectation–Maximization (EM) based inference procedure to generate state-feedback controls for constrained SOC problems. We consider the inequality constraints for the state and controls and also the structural constraints for the controls. We employ barrier functions to address state and control constraints. We show that the expectation step leads to smoothing of the state-control pair while the maximization step on the non-zero subsets of the control parameters allows inference of structured stochastic optimal controllers. We demonstrate the effectiveness of the algorithm on unicycle obstacle avoidance and four-unicycle formation control examples. In these examples, we perform an empirical study on the parametric effect of barrier functions on the state constraint satisfaction. We also present a comparative study of smoothing algorithms on the performance of the proposed approach.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142526133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-20DOI: 10.1016/j.automatica.2024.111958
{"title":"Corrigendum to “Assessment of initial-state-opacity in live and bounded labeled Petri net systems via optimization techniques” [Automatica 152 (2023) 110911]","authors":"","doi":"10.1016/j.automatica.2024.111958","DOIUrl":"10.1016/j.automatica.2024.111958","url":null,"abstract":"","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142525970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AutomaticaPub Date : 2024-10-18DOI: 10.1016/j.automatica.2024.111969
{"title":"Non-identifier based adaptive control of a chain of integrators and perturbations with unknown delays and parameters","authors":"","doi":"10.1016/j.automatica.2024.111969","DOIUrl":"10.1016/j.automatica.2024.111969","url":null,"abstract":"<div><div>This paper solves the problem of how to control a chain of integrators with <em>unknown delays</em> in both state and input by memoryless state feedback. Inspired by the non-identifier based adaptive control scheme (Lei and Lin, 2006) as well as the recent progress in stabilizing time-delay feedforward systems with unknown parameters via dynamic state compensation (Sun and Lin, 2023), we design a memoryless universal controller (finite-dimensional) which adaptively stabilizes the time-delay integrators whose all delays are not known a priori. As a byproduct of this development, global adaptive regulation of a class of nonlinear systems with unknown delays and unknown parameters is also solved by the proposed universal control strategy.</div></div>","PeriodicalId":55413,"journal":{"name":"Automatica","volume":null,"pages":null},"PeriodicalIF":4.8,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142525965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}