{"title":"Adversarial Learning for Safe Highway Driving based on Two-Player Zero-Sum Game","authors":"Fangjian Li, Mengtao Zhao, J. Wagner, Yue Wang","doi":"10.23919/ACC55779.2023.10156179","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10156179","url":null,"abstract":"In this paper, we set up a two-player-zero-sum Markov game (TZMG) framework to train a safe driving policy network so that the worst intentions of the neighbor vehicles can be considered. Compared to the conventional policy learning frameworks, the TZMG framework can embed the adversary from the neighbor vehicle throughout its training process. Furthermore, a novel TZMG Q-learning algorithm based on the Wolpertinger policy is proposed to be scalable to multiple adversarial neighbor vehicles. Finally, simulations and humansin-the-loop experiments are conducted to verify the effectiveness of the TZMG framework and novel algorithm. Compared to the benchmarking safety controllers in the literature, our proposed novel TZMG algorithm can achieve a much lower collision rate when dealing with adversarial neighbor vehicles.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"151 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125047989","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Asymptotic Stabilization of Aperiodic Trajectories of a Hybrid-Linear Inverted Pendulum Walking on a Vertically Moving Surface","authors":"Amir Iqbal, Sushant Veer, Yan Gu","doi":"10.23919/ACC55779.2023.10156645","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10156645","url":null,"abstract":"This paper presents the analysis and stabilization of a hybrid-linear inverted pendulum (H-LIP) model that describes the essential robot dynamics associated with legged locomotion on a dynamic rigid surface (DRS) with a general vertical motion. The H-LIP model is analytically derived by explicitly capturing the discrete-time foot placement and the continuous-phase dynamics associated with DRS locomotion, and by considering aperiodic DRS motions and variable H-LIP continuous-phase durations. The closed-loop tracking error dynamics of the H-LIP model is then established under a discrete-time feedback footstep control law. The stability of the closed-loop H-LIP error dynamics is analyzed to construct sufficient conditions on the control gains for ensuring the asymptotic error convergence. Simulation results of the proposed H-LIP walking on a vertically moving DRS confirm the proposed control law stabilizes the H-LIP model under various vertical, aperiodic DRS motion profiles and variable H-LIP step durations.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114277698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Distributed Robust Control Framework for Adaptive Cruise Control Systems","authors":"Mohammad Mahmoudi Filabadi, E. Hashemi","doi":"10.23919/ACC55779.2023.10155952","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10155952","url":null,"abstract":"A distributed robust adaptive control framework is proposed for an adaptive cruise control system. The proposed approach is designed based on the model reference adaptive control approach. A robust control term is employed to make the system robust to any bounded disturbances, and a concurrent learning framework is leveraged to ensure the convergence of estimated parameters. The main feature of the developed robust adaptive cruise controller is that it does not require the speed of the lead vehicle. It also considers uncertainties in both position and speed in the double integrator model. The string stability notion of the proposed approach is also investigated, and the performance of the control framework is evaluated in simulations in the presence of parametric uncertainties, disturbances, and noise.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125275381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Tracking Control of Multi-Input Multi-Output Multirotor Unmanned Aerial Vehicles with Auxiliary Systems","authors":"S. Lyshevski, Trevor C. Smith","doi":"10.23919/ACC55779.2023.10156479","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10156479","url":null,"abstract":"We research control schemes for unmanned aerial vehicles (UAVs) with propulsion, steering and power modules. Physical limits, aerodynamic instabilities, blade flapping, cross-axis coupling, data heterogeneity and other factors affect design. In multirotor UAVs, the differential thrust is regulated by changing the angular velocity of propellers, rotated by brushless electric motors. Voltages applied, phase currents, propeller speed and thrust cannot exceed specific limits. To accomplish aerial photography, airborne intelligence, surveillance, reconnaissance and support missions, multirotor and fixed-wing vehicles integrate active electronically scanned array radar, light detection and ranging modules, transceivers, controllers-drivers, steered pylon mounts, dc-dc regulators, battery pack, charger, etc. The differential thrust is regulated by changing propellers’ angular velocity. We design constrained tracking control laws to govern aerial systems regulating state and error dynamics. Minimizing design-consistent functionals with range-restricted descriptive bounded functions, limits are accounted for by integrands, and control laws are analytically designed. Nonquadratic functionals with domain-specific positive-definite integrands and Hamiltonians admit closed-form solutions. The Hamilton-Jacobi equation is satisfied by continuous positive-definite return functions. Descriptive state-space models and error governance support a design to ensure optimal tracking error evolution. Bounded algorithms with state and tracking error feedback guarantee system optimality subject to minimized functionals. Control schemes, optimization tools, and algorithms are experimentally substantiated for a quadrotor helicopter. Controllers are designed and characterized for flight control systems, direct-drive steering mount pylons, brushless motors, and dc-dc switching regulators.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"151 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115182078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pradeep Sharma Oruganti, Parinaz Naghizadeh Ardabili, Q. Ahmed
{"title":"Safe Control Using High-Order Measurement Robust Control Barrier Functions","authors":"Pradeep Sharma Oruganti, Parinaz Naghizadeh Ardabili, Q. Ahmed","doi":"10.23919/ACC55779.2023.10155975","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10155975","url":null,"abstract":"We study the problem of providing safety guarantees for dynamic systems of high relative degree in the presence of state measurement errors. To this end, we propose High-Order Measurement Robust Control Barrier Functions (HO-MR-CBFs), an extension of the recently proposed Measurement Robust Control Barrier Functions. We begin by formally defining HO-MR-CBF, and identify conditions under which the proposed HO-MR-CBF can render the system’s safe set forward invariant. In addition, we provide bounds on the state measurement errors for which the optimization problem for identifying the corresponding safe controllers is feasible for all states within the safe set and given restricted control inputs. We demonstrate the proposed approach through numerical experiments on a collision avoidance scenario in presence of measurement noise. We show that using our proposed control method, the robot, which has access to only biased state estimates, will be successful in avoiding the obstacle.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123870456","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Passivity, RL and Learning in Multi-Agent Games","authors":"Lacra Pavel","doi":"10.23919/ACC55779.2023.10156507","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10156507","url":null,"abstract":"Learning algorithm behavior highly depends on the game setting. In this tutorial talk, we discuss how these dependencies can be explained, if one regards them through a passivity lens. We focus on two representative instances in reinforcement learning: payoff-based play, and Q-learning. We show how one can exploit geometric features of different classes of games, together with dissipativity/passivity properties of interconnected systems to guarantee global convergence to a Nash equilibrium. Besides simplifying the proof of convergence, one can generate algorithms that work for classes of games with less stringent assumptions, by using passivity and basic properties of interconnected systems.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131153469","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exponential TD Learning: A Risk-Sensitive Actor-Critic Reinforcement Learning Algorithm","authors":"Erfaun Noorani, Christos N. Mavridis, J. Baras","doi":"10.23919/ACC55779.2023.10156626","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10156626","url":null,"abstract":"Incorporating risk in the decision-making process has been shown to lead to significant performance improvement in optimal control and reinforcement learning algorithms. We construct a temporal-difference risk-sensitive reinforcement learning algorithm using the exponential criteria commonly used in risk-sensitive control. The proposed method resembles an actor-critic architecture with the ‘actor’ implementing a policy gradient algorithm based on the exponential of the reward-to-go, which is estimated by the ‘critic’. The novelty of the update rule of the ‘critic’ lies in the use of a modified objective function that corresponds to the underlying multiplicative Bellman’s equation. Our results suggest that the use of the exponential criteria accelerates the learning process and reduces its variance, i.e., risk-sensitiveness can be utilized by actor-critic methods and can lead to improved performance.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"19 1 Suppl 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131172025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bearing-only formation control using sign-elevation angle rigidity for avoiding formation ambiguities","authors":"Chinmay Garanayak, Dwaipayan Mukherjee","doi":"10.23919/ACC55779.2023.10156015","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10156015","url":null,"abstract":"Flip, flex, and reflection ambiguities, which can arise in bearing-only formation control with elevation angle rigid configurations, are addressed in this paper. Elevation angle rigidity achieves formation control in agents’ local co-ordinate system using bearing-only sensors, without any orientation synchronization or estimation algorithm. Considering elevation angle constraints to determine the formation shape, and then using a gradient based control law offers the benefit of a co-ordinate free control. However, flip, flex, and reflection ambiguities might be present in the final formation shape. To tackle this, we first develop sign-elevation angle rigidity theory to uniquely (locally) characterize formation shapes upto a translation and rotation using elevation angle and signed area/volume constraints. Thereafter, a formation control law is proposed (for 2-D and 3-D) using bearing-only information for single integrator systems, and local exponential stability is proved for formation tracking error. Finally, simulations validate the presented results.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127597554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Mulders, L. Brandetti, F. Spagnolo, Y. Liu, P. Christensen, J. V. van Wingerden
{"title":"A learning algorithm for the calibration of internal model uncertainties in advanced wind turbine controllers: A wind speed measurement-free approach","authors":"S. Mulders, L. Brandetti, F. Spagnolo, Y. Liu, P. Christensen, J. V. van Wingerden","doi":"10.23919/ACC55779.2023.10156125","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10156125","url":null,"abstract":"Wind turbine partial-load controllers have evolved from simple static nonlinear function implementations to more advanced dynamic controller structures. Such dynamic control schemes have the potential to improve power production performance in realistic environmental conditions and allow for a more granular trade-off between loads and energy capture. The control structure generally consists of a wind speed estimator (WSE) combined with a controller aiming to track the commanded tip-speed ratio (TSR) reference. The performance and resulting closed-loop system stability are however highly dependent on the accuracy of the internal model in the WSE-TSR tracking scheme. Therefore, developing learning algorithms to calibrate the internal model is of particular interest. Previous works have proposed such algorithms; however, they all rely on the availability of (rotor-effective) wind speed measurements. For the first time, this paper proposes an excitation-based learning algorithm that exploits the closed-loop dynamic structure of the WSE-TSR tracking scheme. This algorithm calibrates the internal model without the need for wind speed measurements. Analysis and simulations show that the proposed algorithm corrects for model uncertainties in the form of magnitude scaling errors under ideal constant and realistic turbulent wind conditions.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127761743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Discretization of the Hybrid Gradient Algorithm for Linear Regression with Sampled Hybrid Signals","authors":"Nathan Wu, Ryan S. Johnson, R. Sanfelice","doi":"10.23919/ACC55779.2023.10155974","DOIUrl":"https://doi.org/10.23919/ACC55779.2023.10155974","url":null,"abstract":"We consider the problem of estimating a vector of unknown constant parameters for a linear regression model whose inputs and outputs are discretized hybrid signals – that is, they are samples of hybrid signals that exhibit both continuous (flow) and discrete (jump) evolution. Using a hybrid systems framework, we propose a hybrid gradient descent algorithm that operates during both flows and jumps. We show that this algorithm guarantees exponential convergence of the parameter estimate to the unknown parameter under a new notion of discretized hybrid persistence of excitation that relaxes the classical discrete-time persistence of excitation condition. Simulation results validate the properties guaranteed by the new algorithm.","PeriodicalId":397401,"journal":{"name":"2023 American Control Conference (ACC)","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132565178","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}