Bolun Dai, P. Krishnamurthy, A. Papanicolaou, F. Khorrami
{"title":"State Constrained Stochastic Optimal Control for Continuous and Hybrid Dynamical Systems Using DFBSDE","authors":"Bolun Dai, P. Krishnamurthy, A. Papanicolaou, F. Khorrami","doi":"10.48550/arXiv.2305.06499","DOIUrl":"https://doi.org/10.48550/arXiv.2305.06499","url":null,"abstract":"We develop a computationally efficient learning-based forward-backward stochastic differential equations (FBSDE) controller for both continuous and hybrid dynamical (HD) systems subject to stochastic noise and state constraints. Solutions to stochastic optimal control (SOC) problems satisfy the Hamilton-Jacobi-Bellman (HJB) equation. Using current FBSDE-based solutions, the optimal control can be obtained from the HJB equations using deep neural networks (e.g., long short-term memory (LSTM) networks). To ensure the learned controller respects the constraint boundaries, we enforce the state constraints using a soft penalty function. In addition to previous works, we adapt the deep FBSDE (DFBSDE) control framework to handle HD systems consisting of continuous dynamics and a deterministic discrete state change. We demonstrate our proposed algorithm in simulation on a continuous nonlinear system (cart-pole) and a hybrid nonlinear system (five-link biped).","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91135719","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sina Arefizadeh, Sadegh Arefizadeh, S. Etesami, Sadegh Bolouki
{"title":"Robustness of Dynamics in Games: A Contraction Mapping Decomposition Approach","authors":"Sina Arefizadeh, Sadegh Arefizadeh, S. Etesami, Sadegh Bolouki","doi":"10.48550/arXiv.2303.15386","DOIUrl":"https://doi.org/10.48550/arXiv.2303.15386","url":null,"abstract":"A systematic framework for analyzing dynamical attributes of games has not been well-studied except for the special class of potential or near-potential games. In particular, the existing results have shortcomings in determining the asymptotic behavior of a given dynamic in a designated game. Although there is a large body literature on developing convergent dynamics to the Nash equilibrium (NE) of a game, in general, the asymptotic behavior of an underlying dynamic may not be even close to a NE. In this paper, we initiate a new direction towards game dynamics by studying the fundamental properties of the map of dynamics in games. To this aim, we first decompose the map of a given dynamic into contractive and non-contractive parts and then explore the asymptotic behavior of those dynamics using the proximity of such decomposition to contraction mappings. In particular, we analyze the non-contractive behavior for better/best response dynamics in discrete-action space sequential/repeated games and show that the non-contractive part of those dynamics is well-behaved in a certain sense. That allows us to estimate the asymptotic behavior of such dynamics using a neighborhood around the fixed point of their contractive part proxy. Finally, we demonstrate the practicality of our framework via an example from duopoly Cournot games.","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80403652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Kernel-based Regularized Iterative Learning Control of Repetitive Linear Time-varying Systems","authors":"Xian Yu, Xiaozhu Fang, Biqiang Mu, Tianshi Chen","doi":"10.48550/arXiv.2303.03822","DOIUrl":"https://doi.org/10.48550/arXiv.2303.03822","url":null,"abstract":"For data-driven iterative learning control (ILC) methods, both the model estimation and controller design problems are converted to parameter estimation problems for some chosen model structures. It is well-known that if the model order is not chosen carefully, models with either large variance or large bias would be resulted, which is one of the obstacles to further improve the modeling and tracking performances of data-driven ILC in practice. An emerging trend in the system identification community to deal with this issue is using regularization instead of the statistical tests, e.g., AIC, BIC, and one of the representatives is the so-called kernel-based regularization method (KRM). In this paper, we integrate KRM into data-driven ILC to handle a class of repetitive linear time-varying systems, and moreover, we show that the proposed method has ultimately bounded tracking error in the iteration domain. The numerical simulation results show that in contrast with the least squares method and some existing data-driven ILC methods, the proposed one can give faster convergence speed, better accuracy and robustness in terms of the tracking performance.","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82503101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Ruminations on CES, Robotics, and the Future [President's Message]","authors":"F. Park","doi":"10.1109/mra.2023.3237459","DOIUrl":"https://doi.org/10.1109/mra.2023.3237459","url":null,"abstract":"","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88108748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"RAS New Quadruped Robot Challenges: Pushing the Boundaries of Robotics Research and Development [Competitions]","authors":"H. Moon","doi":"10.1109/mra.2023.3238203","DOIUrl":"https://doi.org/10.1109/mra.2023.3238203","url":null,"abstract":"","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73224297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Robotics Innovation for the Elderly [From the Editor's Desk]","authors":"Yi Guo","doi":"10.1109/mra.2023.3237423","DOIUrl":"https://doi.org/10.1109/mra.2023.3237423","url":null,"abstract":"","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76936507","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tat’ána Příhodová, Giuseppe Silano, Afzal Ahmad, V. Krátký, T. Báča, Pavel Petráček, Věra Sasková, J. Bednář, M. Saska
{"title":"2022 IEEE Robotics and Automation Society Summer School on Multi-Robot Systems in Prague [Education]","authors":"Tat’ána Příhodová, Giuseppe Silano, Afzal Ahmad, V. Krátký, T. Báča, Pavel Petráček, Věra Sasková, J. Bednář, M. Saska","doi":"10.1109/mra.2023.3238213","DOIUrl":"https://doi.org/10.1109/mra.2023.3238213","url":null,"abstract":"","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"91427866","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daniel Leidner, G. Chalvatzaki, Karinne Ramirez-Amaro
{"title":"Diversity as a Game Changer for Successful Teams in RoboCup [Women in Engineering]","authors":"Daniel Leidner, G. Chalvatzaki, Karinne Ramirez-Amaro","doi":"10.1109/mra.2023.3238211","DOIUrl":"https://doi.org/10.1109/mra.2023.3238211","url":null,"abstract":"","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79733067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"From University to Start-Up: The Experience of Akina [Student's Corner]","authors":"Francesco Missiroli","doi":"10.1109/mra.2023.3238212","DOIUrl":"https://doi.org/10.1109/mra.2023.3238212","url":null,"abstract":"","PeriodicalId":13196,"journal":{"name":"IEEE Robotics Autom. Mag.","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81439344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}