自适应控制和交叉口与强化学习

IF 11.2 1区 计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS
A. Annaswamy
{"title":"自适应控制和交叉口与强化学习","authors":"A. Annaswamy","doi":"10.1146/annurev-control-062922-090153","DOIUrl":null,"url":null,"abstract":"This article provides an exposition of the field of adaptive control and its intersections with reinforcement learning. Adaptive control and reinforcement learning are two different methods that are both commonly employed for the control of uncertain systems. Historically, adaptive control has excelled at real-time control of systems with specific model structures through adaptive rules that learn the underlying parameters while providing strict guarantees on stability, asymptotic performance, and learning. Reinforcement learning methods are applicable to a broad class of systems and are able to produce near-optimal policies for highly complex control tasks. This is often enabled by significant offline training via simulation or the collection of large input-state datasets. This article attempts to compare adaptive control and reinforcement learning using a common framework. The problem statement in each field and highlights of their results are outlined. Two specific examples of dynamic systems are used to illustrate the details of the two methods, their advantages, and their deficiencies. The need for real-time control methods that leverage tools from both approaches is motivated through the lens of this common framework. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 14 is May 2023. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.","PeriodicalId":29961,"journal":{"name":"Annual Review of Control Robotics and Autonomous Systems","volume":null,"pages":null},"PeriodicalIF":11.2000,"publicationDate":"2023-01-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"5","resultStr":"{\"title\":\"Adaptive Control and Intersections with Reinforcement Learning\",\"authors\":\"A. Annaswamy\",\"doi\":\"10.1146/annurev-control-062922-090153\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This article provides an exposition of the field of adaptive control and its intersections with reinforcement learning. Adaptive control and reinforcement learning are two different methods that are both commonly employed for the control of uncertain systems. Historically, adaptive control has excelled at real-time control of systems with specific model structures through adaptive rules that learn the underlying parameters while providing strict guarantees on stability, asymptotic performance, and learning. Reinforcement learning methods are applicable to a broad class of systems and are able to produce near-optimal policies for highly complex control tasks. This is often enabled by significant offline training via simulation or the collection of large input-state datasets. This article attempts to compare adaptive control and reinforcement learning using a common framework. The problem statement in each field and highlights of their results are outlined. Two specific examples of dynamic systems are used to illustrate the details of the two methods, their advantages, and their deficiencies. The need for real-time control methods that leverage tools from both approaches is motivated through the lens of this common framework. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 14 is May 2023. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.\",\"PeriodicalId\":29961,\"journal\":{\"name\":\"Annual Review of Control Robotics and Autonomous Systems\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":11.2000,\"publicationDate\":\"2023-01-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"5\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annual Review of Control Robotics and Autonomous Systems\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1146/annurev-control-062922-090153\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annual Review of Control Robotics and Autonomous Systems","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1146/annurev-control-062922-090153","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 5

摘要

本文阐述了自适应控制领域及其与强化学习的交叉点。自适应控制和强化学习是两种不同的方法,通常用于不确定系统的控制。从历史上看,自适应控制擅长于对具有特定模型结构的系统进行实时控制,通过自适应规则学习底层参数,同时严格保证稳定性、渐近性能和学习。强化学习方法适用于广泛的系统,并且能够为高度复杂的控制任务产生接近最优的策略。这通常通过通过模拟或收集大型输入状态数据集进行重要的离线训练来实现。本文试图使用一个通用的框架来比较自适应控制和强化学习。概述了每个领域的问题陈述及其结果的重点。用两个具体的动态系统的例子来说明这两种方法的细节,它们的优点和缺点。利用这两种方法的工具的实时控制方法的需求是通过这个共同框架的透镜激发的。预计《控制、机器人和自主系统年度评论》第14卷的最终在线出版日期是2023年5月。修订后的估计数请参阅http://www.annualreviews.org/page/journal/pubdates。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Adaptive Control and Intersections with Reinforcement Learning
This article provides an exposition of the field of adaptive control and its intersections with reinforcement learning. Adaptive control and reinforcement learning are two different methods that are both commonly employed for the control of uncertain systems. Historically, adaptive control has excelled at real-time control of systems with specific model structures through adaptive rules that learn the underlying parameters while providing strict guarantees on stability, asymptotic performance, and learning. Reinforcement learning methods are applicable to a broad class of systems and are able to produce near-optimal policies for highly complex control tasks. This is often enabled by significant offline training via simulation or the collection of large input-state datasets. This article attempts to compare adaptive control and reinforcement learning using a common framework. The problem statement in each field and highlights of their results are outlined. Two specific examples of dynamic systems are used to illustrate the details of the two methods, their advantages, and their deficiencies. The need for real-time control methods that leverage tools from both approaches is motivated through the lens of this common framework. Expected final online publication date for the Annual Review of Control, Robotics, and Autonomous Systems, Volume 14 is May 2023. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
28.30
自引率
2.20%
发文量
25
期刊介绍: The Annual Review of Control, Robotics, and Autonomous Systems offers comprehensive reviews on theoretical and applied developments influencing autonomous and semiautonomous systems engineering. Major areas covered include control, robotics, mechanics, optimization, communication, information theory, machine learning, computing, and signal processing. The journal extends its reach beyond engineering to intersect with fields like biology, neuroscience, and human behavioral sciences. The current volume has transitioned to open access through the Subscribe to Open program, with all articles published under a CC BY license.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信