考虑相邻干线交通的q -学习综合高速公路交通控制

IF 8.4 1区 工程技术 Q1 ENGINEERING, CIVIL
Tianchen Yuan;Petros A. Ioannou
{"title":"考虑相邻干线交通的q -学习综合高速公路交通控制","authors":"Tianchen Yuan;Petros A. Ioannou","doi":"10.1109/TITS.2025.3559893","DOIUrl":null,"url":null,"abstract":"Numerous studies have shown the effectiveness of intelligent transportation system techniques such as variable speed limit (VSL), lane change (LC) control, and ramp metering (RM) in freeway traffic flow control. The integration of these techniques has the potential to further enhance the traffic operation efficiency of both freeway and adjacent arterial networks. In this regard, we propose a freeway traffic control (FTC) strategy that coordinates VSL, LC, RM actions using a Q-learning (QL) framework which takes into account arterial traffic characteristics. The signal timing and demands of adjacent arterial intersections are incorporated as state variables of the FTC agent. The FTC agent is initially trained offline using a single-section road network, and subsequently deployed online in a connected freeway and arterial simulation network for continuous learning. The arterial network is assumed to be regulated by a traffic-responsive signal control strategy based on a cycle length model. Microscopic simulations demonstrate that the fully-trained FTC agent provides significant reductions in freeway travel time and the number of stops in scenarios with traffic congestion. It clearly outperforms an uncoordinated FTC and a decentralized feedback control strategy. Even though the FTC agent does not control the arterial traffic signals, it leads to shorter average queue lengths at arterial intersections by taking into account the arterial traffic conditions in controlling freeway traffic. These results motivate a future research where the QL framework will also include the control of arterial traffic signals.","PeriodicalId":13416,"journal":{"name":"IEEE Transactions on Intelligent Transportation Systems","volume":"26 6","pages":"7655-7666"},"PeriodicalIF":8.4000,"publicationDate":"2025-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Integrated Freeway Traffic Control Using Q-Learning With Adjacent Arterial Traffic Considerations\",\"authors\":\"Tianchen Yuan;Petros A. Ioannou\",\"doi\":\"10.1109/TITS.2025.3559893\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Numerous studies have shown the effectiveness of intelligent transportation system techniques such as variable speed limit (VSL), lane change (LC) control, and ramp metering (RM) in freeway traffic flow control. The integration of these techniques has the potential to further enhance the traffic operation efficiency of both freeway and adjacent arterial networks. In this regard, we propose a freeway traffic control (FTC) strategy that coordinates VSL, LC, RM actions using a Q-learning (QL) framework which takes into account arterial traffic characteristics. The signal timing and demands of adjacent arterial intersections are incorporated as state variables of the FTC agent. The FTC agent is initially trained offline using a single-section road network, and subsequently deployed online in a connected freeway and arterial simulation network for continuous learning. The arterial network is assumed to be regulated by a traffic-responsive signal control strategy based on a cycle length model. Microscopic simulations demonstrate that the fully-trained FTC agent provides significant reductions in freeway travel time and the number of stops in scenarios with traffic congestion. It clearly outperforms an uncoordinated FTC and a decentralized feedback control strategy. Even though the FTC agent does not control the arterial traffic signals, it leads to shorter average queue lengths at arterial intersections by taking into account the arterial traffic conditions in controlling freeway traffic. These results motivate a future research where the QL framework will also include the control of arterial traffic signals.\",\"PeriodicalId\":13416,\"journal\":{\"name\":\"IEEE Transactions on Intelligent Transportation Systems\",\"volume\":\"26 6\",\"pages\":\"7655-7666\"},\"PeriodicalIF\":8.4000,\"publicationDate\":\"2025-04-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Intelligent Transportation Systems\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10974412/\",\"RegionNum\":1,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, CIVIL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Intelligent Transportation Systems","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10974412/","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, CIVIL","Score":null,"Total":0}
引用次数: 0

摘要

大量研究表明,可变限速(VSL)、变道(LC)控制和匝道计量(RM)等智能交通系统技术在高速公路交通流控制中的有效性。这些技术的整合有可能进一步提高高速公路和邻近主干道网络的交通运行效率。在这方面,我们提出了一种高速公路交通控制(FTC)策略,该策略使用q -学习(QL)框架来协调VSL, LC, RM动作,该框架考虑了干线交通特征。将相邻干道交叉口的信号时序和需求作为FTC代理的状态变量。FTC代理最初使用单路段道路网络进行离线训练,随后在连接的高速公路和动脉模拟网络中进行在线部署,以进行持续学习。假设主干道网络由基于周期长度模型的交通响应信号控制策略调节。微观模拟表明,在交通拥堵的情况下,训练有素的FTC代理可以显著减少高速公路行驶时间和停车次数。它明显优于不协调的联邦贸易委员会和分散的反馈控制策略。尽管联邦贸易委员会代理人不控制干线交通信号,但在控制高速公路交通时,由于考虑了干线交通状况,使得干线交叉口的平均排队长度缩短。这些结果激发了未来的研究,QL框架也将包括动脉交通信号的控制。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Integrated Freeway Traffic Control Using Q-Learning With Adjacent Arterial Traffic Considerations
Numerous studies have shown the effectiveness of intelligent transportation system techniques such as variable speed limit (VSL), lane change (LC) control, and ramp metering (RM) in freeway traffic flow control. The integration of these techniques has the potential to further enhance the traffic operation efficiency of both freeway and adjacent arterial networks. In this regard, we propose a freeway traffic control (FTC) strategy that coordinates VSL, LC, RM actions using a Q-learning (QL) framework which takes into account arterial traffic characteristics. The signal timing and demands of adjacent arterial intersections are incorporated as state variables of the FTC agent. The FTC agent is initially trained offline using a single-section road network, and subsequently deployed online in a connected freeway and arterial simulation network for continuous learning. The arterial network is assumed to be regulated by a traffic-responsive signal control strategy based on a cycle length model. Microscopic simulations demonstrate that the fully-trained FTC agent provides significant reductions in freeway travel time and the number of stops in scenarios with traffic congestion. It clearly outperforms an uncoordinated FTC and a decentralized feedback control strategy. Even though the FTC agent does not control the arterial traffic signals, it leads to shorter average queue lengths at arterial intersections by taking into account the arterial traffic conditions in controlling freeway traffic. These results motivate a future research where the QL framework will also include the control of arterial traffic signals.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Intelligent Transportation Systems
IEEE Transactions on Intelligent Transportation Systems 工程技术-工程:电子与电气
CiteScore
14.80
自引率
12.90%
发文量
1872
审稿时长
7.5 months
期刊介绍: The theoretical, experimental and operational aspects of electrical and electronics engineering and information technologies as applied to Intelligent Transportation Systems (ITS). Intelligent Transportation Systems are defined as those systems utilizing synergistic technologies and systems engineering concepts to develop and improve transportation systems of all kinds. The scope of this interdisciplinary activity includes the promotion, consolidation and coordination of ITS technical activities among IEEE entities, and providing a focus for cooperative activities, both internally and externally.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信