Traffic signal optimization control method based on attention mechanism updated weights double deep Q network

IF 4.6 2区 计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Huizhen Zhang, Zhenwei Fang, Youqing Chen, Haotian Dai, Qi Jiang, Xinyan Zeng
{"title":"Traffic signal optimization control method based on attention mechanism updated weights double deep Q network","authors":"Huizhen Zhang, Zhenwei Fang, Youqing Chen, Haotian Dai, Qi Jiang, Xinyan Zeng","doi":"10.1007/s40747-025-01841-9","DOIUrl":null,"url":null,"abstract":"<p>As a critical guidance facility for vehicle convergence and diversion in urban traffic networks, the control effect of traffic signals directly affects traffic efficiency and road congestion level. As a mature deep reinforcement learning algorithm, the double deep Q network has shown a significant optimization effect in intelligent traffic signal control research. In this paper, for the feature extraction defects of deep double Q network and the problem of underestimating the evaluation value of actions, we propose an Attention Mechanism Updated Weights Double Deep Q Network (AMUW–DDQN) based on the attention mechanism for the optimal control of traffic signals. The AMUW–DDQN method enhances the perceptual ability of the network by introducing the attention mechanism of Squeeze And Excitation Networks (SENet) to make the neural network pay attention to important state components automatically, and based on the idea that accurate representation of potentially optimal action values is better than the balanced representation of all the action values, it is considered that underestimated actions have a certain probability of being the optimal action and the loss function is weighted to optimize the action values. Simulation experiments were also conducted using the traffic flow data of the intersection of Fengze Street–Tian’an South Road, Fengze District, Quanzhou City, Fujian Province, China. The experimental results show that the method proposed in this paper has the most significant final convergence effect for the same number of iterations, and has better performance in the evaluation indexes such as vehicle queue length and vehicle delay time.</p>","PeriodicalId":10524,"journal":{"name":"Complex & Intelligent Systems","volume":"21 1","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Complex & Intelligent Systems","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s40747-025-01841-9","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

As a critical guidance facility for vehicle convergence and diversion in urban traffic networks, the control effect of traffic signals directly affects traffic efficiency and road congestion level. As a mature deep reinforcement learning algorithm, the double deep Q network has shown a significant optimization effect in intelligent traffic signal control research. In this paper, for the feature extraction defects of deep double Q network and the problem of underestimating the evaluation value of actions, we propose an Attention Mechanism Updated Weights Double Deep Q Network (AMUW–DDQN) based on the attention mechanism for the optimal control of traffic signals. The AMUW–DDQN method enhances the perceptual ability of the network by introducing the attention mechanism of Squeeze And Excitation Networks (SENet) to make the neural network pay attention to important state components automatically, and based on the idea that accurate representation of potentially optimal action values is better than the balanced representation of all the action values, it is considered that underestimated actions have a certain probability of being the optimal action and the loss function is weighted to optimize the action values. Simulation experiments were also conducted using the traffic flow data of the intersection of Fengze Street–Tian’an South Road, Fengze District, Quanzhou City, Fujian Province, China. The experimental results show that the method proposed in this paper has the most significant final convergence effect for the same number of iterations, and has better performance in the evaluation indexes such as vehicle queue length and vehicle delay time.

基于注意机制更新权值的双深度Q网络交通信号优化控制方法
交通信号作为城市交通网络中车辆汇聚和分流的重要引导设施,其控制效果直接影响交通效率和道路拥堵程度。双深度Q网络作为一种成熟的深度强化学习算法,在智能交通信号控制研究中显示出显著的优化效果。本文针对深度双Q网络特征提取缺陷和低估动作评价值的问题,提出了一种基于注意机制的交通信号最优控制关注机制更新权值双深度Q网络(AMUW-DDQN)。AMUW-DDQN方法通过引入挤压激励网络(SENet)的注意机制,使神经网络自动关注重要的状态分量,增强了网络的感知能力,并基于潜在最优动作值的准确表示优于所有动作值的平衡表示的思想。认为低估的动作有一定的概率成为最优动作,并对损失函数进行加权以优化动作值。利用福建省泉州市丰泽区丰泽街-天安南路交叉口交通流数据进行了仿真实验。实验结果表明,在相同迭代次数下,本文提出的方法具有最显著的最终收敛效果,并且在车辆排队长度和车辆延误时间等评价指标上具有更好的性能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Complex & Intelligent Systems
Complex & Intelligent Systems COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE-
CiteScore
9.60
自引率
10.30%
发文量
297
期刊介绍: Complex & Intelligent Systems aims to provide a forum for presenting and discussing novel approaches, tools and techniques meant for attaining a cross-fertilization between the broad fields of complex systems, computational simulation, and intelligent analytics and visualization. The transdisciplinary research that the journal focuses on will expand the boundaries of our understanding by investigating the principles and processes that underlie many of the most profound problems facing society today.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信