IEEE Transactions on Cognitive and Developmental Systems最新文献_第3页

CMFC-MOT: Contextual Multimodal Fusion With Centroid-Aware Query for 3-D Multiobject Tracking CMFC-MOT：基于质心感知查询的上下文多模态融合三维多目标跟踪

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-10-01 Epub Date: 2026-03-05 DOI: 10.1109/TCDS.2026.3670666

Xiaoxue Ai;Hui Li;Xiangzhen Kong;Qinghua Lin;Wentao Chen;Tongao Ge

{"title":"CMFC-MOT: Contextual Multimodal Fusion With Centroid-Aware Query for 3-D Multiobject Tracking","authors":"Xiaoxue Ai;Hui Li;Xiangzhen Kong;Qinghua Lin;Wentao Chen;Tongao Ge","doi":"10.1109/TCDS.2026.3670666","DOIUrl":"https://doi.org/10.1109/TCDS.2026.3670666","url":null,"abstract":"3-D multiobject tracking is a core perception task for autonomous driving, yet it remains challenging in complex scenarios such as occlusion and dense interactions. Existing methods do not fully exploit the complementary cues of LiDAR geometry and image semantics, and cross-modal interactions are often coarse-grained, leading to insufficient object feature representations. Moreover, bird’s eye view (BEV) representations can suffer from detail loss, which hinders stable object perception. In the data association stage, cost modeling is typically simplistic and fails to leverage interobject differences, thereby limiting overall tracking robustness. To address these challenges, we propose a cognition-inspired framework, termed contextual multimodal fusion with centroid-aware query for 3-D multiobject tracking. It enhances the representation of key target features and improves association stability. First, we design a contextual aggregation with centroid-aware query module. It reconstructs centroid voxels and guides context aggregation. This achieves fine-grained cross-modal feature interaction and enhances the mining of multimodal information. Subsequently, we introduce a multimodal adaptive hierarchical fusion module. It employs a dual-domain attention mechanism across spatial and channel dimensions to emphasize salient target features. This improves discriminative performance in complex scenes. Finally, we employ the cascaded confidence-aware multicost association module to construct a more discriminative association matrix. Based on target quality, we select the optimal association strategy to reduce mismatches and enhance tracking robustness in occlusion and dense scenes. On the KITTI test dataset, our method achieves 81.04% HOTA and 89.46% MOTA on the car category, demonstrating superior tracking performance.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 5","pages":"1227-1240"},"PeriodicalIF":4.7,"publicationDate":"2026-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149176318","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Contextual Semantic Relevance Predicting Human Visual Attention 上下文语义关联预测人类视觉注意

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-10-01 Epub Date: 2026-04-27 DOI: 10.1109/TCDS.2026.3683873

Kun Sun;Rong Wang

{"title":"Contextual Semantic Relevance Predicting Human Visual Attention","authors":"Kun Sun;Rong Wang","doi":"10.1109/TCDS.2026.3683873","DOIUrl":"https://doi.org/10.1109/TCDS.2026.3683873","url":null,"abstract":"Semantic relevance metrics capture not only the inherent meaning of individual objects but also their relationships to other elements in a visual scene. Prior research has shown their importance in predicting human visual processing. However, many studies have overlooked contextual information or failed to employ advances in deep learning for more accurate computation. This study introduces the metrics of contextual semantic relevance to investigate human visual perception and processing. We compute semantic relationships between target objects and their surroundings from both vision-based and language-based perspectives. Using a large eye-movement dataset on visual comprehension, we apply state-of-the-art deep learning techniques to derive these metrics and examine their effects on fixation patterns with advanced statistical models. The metrics we proposed also simulate both top-down and bottom-up processes in visual perception. Furthermore, we integrate vision-based and language-based measures into a novel combined metric, addressing a critical limitation of previous research that treated visual and semantic similarities separately. Results show that all metrics accurately predict fixation behavior, though with distinct predictive roles. The combined metric outperforms the individual ones, supporting theories that highlight the interaction between semantic and visual information in influencing human visual processing. These findings advance our understanding of the cognitive mechanisms underlying visual processing and inform the development of more accurate computational models in cognitive science and human–computer interaction.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 5","pages":"1355-1368"},"PeriodicalIF":4.7,"publicationDate":"2026-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149176319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HumanoidMamba: Generalized Mamba-Based Policy Learning With Next-Action Prediction for Humanoid Locomotion HumanoidMamba：基于广义mamba的人形运动下一动作预测策略学习

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-10-01 Epub Date: 2026-03-30 DOI: 10.1109/TCDS.2026.3678742

Yi Zhou;Jianbin Qiu;Wei Zhang;Fenglei Ni

{"title":"HumanoidMamba: Generalized Mamba-Based Policy Learning With Next-Action Prediction for Humanoid Locomotion","authors":"Yi Zhou;Jianbin Qiu;Wei Zhang;Fenglei Ni","doi":"10.1109/TCDS.2026.3678742","DOIUrl":"https://doi.org/10.1109/TCDS.2026.3678742","url":null,"abstract":"In various human-centered unstructured environments, humanoid robots tend to exhibit superior adaptability and operational performance relative to other types of robots. Existing approaches to whole-body control in humanoid robots can be primarily composed of model-based and learning-based methods. While model-based methods rely heavily on manual design, current state-of-the-art (SOTA) learning-based methods employ Transformers as backbone networks, often leading to suboptimal computational efficiency. Inspired by the demonstrated superiority of Mamba over Transformers in large language models (LLMs) and computer vision domains, HumanoidMamba is proposed as a framework that leverages Mamba as the core backbone of the policy network to generate humanoid locomotion. Mamba, serving as the backbone, improves computational efficiency compared to Transformers while maintaining the capacity of the model for content-based reasoning. In addition, analogous to next-token prediction in LLMs, a modality-aligned prediction approach is introduced for generating the next action in the policy network, referred to as next-action prediction. The modality-aligned next-action prediction approach establishes a paradigm that not only enhances the understanding of temporal information by the policy but also facilitates extension to handle trajectories with missing modalities. Experimental results substantiate that the proposed HumanoidMamba consistently generates humanoid locomotion characterized by exceptional stability and robustness. Beyond surpassing existing SOTA methods across multiple performance metrics, HumanoidMamba further unveils remarkable generalization capability by adapting to velocity commands that are absent from the expert demonstration trajectories.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 5","pages":"1292-1304"},"PeriodicalIF":4.7,"publicationDate":"2026-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149176145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Aligned Pseudo Feature Generation for Zero-Shot Object Detection 零射击目标检测的对齐伪特征生成

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-08-01 Epub Date: 2025-10-27 DOI: 10.1109/TCDS.2025.3625782

Xinmiao Dai;Zhehao Li;Chong Wang;Yinghao Lu;Yi Chen;Jiangbo Qian;Jiafei Wu

{"title":"Aligned Pseudo Feature Generation for Zero-Shot Object Detection","authors":"Xinmiao Dai;Zhehao Li;Chong Wang;Yinghao Lu;Yi Chen;Jiangbo Qian;Jiafei Wu","doi":"10.1109/TCDS.2025.3625782","DOIUrl":"https://doi.org/10.1109/TCDS.2025.3625782","url":null,"abstract":"The goal of zero-shot object detection (ZSD) is to localize and classify objects that have not been presented in the training set. Drawing inspiration from the success of generative models, many feature generation-based approaches have been explored for the ZSD problem, which have had some promising results. However, the absence of visual samples from unseen classes inevitably leads to subpar synthesized features, thereby constraining the generator’s efficacy in prior approaches. This limitation highlights a key challenge in current ZSD methods: generating discriminative and realistic visual features without real data support. To address this, our method is motivated by the insight that semantic embeddings alone are insufficient for generating high-quality features. Instead, we align them with pseudovisual features extracted from a diffusion model, which provides more visually plausible guidance and significantly improves the quality of synthesized features for unseen categories. Additionally, unlike previous works, the real visual features of seen classes are also used to train the unseen classifier, which is considered as the background class. Such a trained unseen classifier has less chance to misclassify a seen object into an unseen category. While working with the pretrained seen classifier, it performs better in the real-world application, i.e., the generalized ZSD situation. Extensive experiments on MS COCO and PASCAL VOC demonstrate that our method achieves state-of-the-art performance.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 4","pages":"1013-1027"},"PeriodicalIF":4.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148710202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning Expressive Task Embeddings and Sample-Efficient Exploration for Context Shift Reduction in Offline Meta-Reinforcement Environment 学习表达任务嵌入与离线元强化环境下上下文移位减少的样本高效探索

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-08-01 Epub Date: 2025-10-31 DOI: 10.1109/TCDS.2025.3627812

Haoyu Wang;Xiaoyu Tan;Tianliang Yao;Zhijun Fang;Peng Qi;Xihe Qiu

{"title":"Learning Expressive Task Embeddings and Sample-Efficient Exploration for Context Shift Reduction in Offline Meta-Reinforcement Environment","authors":"Haoyu Wang;Xiaoyu Tan;Tianliang Yao;Zhijun Fang;Peng Qi;Xihe Qiu","doi":"10.1109/TCDS.2025.3627812","DOIUrl":"https://doi.org/10.1109/TCDS.2025.3627812","url":null,"abstract":"Context shift reduction for offline meta-reinforcement learning (CSRO) has made notable progress in addressing the context shift problem arising from the distribution mismatch between training and testing data. However, limitations remain in capturing rich task representations and efficiently exploring them during meta-testing. We propose two key enhancements: First, replacing the simple mean context encoding with an expressive attention-based mechanism to selectively focus on relevant transitions and learn intricate task representations for improved inference. Second, introducing an adaptive weighted prior exploration strategy that constructs a belief over possible tasks and iteratively refines this belief by weighting simulated trajectory rollouts under each hypothesis, enabling sample-efficient adaptation. Unlike random exploration, this strategy accelerates task representation updates by leveraging trajectory simulations under each task hypothesis. Through comprehensive experiments on diverse robotic control benchmarks with varying rewards and dynamics, we demonstrate significant performance gains over the original CSRO and surpass prior offline meta-RL methods. Our enhanced model highlights the importance of principled representation learning and exploration techniques in mitigating context shifts using only offline data while minimizing sample complexity during fast adaptation to new tasks.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 4","pages":"888-903"},"PeriodicalIF":4.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148710214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Developmentally Staged Transformer Framework for Progressive Cognitive Learning in Language Understanding 语言理解中渐进式认知学习的发展阶段转换框架

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-08-01 Epub Date: 2025-11-12 DOI: 10.1109/TCDS.2025.3630210

V. Jothi Prakash;S. Arul Antran Vijay;Gopikrishnan Sundaram;Ganeshkumar Pugalendhi

{"title":"A Developmentally Staged Transformer Framework for Progressive Cognitive Learning in Language Understanding","authors":"V. Jothi Prakash;S. Arul Antran Vijay;Gopikrishnan Sundaram;Ganeshkumar Pugalendhi","doi":"10.1109/TCDS.2025.3630210","DOIUrl":"https://doi.org/10.1109/TCDS.2025.3630210","url":null,"abstract":"Human language understanding develops progressively through stages of cognitive maturation, characterized by increasing memory capacity, attentional control, and abstraction. Inspired by this trajectory, we propose the developmentally staged transformer for progressive cognitive learning (DST-PCL), a novel framework that models language understanding through staged architectural and functional growth. DST-PCL simulates cognitive development by incrementally expanding transformer depth, input memory span, and attention complexity across a curriculum-aligned training schedule. Each stage activates a subset of model layers and imposes entropy-regularized attention constraints to reflect bounded human focus. A readiness-based transition controller governs stage advancement based on validation accuracy and attention sharpness. Theoretical analyses guarantee hypothesis class expansion, bounded generalization error, and stability under stage transitions. Empirical evaluations across three benchmark datasets—BoolQ, MultiNLI, and ReClor—demonstrate consistent gains over strong baselines, with up to +5.1% accuracy and +6.3 F1 improvement. Further, attention entropy and interpretability metrics confirm the cognitively plausible progression of model behavior. DST-PCL offers a new paradigm for cognitively grounded natural language processing (NLP) systems that unify performance, developmental structure, and interpretability.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 4","pages":"997-1012"},"PeriodicalIF":4.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11244271","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148710248","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Spiking Neural Network Feature Discrimination Boosts Modality Fusion 尖峰神经网络特征判别促进情态融合

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-08-01 Epub Date: 2025-12-11 DOI: 10.1109/TCDS.2025.3642931

Katerina Maria Oikonomou;Ioannis Kansizoglou;Antonios Gasteratos

{"title":"Spiking Neural Network Feature Discrimination Boosts Modality Fusion","authors":"Katerina Maria Oikonomou;Ioannis Kansizoglou;Antonios Gasteratos","doi":"10.1109/TCDS.2025.3642931","DOIUrl":"https://doi.org/10.1109/TCDS.2025.3642931","url":null,"abstract":"Feature discrimination is a crucial aspect of neural network design, as it directly impacts the network’s ability to distinguish between classes and generalize across diverse datasets. The accomplishment of achieving high-quality feature representations ensures high intra-class separability and poses one of the most challenging research directions. While conventional deep neural networks (DNNs) rely on complex transformations and very deep networks to come up with meaningful feature representations, they usually require days of training and consume significant energy amounts. To this end, spiking neural networks (SNNs) offer a promising alternative. SNN’s ability to capture temporal and spatial dependencies renders them particularly suitable for complex tasks, where multimodal data are required. In this article, we propose a feature discrimination approach for multimodal learning with SNNs, focusing on audio–visual data. We employ deep spiking residual learning for visual modality processing and a simpler yet efficient spiking network for auditory modality processing. Last, we deploy a spiking multilayer perceptron for modality fusion. We present our findings and evaluate our approach against similar works in the field of classification challenges. To the best of our knowledge, this is the first work investigating feature discrimination in SNNs.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 4","pages":"943-955"},"PeriodicalIF":4.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148710265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FIT: Future-Aware Imitation Transformer for Long-Horizon Robotic Manipulation FIT：用于长视界机器人操作的未来感知模拟变压器

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-08-01 Epub Date: 2025-12-16 DOI: 10.1109/TCDS.2025.3644126

Yasir Salam;Yinbei Li;Rafik Ighil Nessouk;Jiaqiang Yang

{"title":"FIT: Future-Aware Imitation Transformer for Long-Horizon Robotic Manipulation","authors":"Yasir Salam;Yinbei Li;Rafik Ighil Nessouk;Jiaqiang Yang","doi":"10.1109/TCDS.2025.3644126","DOIUrl":"https://doi.org/10.1109/TCDS.2025.3644126","url":null,"abstract":"Developing a generalist agent capable of performing multiple long-horizon tasks and solving novel ones remains an open challenge in robotic learning. Current imitation learning methods show promise in handling individual long-horizon tasks, but they struggle to scale to multitask scenarios and fail to execute new tasks. This work proposes a framework that learns a unified imitation policy capable of generalizing to diverse interaction-rich tasks and environmental variations. Our approach introduces a task decomposition strategy that leverages a large language model (LLM) and robot proprioception to infer agent–object interaction events from expert demonstrations. With these interaction events as input, we introduce the future-aware imitation transformer (FIT), a transformer-based policy that acquires interaction-aware skills. At every time step, FIT predicts the actions, the latent representations of the future states to anticipate the outcomes of actions, and a task embedding to maintain awareness of the long-horizon manipulation objective. We evaluated our method in simulated LIBERO, Franka kitchen, and real-world xARM6 environments. Experimental results demonstrate that our method outperforms all baselines, particularly on novel tasks, where existing methods fail to generalize.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 4","pages":"1043-1056"},"PeriodicalIF":4.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148710190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PlanAgent: A Multimodal Large Language Agent for Closed-Loop Vehicle Motion Planning PlanAgent：用于闭环车辆运动规划的多模态大语言智能体

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-08-01 Epub Date: 2026-02-12 DOI: 10.1109/TCDS.2026.3664120

Yupeng Zheng;Zebin Xing;Qichao Zhang;Bu Jin;Pengfei Li;Yuhang Zheng;Zhongpu Xia;Yaran Chen;Dongbin Zhao

{"title":"PlanAgent: A Multimodal Large Language Agent for Closed-Loop Vehicle Motion Planning","authors":"Yupeng Zheng;Zebin Xing;Qichao Zhang;Bu Jin;Pengfei Li;Yuhang Zheng;Zhongpu Xia;Yaran Chen;Dongbin Zhao","doi":"10.1109/TCDS.2026.3664120","DOIUrl":"https://doi.org/10.1109/TCDS.2026.3664120","url":null,"abstract":"Vehicle motion planning. is an essential component of autonomous driving technology. Current rule-based vehicle motion planning methods perform satisfactorily in common scenarios but struggle to generalize to long-tailed situations. Meanwhile, learning-based methods have yet to achieve superior performance over rule-based approaches in large-scale closed-loop scenarios. To address these issues, we propose PlanAgent, the first mid-to-mid planning system based on a multimodal large language model (MLLM). MLLM is used as a cognitive agent to introduce humanlike knowledge, interpretability, and common-sense reasoning into closed-loop planning. Specifically, PlanAgent leverages the power of MLLM through three core modules. First, an environment transformation module constructs a bird’s eye view (BEV) map and a lane-graph-based textual description from the environment as inputs. Second, a reasoning engine module introduces a hierarchical chain-of-thought from scene understanding to lateral and longitudinal motion instructions, culminating in planner code generation. Last, a reflection module is integrated to simulate and evaluate the generated planner for reducing MLLM’s uncertainty. PlanAgent is endowed with the common-sense reasoning and generalization capability of MLLM, which empowers it to effectively tackle both common and complex long-tailed scenarios. Our proposed PlanAgent is evaluated on the large-scale and challenging nuPlan benchmarks. A comprehensive set of experiments convincingly demonstrates that PlanAgent outperforms the existing state-of-the-art in the closed-loop motion planning task","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 4","pages":"1113-1126"},"PeriodicalIF":4.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148710207","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Object Detection With Spiking Neural Networks on Event Videos for Driving Scenarios 基于脉冲神经网络的驾驶场景事件视频目标检测

IF 4.7 3区计算机科学

IEEE Transactions on Cognitive and Developmental Systems Pub Date : 2026-08-01 Epub Date: 2026-02-20 DOI: 10.1109/TCDS.2026.3666360

Shengyu Gao;Ying Tao;Amelie Gruel;Sylvain Saïghi;Chip-Hong Chang

{"title":"Object Detection With Spiking Neural Networks on Event Videos for Driving Scenarios","authors":"Shengyu Gao;Ying Tao;Amelie Gruel;Sylvain Saïghi;Chip-Hong Chang","doi":"10.1109/TCDS.2026.3666360","DOIUrl":"https://doi.org/10.1109/TCDS.2026.3666360","url":null,"abstract":"Event cameras excel in imaging objects under challenging lighting conditions with low latency. Their sparse and asynchronous event outputs closely align with the operational paradigm of spiking neural networks (SNNs). An SNN mimics biological neurons of brain by transmitting information via binary spikes. The combination of biologically inspired event-based sensing with neuromorphic computing, while individually promising, faces significant challenges in their coherent integration to deliver better performance than existing solutions for real-time multiscale, multirate object detection problems. In this article, we propose a new spike tensor-based multiscale object detector (STMOD) for event video analytics in driving scenarios. STMOD features a cross-attention spatio-temporal spikes generator to convert asynchronous events into a high-dimension binary vector, a spiking backbone with three pyramid feature levels to fuse multiscale spatio-temporal features and an asymmetric decoupled detection head to achieve high precision, energy efficient and responsive object detection without compromising the temporal resolution. Additionally, a new training method is proposed to overcome the mismatch in SNN’s state between training with independent samples and testing with video streaming to improve its performance on continuous video feed without impacting inference time resolution. Experimental results on the GEN1 and DSEC-detection datasets show that STMOD achieves state-of-the-art mAP of 44.3% and 34.2%, respectively, with an inference time resolution of 5 ms. Finally, it also rivals existing event-based object detectors with significantly lower number of parameters and higher energy efficiency.","PeriodicalId":54300,"journal":{"name":"IEEE Transactions on Cognitive and Developmental Systems","volume":"18 4","pages":"1083-1098"},"PeriodicalIF":4.7,"publicationDate":"2026-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148710227","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0