行为批评型脉冲神经网络中奖励调节的脉冲时间依赖的可塑性和时间差异的长期增强

IF 2.1 3区心理学 Q3 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Cognitive Systems Research Pub Date : 2025-02-06 DOI:10.1016/j.cogsys.2025.101334

Yunes Tihomirov , Roman Rybka , Alexey Serenko , Alexander Sboev

{"title":"行为批评型脉冲神经网络中奖励调节的脉冲时间依赖的可塑性和时间差异的长期增强","authors":"Yunes Tihomirov , Roman Rybka , Alexey Serenko , Alexander Sboev","doi":"10.1016/j.cogsys.2025.101334","DOIUrl":null,"url":null,"abstract":"<div><div>This paper presents a method for training spiking neural networks (SNNs) with the actor–critic architecture. The actor SNN is trained using reward-modulated spike-timing dependent plasticity (RSTDP), and the critic SNN is trained using temporal difference long-term potentiation (TD-LTP). The proposed method achieves competitive performance on the Acrobot and CartPole benchmarks. Due to RSTDP being prospectively suitable for implementation in memristors, this result is a preliminary step towards a fully-spiking actor–critic network deployable to analog neuromorphic devices.</div></div>","PeriodicalId":55242,"journal":{"name":"Cognitive Systems Research","volume":"90 ","pages":"Article 101334"},"PeriodicalIF":2.1000,"publicationDate":"2025-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Combination of reward-modulated spike-timing dependent plasticity and temporal difference long-term potentiation in actor–critic spiking neural network\",\"authors\":\"Yunes Tihomirov , Roman Rybka , Alexey Serenko , Alexander Sboev\",\"doi\":\"10.1016/j.cogsys.2025.101334\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>This paper presents a method for training spiking neural networks (SNNs) with the actor–critic architecture. The actor SNN is trained using reward-modulated spike-timing dependent plasticity (RSTDP), and the critic SNN is trained using temporal difference long-term potentiation (TD-LTP). The proposed method achieves competitive performance on the Acrobot and CartPole benchmarks. Due to RSTDP being prospectively suitable for implementation in memristors, this result is a preliminary step towards a fully-spiking actor–critic network deployable to analog neuromorphic devices.</div></div>\",\"PeriodicalId\":55242,\"journal\":{\"name\":\"Cognitive Systems Research\",\"volume\":\"90 \",\"pages\":\"Article 101334\"},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2025-02-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Cognitive Systems Research\",\"FirstCategoryId\":\"102\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1389041725000142\",\"RegionNum\":3,\"RegionCategory\":\"心理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cognitive Systems Research","FirstCategoryId":"102","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1389041725000142","RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

摘要

提出了一种基于actor-critic结构的尖峰神经网络训练方法。行动者SNN使用奖励调制spike-timing dependent plasticity （RSTDP）进行训练，批评SNN使用temporal difference long-term potentiation （TD-LTP）进行训练。所提出的方法在Acrobot和CartPole基准测试中取得了具有竞争力的性能。由于RSTDP有望在忆阻器中实现，这一结果是迈向可部署到模拟神经形态设备的全尖峰行为批评网络的初步步骤。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Combination of reward-modulated spike-timing dependent plasticity and temporal difference long-term potentiation in actor–critic spiking neural network

This paper presents a method for training spiking neural networks (SNNs) with the actor–critic architecture. The actor SNN is trained using reward-modulated spike-timing dependent plasticity (RSTDP), and the critic SNN is trained using temporal difference long-term potentiation (TD-LTP). The proposed method achieves competitive performance on the Acrobot and CartPole benchmarks. Due to RSTDP being prospectively suitable for implementation in memristors, this result is a preliminary step towards a fully-spiking actor–critic network deployable to analog neuromorphic devices.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Cognitive Systems Research 工程技术-计算机：人工智能

CiteScore

9.40

自引率

5.10%

发文量

审稿时长

>12 weeks

期刊介绍： Cognitive Systems Research is dedicated to the study of human-level cognition. As such, it welcomes papers which advance the understanding, design and applications of cognitive and intelligent systems, both natural and artificial. The journal brings together a broad community studying cognition in its many facets in vivo and in silico, across the developmental spectrum, focusing on individual capacities or on entire architectures. It aims to foster debate and integrate ideas, concepts, constructs, theories, models and techniques from across different disciplines and different perspectives on human-level cognition. The scope of interest includes the study of cognitive capacities and architectures - both brain-inspired and non-brain-inspired - and the application of cognitive systems to real-world problems as far as it offers insights relevant for the understanding of cognition. Cognitive Systems Research therefore welcomes mature and cutting-edge research approaching cognition from a systems-oriented perspective, both theoretical and empirically-informed, in the form of original manuscripts, short communications, opinion articles, systematic reviews, and topical survey articles from the fields of Cognitive Science (including Philosophy of Cognitive Science), Artificial Intelligence/Computer Science, Cognitive Robotics, Developmental Science, Psychology, and Neuroscience and Neuromorphic Engineering. Empirical studies will be considered if they are supplemented by theoretical analyses and contributions to theory development and/or computational modelling studies.