{"title":"Memristive leaky integrate-and-fire neuron and learnable straight-through estimator in spiking neural networks","authors":"Tao Chen, Chunyan She, Lidan Wang, Shukai Duan","doi":"10.1007/s11571-024-10133-w","DOIUrl":null,"url":null,"abstract":"<p>Compared to artificial neural networks (ANNs), spiking neural networks (SNNs) present a more biologically plausible model of neural system dynamics. They rely on sparse binary spikes to communicate information and operate in an asynchronous, event-driven manner. Despite the high heterogeneity of the neural system at the neuronal level, most current SNNs employ the widely used leaky integrate-and-fire (LIF) neuron model, which assumes uniform membrane-related parameters throughout the entire network. This approach hampers the expressiveness of spiking neurons and restricts the diversity of neural dynamics. In this paper, we propose replacing the resistor in the LIF model with a discrete memristor to obtain the heterogeneous memristive LIF (MLIF) model. The memristance of the discrete memristor is determined by the voltage and flux at its terminals, leading to dynamic changes in the membrane time parameter of the MLIF model. SNNs composed of MLIF neurons can not only learn synaptic weights but also adaptively change membrane time parameters according to the membrane potential of the neuron, enhancing the learning ability and expression of SNNs. Furthermore, since the proper threshold of spiking neurons can improve the information capacity of SNNs, a learnable straight-through estimator (LSTE) is proposed. The LSTE, based on the straight-through estimator (STE) surrogate function, features a learnable threshold that facilitates the backward propagation of gradients through neurons firing spikes. Extensive experiments on several popular static and neuromorphic benchmark datasets demonstrate the effectiveness of the proposed MLIF and LSTE, especially on the DVS-CIFAR10 dataset, where we achieved the top-1 accuracy of 84.40<span>\\(\\%\\)</span>.</p>","PeriodicalId":10500,"journal":{"name":"Cognitive Neurodynamics","volume":null,"pages":null},"PeriodicalIF":3.1000,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cognitive Neurodynamics","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1007/s11571-024-10133-w","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"NEUROSCIENCES","Score":null,"Total":0}
引用次数: 0
Abstract
Compared to artificial neural networks (ANNs), spiking neural networks (SNNs) present a more biologically plausible model of neural system dynamics. They rely on sparse binary spikes to communicate information and operate in an asynchronous, event-driven manner. Despite the high heterogeneity of the neural system at the neuronal level, most current SNNs employ the widely used leaky integrate-and-fire (LIF) neuron model, which assumes uniform membrane-related parameters throughout the entire network. This approach hampers the expressiveness of spiking neurons and restricts the diversity of neural dynamics. In this paper, we propose replacing the resistor in the LIF model with a discrete memristor to obtain the heterogeneous memristive LIF (MLIF) model. The memristance of the discrete memristor is determined by the voltage and flux at its terminals, leading to dynamic changes in the membrane time parameter of the MLIF model. SNNs composed of MLIF neurons can not only learn synaptic weights but also adaptively change membrane time parameters according to the membrane potential of the neuron, enhancing the learning ability and expression of SNNs. Furthermore, since the proper threshold of spiking neurons can improve the information capacity of SNNs, a learnable straight-through estimator (LSTE) is proposed. The LSTE, based on the straight-through estimator (STE) surrogate function, features a learnable threshold that facilitates the backward propagation of gradients through neurons firing spikes. Extensive experiments on several popular static and neuromorphic benchmark datasets demonstrate the effectiveness of the proposed MLIF and LSTE, especially on the DVS-CIFAR10 dataset, where we achieved the top-1 accuracy of 84.40\(\%\).
期刊介绍:
Cognitive Neurodynamics provides a unique forum of communication and cooperation for scientists and engineers working in the field of cognitive neurodynamics, intelligent science and applications, bridging the gap between theory and application, without any preference for pure theoretical, experimental or computational models.
The emphasis is to publish original models of cognitive neurodynamics, novel computational theories and experimental results. In particular, intelligent science inspired by cognitive neuroscience and neurodynamics is also very welcome.
The scope of Cognitive Neurodynamics covers cognitive neuroscience, neural computation based on dynamics, computer science, intelligent science as well as their interdisciplinary applications in the natural and engineering sciences. Papers that are appropriate for non-specialist readers are encouraged.
1. There is no page limit for manuscripts submitted to Cognitive Neurodynamics. Research papers should clearly represent an important advance of especially broad interest to researchers and technologists in neuroscience, biophysics, BCI, neural computer and intelligent robotics.
2. Cognitive Neurodynamics also welcomes brief communications: short papers reporting results that are of genuinely broad interest but that for one reason and another do not make a sufficiently complete story to justify a full article publication. Brief Communications should consist of approximately four manuscript pages.
3. Cognitive Neurodynamics publishes review articles in which a specific field is reviewed through an exhaustive literature survey. There are no restrictions on the number of pages. Review articles are usually invited, but submitted reviews will also be considered.