Endowing Spiking Neural Networks with Homeostatic Adaptivity for APS-DVS Bimodal Scenarios

Plastic changes with intrinsic dynamics in synaptic efficacy underlie the cellular level of expression of brain functions regarding multimodal information processing. Among diverse plasticity mechanisms, synaptic scaling exerts indispensable effects on the homeostatic state maintenance and synaptic strength regulation in biological neural networks. Despite recent tremendous progress in developing spiking neural networks (SNNs) for multiple complex scenarios, most of the work remains in the pure backpropagation-based framework where the synaptic scaling mechanism is rarely effectively incorporated. In this work, we present a biologically inspired neuronal model with an activity-dependent adaptive synaptic scaling mechanism that endows each synapse with both short-term enhancement and depression properties. The learning process is completed in two phases. Firstly, in the forward conduction circuits, adaptive short-term enhancement or depression response is triggered in the light of afferent stimuli intensity; Then long-term consolidation is executed by back-propagated error signals. These processes dramatically shape the pattern selectivity of synapses and the diverse information transfer they mediate. Experiments reveal remarkable advantages in three tasks regarding bimodal learning. Specifically, On the continual learning and perturbation-resistant task for Dynamic Vision Sensor (DVS) modal information, our method improves the mean accuracy on the benchmark of N-MNIST dataset than the baseline by and , respectively. On sequence learning task for Active Pixel Sensor (APS) model information, our method improve the generalization capability and training stability by a large margin. These results demonstrate favourable effectiveness of such non-parametric adaptive strategy on bimodal information inference for APS and DVS, facilitating intelligence understanding and bio-inspired modelling.