Multi-dimensional Attention Spiking Transformer for Event-based Image Classification

Abstract

Image classification is a vital research area in deep learning. However, the use of Artificial Neural Networks (ANNs) in conventional approaches requires vast computational power and memory. As a potential energy-efficient alternative, Spiking Neural Networks (SNNs) leverage temporal information and low-power sensors. Nonetheless, extracting spatio-temporal features from event-based image sequences for improved classification accuracies in SNNs poses a significant challenge. To address this, we propose a Multi-Dimensional Attention Spiking Transformer (MAST) model that integrates attention mechanisms and SNNs to capture spatio-temporal features in event-based image sequences. Consequently, the MAST model achieves state-of-the-art performance in various classification tasks, as shown by the evaluations on the CIFAR, DVS128 Gesture, and CIFAR10-DVS datasets. Overall, MAST exhibits promise in event-based image classification tasks, providing a new perspective on the integration of attention mechanisms and SNNs for improved image classification.