Microsaccades for asynchronous feature extraction with spiking networks

Abstract

While extracting spatial features from images has been studied for decades, extracting spatio-temporal features from event streams is still a young field of research. A particularity of event streams is that the same network architecture can be used for recognition of static objects or motions. However, it is not clear what features provide a good abstraction and in what scenario. In this paper, we evaluate the quality of the features of a spiking HMAX architecture by computing classification performance before and after each layer. We demonstrate the abstraction capability of classical edge features, as were found in the V1 area of the visual cortex, combined with fixational eye movements. Specifically, our performance on N-Caltech101 dataset outperforms previously reported $F_{1}$ score on Caltech101, with a similar architecture but without a STDP learning layer. However, we show that the same edge features do not manage to abstract motions observed with a static DVS from the DvsGesture dataset. Additionally, we show that liquid state machines are a promising computational model for the classification of DVS data with temporal dynamics. This paper is a step forward towards understanding and reproducing biological vision.