Realizing Video Time Decoding Machines with recurrent neural networks

Abstract

Video Time Decoding Machines faithfully reconstruct bandlimited stimuli encoded with Video Time Encoding Machines. The key step in recovery calls for the pseudo-inversion of a typically poorly conditioned large scale matrix. We investigate the realization of time decoders employing only neural components. We show that Video Time Decoding Machines can be realized with recurrent neural networks, describe their architecture and evaluate their performance. We provide the first demonstration of recovery of natural and synthetic video scenes encoded in the spike domain with decoders realized with only neural components. The performance in recovery using the latter decoder is not distinguishable from the one based on the pseudo-inversion matrix method.