Large-Scale Memory of Sequences Using Binary Sparse Neural Networks on GPU

Abstract

Associative memories are models capable to store and retrieve messages given only a part of their content. These systems have been used in several applications such as databases engines, network routers, natural language processing and image recognition due to their error correction capability in pattern retrieving. Recently, Gripon and Berrou introduced a sparse associative memory based on cliques which achieves almost optimal storage efficiency (ratio of useful bits stored to bits used). Binary Tournament-based Neural Network is an extension of sparse associative memories based on cliques and it is used to store and retrieve long sequences of patterns. This paper proposes solutions to increase the retrieval performance in a parallel architecture for this memory and a scalable and optimized implementation of the algorithms needed to use.