A Python Library for Memory Augmented Neural Networks

Abstract

A Memory Augmented Neural Network (MANN) is an extension to an RNN which enables it to save large amount of data to a memory object which is dimensionally separated from the Neural Network. This paper introduces a new Python library based on TensorFlow to define MANNs as Python objects. In addition to the standard implementation of the MANN, this contribution proposes a modification to the head calculation which decreases the noise while searching through the memory. The paper presents two experiments concerning the proposed implementation.FirsttheMANNistrainedtobeabletostoreand reproduce a piece of data (a task with linear data connectivity), and second the MANN is trained to find a Minimum Vertex Cover of a Graph (MVCG). This task was chosen because the connectivity of the vertex in the graph, that would pose a challenge to the MANN. The tests show that he MANN has no problem learning the first task, and that it is able to find an optimal solution for the MVCG problem in most cases.