Iterative architecture for value iteration using memristors

Abstract

Memristors promise higher device density and design flexibility. Besides utilizing memristors for digital memory, another promising avenue for adoption is the advancement of neural network circuits capable of learning. Neural network implementations with memristors have been proposed, including memristor synaptic training methodologies. This work highlights applications of a neural learning methodology inspired by Q-learning. Memristors are used as analog storage elements to store a large Q-table. The method is qualified with a maze problem in order to show that the proposed network can be used to learn to approximate an optimal path to solving the maze problem. Brief results highlighting the methodology on a maze problem and discussion on generating random keys are provided. This work combines model-free reinforcement learning with neural networks.