On the influence of synaptic weight states in a locally competitive algorithm for memristive hardware

Abstract

Memristors promise a means for very compact neu-romorphic nanoscale architectures that leverage in-situ learning algorithms. While traditional learning algorithms simulated in software commonly assume analog values for synaptic weights, actual physical memristors may have a finite set of achievable states during online learning. In this paper we simulate a learning algorithm with limitations on both the resolution of its weights and the means of switching between them to gain an appreciation for how these properties might affect classification performance. For our experiments we use the Locally Competitive Algorithm (LCA) by Rozell et al. in conjunction with the MNIST dataset. We investigate the effects of both linear and non-linear distributions of weight states, concluding that as long as the weights are roughly within a power law distribution close to linear the algorithm is still effective. Our results also show that the resolution required from a device depends on its transition function between states; for transitions akin to round to nearest, synaptic weights should have around 16 possible states (4-bit resolution) to obtain optimal results. We find that lowering the threshold required to change states or adding stochasticity to the system can reduce that requirement down to 4 states (2-bit resolution). The outcomes of our research are relevant for building neuromorphic hardware with state-of-the art memristive devices.