Interconnection Tensor Rank and the Neural Network Storage Capacity

Abstract

Neural network properties are considered in the case of the interconnection tensor rank being higher than two (i.e., when in addition to the synaptic connection matrix, there are presynaptic synapses, pre-presynaptic synapses, etc.). This sort of interconnection tensor occurs in realization of crossbar-based neural networks. It is intrinsic for a crossbar design to suffer from parasitic currents: when a signal travels along a connection to a certain neuron, a part of it always passes to other neurons’ connections through memory cells (synapses). As a result, a signal at the neuron input holds noise—other weak signals going to all other neurons. It means that the conductivity of an analog crossbar cell varies proportionally to the noise signal, and the cell output signal becomes nonlinear. It is shown that the interconnection tensor of a certain form makes the neural network much more efficient: the storage capacity and basin of attraction of the network increase considerably. A network like the Hopfield one is used in the study.