Information capacity and fault tolerance of binary weights Hopfield nets

Abstract

We define a measure for the fault-tolerance of binary weights Hopfield networks and relate it to a measure of information capacity. Using these measures, we compute results on the fault-tolerance and information capacity of certain Hopfield networks employing binary-valued weights. These Hopfield networks are governed by a single scalar parameter that controls their weights and biases. In one extreme value of this parameter, we show that the information capacity is optimal whereas the fault-tolerance is zero. At the other extreme, our results are inexact. We are only able to show that the information capacity is at least of the order of N log/sub 2/ N and N respectively, where N is the number of units. Our fault-tolerance results are even poorer, though nonzero. Nevertheless they do indicate a trade-off between information capacity and fault-tolerance as this parameter is varied from the first extreme to the second. We are also able to show that particular collections of patterns remain stable states as this parameter is varied, and fault-tolerance for them goes from zero at one extreme of this parameter to /spl Theta/(N/sup 2/) at the other extreme.<>