The effect of limited-precision weights on the perfect generalization requirements for threshold Adalines

Abstract

In the design of a dedicated neural network, the number of precision levels used in the hardware circuitry to store weight values is an important consideration as it will impact the functionality and hence the performance of the neural network. One measure of the functionality is the number of training set examples required to achieve perfect generalization. In this paper, we experimentally determine the training set size required for the threshold Adaline (adaptive linear element) with various levels of weight precision to achieve perfect generalization. In all cases, it was found that the training set size required for the perfect generalization was proportional to the number of weights; for the binary, ternary, and 5-ary Adalines, the constants of the proportionality were found to be 1.36, 2.5, and 4.85 respectively.