Indicators of hidden neuron functionality: the weight matrix versus neuron behaviour

Abstract

Pruning of redundant or less important hidden neurons from the popular backpropagation trained neural networks is useful for a host of reasons, ranging from improvements of generalisation performance, to use as a precursor for rule extraction. For pruning it is necessary to identify hidden neurons with similar functionality. We have previously used a pruning process based on the behaviour of the hidden neurons in an image processing application to produce a quality driven compression by eliminating the least different hidden neurons. We consider the computationally cheaper alternative using only the trained weight matrix of the neural networks at each stage of the compression process. We conclude that the weight matrix is not sufficient for differentiating the functionality of the hidden neurons for this task, being essentially the functional equivalence problem which is computationally intractable.