The determination of neural network parameters by information theory

Abstract

The principle of optimal information distribution is a criterion for the efficient use of the different information storage resources in a given network. Furthermore, it can be used as a tool to balance the system parameters and to obtain the optimal network parameter configuration according to the minimal system storage (system description information) for a given maximal performance error. The principle was derived by maximizing the output information of the network. The use of the principle was demonstrated for the example of a simple nonlinear function approximation.<>