On Predicting the Optimal Number of Hidden Nodes

Abstract

Determining the optimal number of hidden nodes is the most challenging aspect of Artificial Neural Network (ANN) design. To date, there are still no reliable methods of determining this a priori, as it depends on so many domain-specific factors. Current methods which take these into account, such as exhaustive search, growing and pruning and evolutionary algorithms are not only inexact, but also extremely time consuming -- in some cases prohibitively so. A novel approach embodied in a system called Heurix is introduced. This rapidly predicts the optimal number of hidden nodes from a small number of sample topologies. It can be configured to favour speed (low complexity), accuracy, or a balance between the two. Single hidden layer feedforward networks (SLFNs) can be built twenty times faster, and with a generalisation error of as little as 0.4% greater than those found by exhaustive search.