Genetic algorithm based pattern allocation schemes for training set parallelism in backpropagation neural networks

Training set parallelization is an efficient method to optimize the training procedure performance of the backpropagation neural network algorithm. In training set parallelism, the training patterns are distributed 'optimally' among a heterogeneous array of processors, optimality criterion obtain the minimum training time per epoch. Earlier studies on heterogeneous transputers connected in a pipeline-ring topology have indicated that the above optimization problem results in a mixed integer programming problem and results in large computation time to find the optimal pattern allocations. In this paper, a genetic algorithm is used as an optimization tool to find the optimal allocation of patterns. The approach is illustrated using two benchmark problems, the 256-8-256 Encoder and NETTALK problems. Results indicate that when 'a priori' information is not used, the computation time needed by the genetic algorithm is comparable to that obtained by mixed integer programming. However, when 'a priori' information is used, the genetic algorithm results in significant reduction in computation time for finding the optimal solution.