ENZO-II-a powerful design tool to evolve multilayer feed forward networks

Abstract

ENZO-II combines two successful search techniques: gradient descent for an efficient local weight optimization and evolution for a global topology optimization. By using these, it takes full advantage of the efficiently computable gradient information without being trapped by local minima. Through knowledge transfer by inheriting parental weights, learning is speeded up by 1-2 orders of magnitude, and the expected fitness of the offspring is far above the average for this network topology. Moreover, ENZO-II impressively thins out the topology by the cooperation between a discrete mutation operator and a continuous weight decay method. Especially, ENZO-II also tries to cut off the connections to possibly redundant input units. Therefore, ENZO-II not only supports the user in the network design but also recognizes redundant input units.<>