GETnet: a general framework for evolutionary temporal neural networks

Abstract

Among the more challenging problems in the design of temporal neural networks are the incorporation of short and long-term memories and the choice of network topology. Delayed copies of network signals can form short-term memory (STM), whereas feedback loops can constitute long-term memories (LTM). This paper introduces a new general evolutionary temporal neural network framework (GETnet) for the automated design of neural networks with distributed STM and LTM. GETnet is a step towards the realization of general intelligent systems that can be applied to a broad range of problems. GETnet utilizes nonlinear moving average and autoregressive nodes and sub-circuits that are trained by enhanced gradient descent and evolutionary search in architecture, synaptic delay, and synaptic weight spaces. The ability to evolve arbitrary time-delay connections enables GETnet to find novel answers to classification and system identification tasks. A new temporal minimum description length policy ensures creation of fast and compact networks with improved generalization capabilities. Simulations using Mackey-Glass time series are presented to demonstrate the above stated capabilities of GETnet.