An analysis of Dynamic Cortex Memory networks

Abstract

The recently introduced Dynamic Cortex Memory (DCM) is an extension of the Long Short Term Memory (LSTM) providing a systematic inter-gate connection infrastructure. In this paper the behavior of DCM networks is studied in more detail and their potential in the field of gradient-based sequence learning is investigated. Hereby, DCM networks are analyzed regarding particular key features of neural signal processing systems, namely, their robustness to noise and their ability of time warping. Throughout all experiments we show that DCMs converge faster and yield better results than LSTMs. Hereby, DCM networks require overall less weights than pure LSTM networks to achieve the same or even better results. Besides, a promising neurally implemented just-in-time online signal filter approach is presented, which is latency-free and still provides an accurate filtering performance much better than conventional low-pass filters. We also show that the neural networks can do explicit time warping even better than the Dynamic Time Warping (DTW) algorithm, which is a specialized method developed for this task.