GMU: A Novel RNN Neuron and Its Application to Handwriting Recognition

Recurrent neural networks (RNNs) have been widely used in many sequential labeling fields. Decades of research fruits show that artificial neuron as the building blocks plays great role in its success. Different RNN neurons are proposed, such as long-short term memory (LSTM) and gated recurrent unit (GRU), and used in most applications let alone character recognition, to encode the long-term contextual dependencies. Inspired by both LSTM and GRU, a new structure named gated memory unit (GMU) is presented which carries forward their merits. GMU preserves the constant error carousels (CEC) which is devoted to enhance a smooth information flow. GMU also lends both the cell structure of LSTM and the interpolation gates of GRU. The proposed neuron is evaluated on both online English handwriting recognition and online Chinese handwriting recognition tasks in terms of parameter volumes, convergence and accuracy. The results show that GMU is of potential choice in handwriting recognition tasks.