An experimental comparison of recurrent neural network for natural language production

Abstract

We study the performance of three types of recurrent neural networks (RNN) for the production of natural language sentences: Simple Recurrent Networks (SRN), Back-Propagation Through Time (BPTT) and Sequential Recursive Auto-Associative Memory (SRAAM). We used simple and complex grammars to compare the ability of learning and being scaled up. Among them, SRAAM is found to have highest performance of training and producing fairly complex and long sentences.