Grammatical inference using higher order recurrent neural networks

Abstract

As a step towards proving theorems on the capabilities of a connectionist cognitive model, a higher order recurrent network's capabilities and limitations in learning to perform grammatical inference on regular languages is evaluated. It is seen that the recurrent neural network can learn to recognize an unknown regular grammar, and the finite state automata (FSA) corresponding to the learned grammar is extractable from the state space of the network. The proposed network can also recognize strings much longer than it was trained on. A straightforward method for incorporating partial knowledge of a deterministic FSA in the recurrent network was implemented, to see its effect on the convergence time. Simulation results show a substantial improvement in the rate of convergence of the network.