Application of the recurrent neural network to the problem of language acquisition

Abstract

The purpose of this paper is to explore the possibility of langnage acquisition by using the recurrent neural net work. The knowledge of language that native speakers have is supposed to be reflected in the socalled “grammatical competence. ” Thus, the problem is to examine whether the recurrent neural network can acquire the grammatical competence. To simplify the experiments, the grammatical competence means the ability to infer the well-formedness of sentences. The training sentences are generated by the limited number of training sentences and the network must make judgments about the well-formedness of new sentences, The experimental results can be summarized as follows. First, the recurrent back-propagation needs only a few of propagations and back-propagations to obtain the necessary approximate values. Second, the recurrent network can infer the well-formedness of new sentences with sentence formulae of training sentences or new sentence formulae quite well. Third, the generalization performance of the network is not necessarily related to the number of hidden units. In some cases, we can obtain the best performance with no hidden units.