Single-output recurrent neural networks for sentence binary classification

Abstract

We report several experiments on using Recurrent Neural Networks (RNNs) for sentence binary classification task. In terms of sentence classification, RNNs have an important advantage compared to well-known traditional machine learning models (e.g. SVM and Maximum Entropy), in which it can naturally take into account neighboring information between contiguous words. In addition, to perform binary classification task, we employed Single-Output RNNs (SORNNs) which only consists of a single output layer located in the last time step. The output layer itself is a vector consisting of two units (since we perform binary classification), in which each unit corresponds to a single label. Our results showed that SORNN achieved better performance than other traditional machine learning models, such as SVM, Maximum Entropy, and Naive Bayes, which have been widely used for sentence classification.