Multi-Attention Network for Sentiment Analysis

Abstract

Sentiment analysis is an active research area in natural language processing. However, most existing methods use extra data such as pre-specified syntactic structure or user preference information. In this work, we propose a multiple attention network (MAN) that learns both word- and phrase-level features for sentiment analysis. MAN uses vector representation of the input sequence as target in the first attention layer to locate the words that contribute to the sentence sentiment. However, although an isolated word may indicate subjectivity, there may be insufficient context to determine sentiment orientation. We argue that the sentence sentiment often requires multiple steps of reasoning. Thus, we apply the second attention layer to explore the phrase information around the keyword. We experiment our method on three benchmark datasets and the results show that our model achieves state-of-the-art performance without any extra data. The visualization of the attention layers illustrates the effectiveness of our attention based model.