Enhancing sentiment analysis via fusion of multiple embeddings using attention encoder with LSTM

Abstract

Different embeddings capture various linguistic aspects, such as syntactic, semantic, and contextual information. Taking into account the diverse linguistic facets, we propose a novel hybrid model. This model hinges on the amalgamation of multiple embeddings through an attention encoder, subsequently channeled into an LSTM framework for sentiment classification. Our approach entails the fusion of Paragraph2vec, ELMo, and BERT embeddings to extract contextual information, while FastText is adeptly employed to capture syntactic characteristics. Subsequently, these embeddings were fused with the embeddings obtained from the attention encoder which forms the final embeddings. LSTM model is used for predicting the final classification. We conducted experiments utilizing both the Twitter Sentiment140 and Twitter US Airline Sentiment datasets. Our fusion model’s performance was evaluated and compared against established models such as LSTM, Bi-directional LSTM, BERT and Att-Coder. The test results clearly demonstrate that our approach surpasses the baseline models in terms of performance.