A Hierarchical Long Short-Term Memory Encoder-Decoder Model for Abstractive Summarization

Abstractive summarization is the task of generating concise summary of a source text, which is a challenging problem in Natural Language Processing (NLP). Many recent studies have relied on encoder-decoder sequence-to-sequence deep neural networks to solve this problem. However, most of these models treat the input as a sequence of words at the same level during the encoding process. This will make the encoding inefficient, especially for long input texts. Addressing this issue, in this paper we propose a model to encode text in a hierarchical manner, which helps to encode information in a way that is consistent with the nature of the text: the text is synthesized from sentences, and each sentence is synthesized from words. Our proposed model is based on Long Short Term Memory model that we called HLSTM (Hierarchical Long Short Term Memory) and applied to the problem of abstractive summarization. We conducted extensive experiments on the two most popular corpora (Gigaword and Amazon Review) and obtain significant improvements in comparison with the baseline models.