Contextual Word Embedding based Clustering for Extractive Summarization

Abstract

Currently, the amount of content on the internet is expanding tremendously. One reason for the abundance of information is that numerous online resources cover similar themes, posing challenges and opportunities for natural language processing (NLP). People find it challenging to summarize thousands of documents on the same topic manually. Consequently, it is desirable to have multiple documents automatically summed up. This work proposed a contextual word embedding-based clustering technique for extractive summarization. At first, documents are split into sentences, and then each word in all sentences is given an embedding based on its context using the FastText embedding method. The averaged word embeddings are then used to create sentence embeddings/vectors. The Fuzzy C-Mean clustering algorithm is then applied to the collection of sentence embeddings to form clusters of semantically similar sentences. Based on the text features, the sentences inside each cluster are ranked. The final extracted summary comprises representative sentences taken from the highest-ranked sentences within each cluster. The effectiveness of the suggested methodology is tested in the context of the ROUGE evaluation metric and Document Understanding Conference (DUC) 2002 data set. Experimental results demonstrated that the presented technique outperformed the benchmark summarization techniques in terms of ROUGE measures.