Automatic summarization of Instagram social network posts by combining semantic and statistical approaches

Abstract

The increasing spread of data and text documents such as articles, web pages, books, posts on social networks, etc. on the Internet, creates a fundamental challenge in various fields of text processing under the title of “automatic text summarization”. Manual processing and summarization of large volumes of textual data is a very difficult, expensive, time-consuming, and impossible process for human users. Text summarization systems are divided into extractive and abstract categories. In the extractive summarization method, the final summary of a text document is extracted from the important sentences of the same document without any kind of change. In this method, it is possible to repeat a series of sentences repeatedly and interfere with pronouns. But in the abstract summarization method, the final summary of a textual document is extracted from the meaning of the sentences and words of the same document or other documents. Many of the performed works have used extraction methods or abstracts to summarize the collection of web documents, each of which has advantages and disadvantages in the results obtained in terms of similarity or size. In this research, by developing a crawler, extracting the popular text posts from the Instagram social network, suitable pre-processing, and combining the set of extractive and abstract algorithms, the researcher showed how to use each of the abstract algorithms. and used extraction as a supplement to increase the accuracy and accuracy of another algorithm. Observations made on 820 popular text posts on the Instagram social network show the accuracy (80%) of the proposed system.