Grouping news events using semantic representations of hierarchical elements of articles and named entities

Abstract

Enormous amount of news articles are being generated through different news agencies. The variation in journalistic content and online availability of news content, makes it difficult to monitor and interpret in real time. Organizing news articles would play a crucial role in its consumption and interpretation. Our work assists end user by grouping news articles based on the story. We present here a novel approach of grouping news articles based on a multi-level embedding representation of articles, coupled with a standard TF-IDF score based on named entities. Our results shows that combining the syntactic(TF-IDF) as well as the semantic (Bert) representations can boost the performance of the news grouping task. We also experiment with transfer learning and fine tuning of state-of-the-art BERT models for the task of document similarity and use the output embeddings as document representations.