Analysis of Subtopic Discovery Algorithms for Real-time Information Summarization

Abstract

The rise of large data streams introduces new challenges regarding the delivery of relevant content towards an information need. This need can be seen as a broad topic of information. By identifying sub-streams within a broader data stream, we can retrieve relevant content that matches the multiple facets of the topic; thus summarizing information, and matching the initial need. In this paper, we propose to study the generation of sub-streams over time and compare various aggregation methods to summarize information. Our experiments were made using the standard TREC Real-Time Summarization (RTS) 2017 dataset.