An Entity Event Deduplication Method Based on Connected Subgraph

Abstract

The news data from different sources has a high degree of repetition. How to accurately get the most timely news is the primary task of text processing and analysis. Therefore, we propose an entity event deduplication method based on a connected subgraph to address the high repetition rate of multisource events. In this paper, according to the correlation of events and development factors of events, we extract relevant features and improve the accuracy of the deduplication method. The final experimental evaluation shows that the proposed method has higher accuracy than the current deduplication methods, and can significantly improve the repeat detection rate of the entity event within the effective time.