Unsupervised concept identification from a large corpus of research documents

Abstract

Research documents play a crucial role in data-driven research. Identifying concepts in a corpus of research documents can lead to a better understanding of the current stage of research. It can reveal fruitful concepts hidden inside the corpus. However, manually analyzing the corpus is laborious and inefficient. Automating the process is challenging due to the lack of background knowledge to fill the semantic gap that exists between humans and machines. To address this issue, we introduce a novel method that leverages information from an online open resource, namely Wikipedia, to build background knowledge automatically. An experiment on a set of 13,636 research documents shows that the framework can effectively and efficiently identify broad range of concepts within a large text corpus by exploiting only Wikipedia categories and documents' titles.