Term Extraction from Japanese Ancient Writings Using Probability of Character N-grams

Abstract

Currently, there are no tools available to separate ancient Japanese sentence into words. Therefore, it is difficult to extract archaic Japanese terms from Japanese ancient writings. In this paper, we propose a method of term extraction for ancient Japanese documents. We calculate the likelihood of character n-grams to be a word, and extract character n-grams with higher likelihood as archaic Japanese terms. We conducted experiments of term separation using the term likelihood by the proposed method.