Recognizing modern Japanese magazines by combining Deep Learning with language models

As one of the most culturally rich countries globally, Japan also has a rich history of magazines. In modern Japanese magazines, which were published during the 19-20th centuries, Japanese usage is similar to the current style of the contemporary Japanese language. However, most of those documents are not digitized but stored as images. Due to their importance to Japanese culture, history, and other socio-scientific topics, the problem of using computers to help identify these image-based modern magazines has been investigated from research and widely disseminated through the use of different methods in Deep Learning. However, these methods are still limited to achieve strong performance in recognizing handwriting images, especially uncommon Kanji characters. In this research, we address this problem by developing a deep learning-based language model and integrating it into the current OCR system for modern Japanese magazine documents. We also propose a combination strategy between the current Japanese OCR tool and our language model. The strategy will learn where the system should rely on OCR (e.g., Hiragana and Common kanji characters recognized correctly by OCR) or language model (uncommon Kanji characters are frequently recognized incorrectly by OCR, the system should rely on the language model). Our method enjoys visible improvement once experimented with real data.