Word recognition in natural scene and video images using Hidden Markov Model

Abstract

Text recognition from a natural scene and video is challenging compared to that in scanned document images. This is due to the problems of text on different sources of various styles, font variation, font size variations, background variations, etc. There are approaches for word segmentation from video and scene images to feed the word image into OCRs. Nevertheless, such methods often fail to yield satisfactory results in recognition. Therefore, in this paper, we propose to combine Hidden Markov Model (HMM) and Convolutional Neural Network (CNN) to achieve good recognition rate. Sequential gradient features with HMM help to find character alignment of a word. Later the character alignments are verified by Convolutional Neural network (CNN). The approach is tested on both video and scene data to show the effectiveness of the proposed approach. The results are found encouraging.