A PDA-based sign translator

Abstract

We propose an effective approach for a PDA-based sign system and present the sign translator. Its main functions include three parts: detection, recognition and translation. Automatic detection and recognition of text in natural scenes is a prerequisite for the automatic sign translator. In order to make the system robust for text detection in various natural scenes, the detection approach efficiently embeds multi-resolution, adaptive search in a hierarchical framework with different emphases at each layer. We also introduce an intensity-based OCR method to recognize characters in various fonts and lighting conditions, where we employ the Gabor transform to obtain local features, and LDA for selection and classification of features. The recognition rate is 92.4% for the testing set obtained from the natural sign. A sign is different from the normal used sentence. It is brief with a lot of abbreviations and place nouns. We only briefly introduce a rule-based place name translation. We have integrated all these functions in a PDA, which can capture sign images, auto segment and recognize the Chinese sign, and translate it into English.