Vietnamese Text Detection, Recognition and Classification in Images

Detecting and recognizing text in images is a task that has received a lot of attention recently due to its high applicability in many fields such as digitization, storage, lookup, authentication However, most current research works and products are focusing on detecting and extracting text from images but not paying very much attention to analyzing and exploiting semantics and nuances of those extracted texts. In this study, we propose a three-in-one system to detect, recognize and classify Vietnamese text in images collected from social media to help authorities in monitoring tasks. The system receives as input images containing Vietnamese text, uses the Character-Region Awareness For Text detection (CRAFT) model to perform background processing to produce areas containing text in the image; these text containers will then be rearranged in the same order as in the original image, and the text in the image will also be extracted out according to the text container. Next, we use VietOCR model to convert these text images into text fragments. Finally, these texts will be classified using an ensemble of machine learning models. Preliminary results show that the proposed model has an accuracy of up to 88.0% in detecting and recognizing text and 94% in classifying text nuances on the collected data set.