English to Tamil Multi-Modal Image Captioning Translation

Abstract

Bridging the gap between the latest technologies and the local languages, Machine Translation has had a big impact on countless non-English speakers who do not get access to these latest technologies coming out which are exclusive only to English Speakers. One such new research field which requires the intervention of Translation is the domain of Image Captioning. With the potential to impact 75 million worldwide users of the language, we have created a one-of-a-kind unique Tamil Image Captioning Dataset, translated from Microsoft’s Common Objects in Context Dataset or commonly called the COCO Dataset, for Captioning of Images in the language of Tamil. With the help of the dataset created, this research work will explore several multi-modal architectures to provide captions of images directly in the language of Tamil for any given input Image. The results of the captions generated have been discussed and evaluated using the popular evaluation metric, Bilingual Evaluation Understudy Score (BLEU).