A Multi-scale Approach for Vietnamese Image Captioning in Healthcare Domain

Abstract

The image caption generator is a task that aims to automatically generate a natural language with syntactically and semantically meaningful sentences to describe the visual content of a given image. This problem is attractive because it is a combination of two fields Computer Vision and Natural Language Processing. Despite some research on this problem, most of this research only focuses on generating English captions. In this paper, we present a Transformer-based model for this problem based on the VieCap4H dataset - the first grand dataset for the Healthcare domain in Vietnamese. In detail, we first propose the TG2F module to enhance visual representations and the BERT-based language model to obtain language presentation. Through experiments on the VieCap4H dataset, our approach achieves competitive results on the public test and private test without using any data augmentation method.