Image Caption Model Based on Multi-Head Attention and Encoder-Decoder Framework

Abstract

In recently, image caption tasks are solved by using the LSTM to generate description. However, the model only accords image features and is hard to learn existing syntactic features, thereby lead to generate inaccurate description. In this paper, an image captioning model based on multi-head attention mechanism is presented. Specifically, the proposed model adopts Encoder-Decoder framework. A five-layer ResNet is used in Encoder module to extract image features. Multi-head attention layer and full connection feed forward layer are added to Decoder module. In addition, to capture the order of extracting feature sequences, the position-coded is used as a determining factor While calculating multi-head self-attention. Compared With the other current models based on various visual attention mechanisms, experimental results show that the proposed model has better performance.