Context-based Image Caption using Deep Learning

Abstract

Image captioning is an important but difficult task. The existing image caption mainly adopts the encoding and decoding structure, the encoder mainly uses CNN as image feature extractors, and the decoder uses LSTM. The attention mechanism is also widely used in the current encoding and decoding structure. However, the existing image caption models based on the convolutional neural networks and recurrent neural networks have low accuracy in extracting useful information from images and have problems such as gradient explosion. To solve these problems, this paper proposes a context-based image caption generation model. The method applies Resnet and context-coding for feature extraction SCST, then SCST and LSTM is used for captioning The experimental results demonstrates the effectiveness of the proposed approach.