Image Captioning Based on Automatic Constraint Loss

Abstract

In recent years, the Encoder-Decoder framework has been widely used in image captioning. In the forecast period, many methods regard the input of the usage model at the previous moment as the output at the moment, which may cause the generated words to get worse. This paper proposes to use the correct rate of the preceding words to constrain the weight of the back words, making the loss weight of the back words increase as the preceding word error rate decreases, namely Automatic Constraint Loss (ACL), reducing the difference in the training and test phase. The experimental results on the MSCOCO dataset show that the addition of the proposed method to the original model, the bleu_1 and bleu_2 scores are greatly improved, and the attention mechanism can more accurately select the image region.