IMAGE CAPTIONING USING TRANSFORMER WITH IMAGE FEATURE EXTRACTION BY XCEPTION AND INCEPTION-V3

Jurnal Ilmiah Kursor Pub Date : 2024-07-01 DOI:10.21107/kursor.v12i3.376

Jasman Pardede, Fandi Ahmad

引用次数: 0

Abstract

Image captioning is a task in image processing that involves creating text descriptions that can describe the image content. The formation of the image captioning system model is influenced by image interpretation related to the given image caption. Image interpretation is influenced by the feature extraction used. This research proposes feature extraction with Xception and Inception-V3 by generating an image captioning model using Transformer. Model performance is measured based on BLUE and METEOR values. Based on the results of research conducted on the Flickr8k Dataset, it shows that the best model performance is using Xception feature extraction and batch_size = 256. The image captioning performance of Xception feature extraction for BLUE-1, BLUE-2, BLUE-3, BLUE-4, and METEOR when compared with Inception-V3 achieves increasing of 13.15%, 18.03%, 18.71%, 27.27%, and 15.43% respectively. The performance for Xception feature extraction with batch_size = 256 compared with batch_size = 128, increasing BLUE-1, BLUE-2, BLUE-3, BLUE-4, and METEOR namely 19.81%, 41.84%, 52.23%, 53.14%, and 31.56% respectively.

查看原文本刊更多论文

通过 Xception 和 Inception-v3 使用转换器和图像特征提取技术制作图像标题

图像标题是图像处理中的一项任务，涉及创建能够描述图像内容的文字说明。图像标题系统模型的形成受到与给定图像标题相关的图像解读的影响。图像解读受所用特征提取的影响。本研究建议使用 Xception 和 Inception-V3 进行特征提取，并使用 Transformer 生成图像标题模型。模型性能根据 BLUE 和 METEOR 值进行测量。基于 Flickr8k 数据集的研究结果表明，使用 Xception 特征提取和 batch_size = 256 的模型性能最佳。与 Inception-V3 相比，Xception 特征提取对 BLUE-1、BLUE-2、BLUE-3、BLUE-4 和 METEOR 的图像标题处理性能分别提高了 13.15%、18.03%、18.71%、27.27% 和 15.43%。批量大小为 256 的 Xception 特征提取与批量大小为 128 的 Xception 特征提取相比，BLUE-1、BLUE-2、BLUE-3、BLUE-4 和 METEOR 的性能分别提高了 19.81%、41.84%、52.23%、53.14% 和 31.56%。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Jurnal Ilmiah Kursor

自引率

0.00%

发文量