Image Caption Bot for Assistive Vision

Abstract

It's challenging to automatically produce brief descriptions of an image's meaning because it can have diverse connotations in different languages. However, due to the vast amount of information packed into a single image, it is challenging to parse out the necessary context to use it to build sentences. It's a great way for the visually impaired to get around independently. This type of system can be built using the emerging programming technique of deep learning. This paper presents the development of an Image Caption Bot designed to aid individuals with visual impairments. We achieve enhanced accuracy in caption generation by modeling on the MSCOCO dataset using a Transformer encoder and Inception v3 for image processing. Image captioning, which entails generating textual descriptions for images, is the primary focus of our research. We achieve enhanced accuracy in caption generation by utilizing a Transformer encoder during training. The MSCOCO dataset serves as a valuable The results of the model are translated into speech for the benefit of the visually handicapped. Keywords—CNN, Google Text To Speech, MS-COCO, Inspection v3.