A YOLO-based Object Simplification Approach for Visual Prostheses

Abstract

Visual prostheses have been introduced to partially restore vision to the blind via visual pathway stimulation. Despite their success, some challenges have been reported by the implanted patients. One of those challenges is the difficulty of object recognition due to the low resolution of the images perceived through these devices. In this paper, a deep learning-based approach combined with image pre-processing is proposed to allow visual prostheses' users to recognize objects in a given scene. The approach simplifies the objects in the scene by displaying the objects in clip art form to enhance object recognition. These clip art images are generated by, first, identifying the objects in the scene using the You Only Look Once (YOLO) deep neural network. The clip art corresponding to each identified object is then retrieved via Google Images. Three experiments were conducted to measure the success of the proposed approach using simulated prosthetic vision. Our results reveal a remarkable decrease in the recognition time, increase in the recognition accuracy and confidence level when using the clip art representation as opposed to using the actual images of the objects. These results demonstrate the utility of object simplification in enhancing the perception of images in prosthetic vision.