Developing a Deep Learning-enabled Guide for the Visually Impaired

Abstract

With visual impairment being a major detriment to quality of life, It is worth exploring options available to them to improve that quality. In this paper we propose using deep learning, real-time object recognition, and text-to-speech capabilities to develop an application to aid the visually impaired. Learning was implemented on the Convolutional Neural Network AlexNet, using two different types of image datasets to recognize both objects and buildings. We integrate a video webcam with our trained model to recognize objects in real-time so the visually impaired will be able to perceive their environment. Finally, using text-to-speech, our application audibly speaks what our trained model recognizes so they will know what's around them. After obtaining initial results from retrained AlexNet, we attempted two modifications of the original architecture to improve its performance for our application for image recognition for the visually impaired, the first change being to the fully connected layers and the second change being to the convolutional layers. Our results show recognition of 92% for internal object data and 88% for external object data. This will go a long way to achieve UN SDG3 goals for good health and well-being for a large percentage of visually impaired people worldwide.