Enhancing Education and Interaction for the Visually Impaired Using Deep Learning and IoT

Abstract

Education for all has been a fundamental ideology behind the prosperity and development of a nation. The proposed system based on IoT aims to improve the quality of education and interaction for the visually challenged through object classification, learning, and digital library. This paper introduces a deep learning enabled percussive methodology for interaction of the blind. The methodology proposed in this work takes recorded audio as input and performs classification based on certain characteristics in order to determine a knock pattern. A proof-of-concept wearable device is introduced allowing visually challenged persons to passively read braille by stimulation on the skin of the forearm. A web application is designed that enables any ebook to be streamed via this web application to the wearable device. An Android application is presented that serves as an application to assist in daily life activities. The proposed system is tested on two different categories of subjects consisting of visually impaired and blindfolded people.