Indoor Navigation Assistance System for Visually Impaired with Semantic Segmentation using EdgeTPU

Abstract

Recognizing the limited supply and high cost of alternative solutions for visual assistance, such as guides, it is critical to create an affordable and accessible means of recognizing walkable paths for visually impaired individuals. In this paper, we assess deep learning and traditional machine learning models for image segmentation and object detection to identify walkable, indoor paths in real-time, given low-resolution images from a camera. Specifically, we leverage the processing capabilities of Google’s EdgeTPU chip, which accelerates inferences of light-weight TensorFlow models deployed in embedded devices. We retrain the MobileNet computer vision model on the ADE MIT Scene Parsing Benchmark Dataset, and improve the performance accuracy by consolidating 150 categories into just two categories. The segmentation model is post-quantize-aware trained and co-compiled with a light-weight object detection model. The resulting model is capable of simultaneous semantic segmentation and object detection with inference times of 65 milliseconds. Our approach lays the foundation for transforming the level of assistance for the visually impaired to sense the world through a wearable device assistance system for indoor navigation.