Tele-assistance system for the blinds using Video-call with remote scene construction

Abstract

Video-call applications may not be originally designed to accommodate the blinds since they are not the main target users. However, this kind of application can be utilized as a tele-assistance solution for the assistants to help the blind navigate or perform any task that require sight. We field tested a popular Video-call application, Line, by blind users while communicating with their assistants for help in various tasks and found several drawbacks that must be overcome in order to make the application usable by the two parties. In this work, we focus on the fact that blind users lack the capability to video necessary frames that their assistants must see in order to help the blinds with sight-demanded tasks. They, therefore, may have to take the video repeatedly by waving the camera rather randomly until necessary frames appear. We propose an algorithm to construct scenes from remotely-recorded video frames that produce wide-angle or panoramic type of image, particularly for the assistants. This feature is integrated with our video-call system using WebRTC technology. Six assistant volunteers tested the proposed system by comparing it with existing video-call application and found 4.17 out of 5 satisfaction rate.