Digital Reading Support for The Blind by Multimodal Interaction

Abstract

Slate-type devices allow Individuals with Blindness or Severe Visual Impairment (IBSVI) to read in place with the touch of their fingertip by audio-rendering the words they touch. Such technologies are helpful for spatial cognition while reading. However, users have to move their fingers slowly or they may lose place on screen. Also, IBSVI may wander between lines without realizing they did. In this paper, we address these two interaction problems by introducing dynamic speech-touch interaction model, and intelligent reading support system. With this model, the speed of the speech will dynamically change coping up with the user's finger speed. The proposed model is composed of: 1- Audio Dynamics Model, and 2- Off-line Speech Synthesis Technique. The intelligent reading support system predicts the direction of reading, corrects the reading word if the user drifts, and notifies the user using a sonic gutter to help her from straying off the reading line. We tested the new audio dynamics model, the sonic gutter, and the reading support model in two user studies. The participants' feedback helped us fine-tune the parameters of the two models. Finally, we ran an evaluation study where the reading support system is compared to other VoiceOver technologies. The results showed preponderance to the reading support system with its audio dynamics and intelligent reading support components.