A User-Centered Evaluation of the Data-Driven Sign Language Avatar System: A Pilot Study

Abstract

Sign Languages (SL) are a form of communication in the visual-gestural modality, and are full-fledged natural languages. Recent years have witnessed the increase in the use of virtual avatars as virtual assistants. Research into sign language recognition has demonstrated promising potential for robust automatic sign language recognition. However, the area of sign language synthesis is still in its infancy. This explains the underdevelopment of virtual intelligent signing systems. Additionally, existing models are often restricted to manually written rules and require expert knowledge, while data-driven approach could provide a better solution. Apart from the development of signing systems, research indicates a gap in the evaluation thereof by sign language users. In this paper, we propose a data-driven sign language interpreting avatar and its subjective evaluation. We present findings from a pilot study with the deaf evaluating two different avatars against a human sign language interpreter using the metrics that are believed to bring out important insights and narratives for the users in terms of their perceptions of the avatars.