Real-time captioning of sign language by groups of deaf and hard-of-hearing people

Abstract

In order to realize smooth communication between people with or without difficulty in hearing, we aim to implement an information support system based on crowdsourcing, a problem-solving model, in which numerous people cooperate to accomplish a job. More specifically, we implement a crowdsourced captioning system that enables groups of deaf and hard-of-hearing people to collaboratively interpret a speech or lecture given by sign language to text in real-time. Deaf and hard-of-hearing people are generally considered as the ones who receive support from others, but our proposal helps them become the ones who offer support to others. In this paper, we explore the issues that need be resolved for the practical use of the proposed system through several experiments and succeed in verifying its feasibility.