Legion scribe: real-time captioning by the non-experts

Real-time captioning provides people who are deaf or hard of hearing access to aural speech in the classroom and at live events. The only reliable approach currently is to recruit a local or remote expert stenographer who is able to type at natural speaking rates, who charge more than $100 USD per hour and must be scheduled in advance. We introduce Legion Scribe (Scribe) that allows 3-5 ordinary people who can hear and type to collectively caption speech in real-time together. Each individual is unable to type at natural speaking rates, and so each is only asked to type part of what they hear. Scribe computationally stitches the partial captions together to form a final caption stream. We have shown that the accuracy of Scribe captions approaches those of a professional stenographer, while its latency and cost is dramatically lower.