Bridging the Auditory Gap: AR Smart Glasses for Real-Time Speech-To-Text and Directional Audio Visualization for the Hearing-Impaired

Abstract

Sign language and hearing aids are among the most widely used communication support methods for individuals with hearing impairments. However, these methods present certain limitations: sign language can only be used between people who understand it, and hearing aids can cause discomfort when worn. To address these issues, we propose and implement a system that displays the direction of arrival of speech, speech recognition results, and speaker recognition results on augmented reality (AR) glasses. We also evaluate its effectiveness through user studies and evaluation experiments.