Interactive Audio Indexing and Speech Recognition based Navigation Assist Tool for Tutoring Videos

Abstract

The volume of worldwide multimedia data increases daily, especially the growth and usage is enormus during the pandemic period. Content-based audio retrieval increases the sophistication in the usage of multimedia content. While listening to the multimedia, the user navigates to the desired location, either manually or with subtitles. Both these methods are time-consuming and not user friendly. To address this issue, we have developed a speech-based searchable system that transcribes the video material to text and allows the user to navigate to the desired content. The user may listen to video lectures and guides to the topic of interest with the voice command. The system analyses the acoustic signature of the input voice command and matches it against the video. The proposed system maintains the index table, which maps the transcription of the content with the instance of the occurrence. The interactive play control retrieves the instance of the keyword and plays the multimedia content from that instant. The combination of effective transcription, storage and retrieval complements the system and improves the ease of access.