Subtitle Generation and Video Scene Indexing using Recurrent Neural Networks

Abstract

Video Subtitles are not only an essential tool for the hearing impaired, but also enhance the user's viewing experience, as they allow users to better understand and interpret different accents, even if they are of the same familiar language. Automatic Speech Recognition Systems would also eradicate the strenuous mechanical process involved in creating subtitle files for movie videos. Searching and indexing of different scenes in a video is still far behind when compared to that available for other forms like text data. With the help of Video Captioning models, the accessibility and indexing requirements of video files can be significantly improved by allowing the users to search for a particular scene/event in a video. This paper discusses about the solution offered to these requirements with the help of sequence-to-sequence recurrent neural networks. The paper also includes the different techniques involved in preprocessing the audio data and extracting features from them, the network architectures, CTC algorithm for backpropagation of error through time, suitable evaluation metrics for Sequence-to-Sequence models and the challenges involved during the designing and training phase of such models.