SynthPipe : AI based Human in the Loop Video Dubbing Pipeline

Abstract

Video content consumption via OTT platforms has been increasing over the years, with content creators worldwide creating exciting new content everyday. With this, regional content has skyrocketed in almost every language possible on platforms like YouTube, Twitch, etc. Such content may include daily soaps, educational or instructional videos, documentaries, movies, vlogs, etc which most of the time are in a language familiar to the creator. The same may not be the case for the viewer, who could be located anywhere and may not know the language of the creator. This creates a language barrier between the creator and viewer where every party involved suffers a loss - the creator loses on a global presence, the consumer loses on international content and both lose on a global community. To combat this, SynthPipe, a simple, modifiable, and efficient pipeline comprising of state-of-the-art techniques for video dubbing is proposed in this paper.