Learning to Understand Audio and Multimodal Content

Abstract

Music, podcasts and audiobooks are rich audio content types with strong listener engagement. Search and recommendation across these content types can be greatly enhanced with a deep understanding of their content; across audio, text, and other multimodal content. In this talk, I discuss some of the challenges and opportunities in understanding this content. This deep understanding of content enables us to delight our users and expand the reach of our content creators. As part of enabling wider academic research into podcast content understanding, Spotify Research [1] has released a podcast dataset [2] with 120,000 hours of podcasts in English [3] and Portuguese [4].