Cover Song Identification with Pairwise Cross-Similarity Matrix using Deep Learning

A cover song, by definition, is a rendition of a previously released song and mapping these cover songs to their original song is defined as ”Cover Song Identification.” In this paper, we propose multiple cover song identification methods using Convolutional Neural Network (CNN) models as well as transfer learning to extract features which can be trained on statistical models for binary classification. We develop two CNN models that are trained on a cross-similarity matrix which is generated from a pair of songs as input. Firstly we designed a simple CNN architecture that was trained on two labels 1. cover pair relationship; 2. non-cover pair relationship. Our second approach uses a CNN model known as the Inception Model. We train the model by generating cross-similarity matrices for both the labels and then converting them into images. At later stage, we use a ranking method that sorts the probabilities of the cover relation in descending order and the song with the highest probability is chosen as a match. Based on the evaluation, Inception model performs the best, scoring the accuracy of 93.4%.