Two systems for automatic music genre recognition: what are they really recognizing?

We re-implement two state-of-the-art systems for music genre recognition, and closely examine their behavior. First, we find specific excerpts each system consistently and persistently mislabels. Second, we test the robustness of each system to spectral adjustments to audio signals. Finally, we expose the internal genre models of each system by testing if human can recognize the genres of music excerpts composed by each system to be highly genre-representative. Our results suggest that, though they have high mean classification accuracies, neither system is recognizing music genre.