Music visualization based on the MIDI specifications for multidimensional polyphonic expression

Abstract

Music visualization is a branch of information visualization. First, the advantages of MIDI specifications for multidimensional polyphonic expression in “channel per note” were analyzed. Second, the new music visualization mapping model was proposed, which was based on the structural similarity of human and computer recognition in sound and image. The model corresponded to pitch bend, control changes, velocity to hue, saturation, and value respectively. Third, the scheme of music visualization was proposed. The base image was obtained by improving the brightness of original image by 20%–30%; then the influence image was divided into three parts as the left, the middle and the right solid color area, which corresponded to the bass voice, the alto voice and the treble voice respectively; next, music information from different areas were mapped to calculate the HSV values of solid color and then the HSV values were converted to the RGB values; finally, the RGB values of these three parts were glued together and combined with the base image in multiply mode to obtain the final image. Fourth, taking advantage of Cycling'74 Max/Jitter 7.3.4 and optimizing image combination calculation, five experiments were completed under the circumstances of bass voice, alto voice, treble voice, two notes shown in the same area and two notes shown in two different areas. Finally, by calculation of Jit.fpsgui, the final frame rate of the image was around 80fps if RGB values were glued every 16 milliseconds. The research shows that this method can meet real-time needs of music visualization and it can be used for live shows.