Recognition of Beat-Motion Gestures of Orchestra Conductor using DTW and Nearest Neighbor Method

Abstract

The conductor is responsible for controlling speed, emotion, instruments, and other musical information in music performances. Using hand gestures, facial expressions, and body movements, the conductor communicates with each member of the band; the conductor primarily uses hand movements to reflect the different beats of different music pieces. In this study, Kinect was used to capture the gestural trajectory of the conductor. Additionally, the three-dimensional spatial data of the obtained motion trajectory were adaptively smoothed. The motion timing data were subsequently segmented, and a dynamic time-warping algorithm was used to match the timing data of the template library with the to-be-classified data.