Optimized Fingering Planning for Automatic Piano Playing Using Dual-arm Robot System

Robots can be used to play musical instruments such as the piano, but existing robots are not automated enough for this purpose. In this paper, we design an automatic finger and arm planning system for dual-arm robots, whose input object is a commonly used bi-phonic musical score with high-pitched and low-pitched voices. The digital score can be obtained from methods in the field of Optical Score Recognition. We combine the characteristics of the score and the robot to automatically generate fingering using a Dynamic Planning approach. Then we generate the movements of the robot's arms based on the annotated fingering. Finally, it is demonstrated by simulated experiments. Our method is more applicable to robots than general fingering generation algorithms.