Brain-computer interface controlled robotic choreography based on motor imagery EEG

Abstract

Brain-computer interface (BCI) provides the ability of the human brain to control external devices directly, and human-robot interaction (HRI) can develop a robot’s autonomous, cognitive, and social abilities. Based on BCI and HRI, this study proposes a brain-controlled robotic choreography approach based on motor imagery, and it belongs to the category of cooperative human-robot dance. Moreover, the whole system includes two parts: offline training and online calibrating. In offline training, a robot-guided experimental paradigm of motor imagery (MI) was first constructed, and electroencephalogram (EEG) samples of MI were collected for training a convolutional neural network (CNN) model. During online calibration, to achieve the robotic choreography, we used a biped humanoid robot named “Yanshee” as the carrier of robotic dance, and the corresponding dance motion library and mapping rules were designed. Based on the well-trained CNN model, a majority voting strategy was used to keep robust recognition, and the recognized MI command was used to drive robotic choreography based on such library and rules. Experimental results have shown an average accuracy of 74.71% for offline classification among seven subjects. Three online controlling strategies have been applied to seven subjects, and an average of 75.40% classification accuracy has been achieved. To measure the brain-controlled robotic choreography, four invited experts gave an overall average score of 7.67 on 21 robotic dance works using a 10-point scale. The constructed framework gives a novel view to integrate science and art, further developing a new entertainment application of social robots.