Brain control jigsaw puzzle system based on hybrid brain computer interface of motor imagery and steady-state visual evoked potential

Abstract

With the development of information decoding technology, the field of Brain-computer interface (BCI) has developed rapidly in recent years. Among them, Motor Imagery Brain-computer Interface (MI-BCI) and Steady state visual evoked potential Brain-computer Interface (SSVEP-BCI) have been effectively applied in some brain-controlled rehabilitation training systems to assist stroke patients in their normal life. In this paper, a brain-controlled jigsaw puzzle system based on Motor Imagery and Steady state visual evoked potential (MI-SSVEP) hybrid brain-machine is constructed. In this system, the left-right moving jigsaw puzzle uses the MI-BCI paradigm and the up-down moving jigsaw puzzle uses the SSVEP-BCI paradigm. To reduce the difficulty for patients, the system will set the moving route of the puzzle in advance. When the puzzle piece needs to move left or right, the system will remind the patient through voice and words that the patient needs to Imagine clenching his fist with his left or right hand at this time. When the puzzle piece needs to move up and down, the system will remind the patient to gaze at the upward or downward flashing arrow. If the patient makes an incorrect recognition, the system will re-open the recognition at the current position until it is correct. Compared with the ordinary rehabilitation training system, this system adds the elements of the jigsaw puzzle, so that patients can complete the training in the process of enjoying the game. The success of the jigsaw puzzle will also increase the sense of achievement for patients, and play the effect of rehabilitation training while maintaining the healthy state of mind of patients. The average recognition time of MI is 2.5s, and the accuracy is 65%. The average recognition time of SSVEP is 1.5s, and the accuracy is 95%. The system operates stably, each subject was able to complete the puzzle task quickly. The experimental results demonstrate the feasibility and potential of this hybrid brain-machine system and provide a new idea for the rehabilitation training of stroke patients.