Multi-gesture drag-and-drop decoding in a 2D iBCI control task.

Abstract

Objective: Intracortical brain-computer interfaces (iBCIs) have demonstrated the ability to enable point and click as well as reach and grasp control for people with tetraplegia. However, few studies have investigated iBCIs during long-duration discrete movements that would enable common computer interactions such as "click-and-hold" or "drag-and-drop".

Approach: Here, we examined the performance of multi-class and binary (attempt/no-attempt) classification of neural activity in the left precentral gyrus of two BrainGate2 clinical trial participants performing hand gestures for 1, 2, and 4 seconds in duration. We then designed a novel "latch decoder" that utilizes parallel multi-class and binary decoding processes and evaluated its performance on data from isolated sustained gesture attempts and a multi-gesture drag-and-drop task.

Main results: Neural activity during sustained gestures revealed a marked decrease in the discriminability of hand gestures sustained beyond 1 second. Compared to standard direct decoding methods, the latch decoder demonstrated substantial improvement in decoding accuracy for gestures performed independently or in conjunction with simultaneous 2D cursor control.

Significance: This work highlights the unique neurophysiologic response patterns of sustained gesture attempts in human motor cortex and demonstrates a promising decoding approach that could enable individuals with tetraplegia to intuitively control a wider range of consumer electronics using an iBCI.