Augmented control of an avatar using an SSVEP based BCI

Abstract

The demonstration shows the usage of an EEG-based brain-computer interface (BCI) for the real-time control of an avatar in World of Warcraft. Visitors can test the installation during the conference after about 5 minutes of training time. World of Warcraft is a common Massively Multiplayer Online Role-Playing Game (MMORPG) in which the player controls an avatar in a virtual environment. The user has to wear newly developed dry EEG electrodes which are connected to a biosignal amplifier. Then the data is transmitted to a computer to perform the real-time analysis of the EEG data. The BCI system is using steady-state visual evoked potentials (SSVEPs) as control signal. Therefore the system shows different icons flickering with different frequencies. If the user focuses now on one of the icons the flickering frequency is visible in the EEG data and can be extracted with frequency analysis algorithms. In order to control an avatar in World of Warcraft it is necessary to have 4 control icons that are analyzed in real-time. Three icons are necessary to turn left or right or to move forward. Additionally a 4th icon is required to perform certain actions like grasping objects, attacking other objects....like shown in Figure 1. The visual stimulation took place via a 60Hz LCD-display with flickering frequencies of 15, 12, 10 and 8.57Hz in combination with an underlying video. To visualize the flickering controls a BCI-Overlay library based on OpenGL was implemented, which can be used by any graphics application. It provides the possibility to generate BCI controls within a virtual reality environment or as overlays in combination with video sequences Figure 2 shows the components of the complete system. The user is connected with 8 EEG electrodes to the BCI system that is running under Windows and MATLAB. The BCI system uses the minimum energy algorithm and a linear discriminant analysis to determine if the user is looking at one of the icons or if the user is not attending. Via a UDP communication channel the BCI system is controlling the BCI-Overlay module that generates the 4 flickering icons around the WoW User Interface. If the BCI system detects a certain command it is transmitted to the game controller which generates the corresponding WoW command. This is straight forward for the left, right and move forward commands, but more complicated for the action command. Action commands are context dependant and the controller has to select certain possible actions. Finally the command is transmitted to WoW and the avatar performs the action. This allows the user to play WoW with the BCI system only by thought.