EEG-based listened-language classification

Abstract

From an early age, individuals are continuously exposed to other languages beyond their native tongue; however, the brain’s response to these auditory stimuli remains unclear. To investigate this, an experiment was designed to record electroencephalography (EEG) signals from subjects listening to sentences in five different languages, and a specific database was built to enable performing classification tests to distinguish between different languages, and varying levels of language comprehension. By analysing the energy difference between the EEG channels to characterize these signals, different classification tests were conducted using bidirectional Long Short-Term Memory (bi-LSTM), Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural networks.

The main objective is the analysis of the brain’s response in two different scenarios: when the subject listens to sentences in different languages, and when the subject understands or misunderstands the meaning of a sentence. In the multi-class classification involving sentences in five different languages, the accuracy attained is 36.37 %. However, in the multi-class classification between ‘understood’/‘understood part of the meaning’/‘didn’t understand’, the accuracy attained reaches 81.36 %. The results obtained for binary classification tests of understand native language or foreign language is 89.09 %. The bi-LSTM neural network achieved the overall best performance.

These results demonstrate that the analysis of the EEG signals alone can give information regarding a person’s language comprehension level, and can be used for monitoring the learning curve of a new language or to assess comprehension in patients with conditions such as aphasia.