Which instrument should play here? Decoding predicted musical timbre from EEG signals during omission

Abstract

The human brain predicts various musical features such as harmony, melody, and rhythm during music perception. A previous electroencephalographic (EEG) study showed that the accuracy of pitch decoding during tone omission was greater when the pitch of the melody was highly predictable than when it was less predictable, reflecting that predictive information of a specific pitch is contained in the EEG signal. However, the specificity of prediction for other musical features has not been fully addressed. The present study investigated whether predicted instruments are decoded from the EEG signal during omission to examine the specificity of prediction in the timbre dimension. Thirty-five participants listened to unfamiliar melodies with simple (high predictability) or complex (low predictability) timbre change rules while watching a silent movie. The EEG was recorded when a tone expected to be played by one of four specific timbres (celesta, electric piano, marimba, organ) was omitted. The results showed that the amplitude of an omitted stimulus potential, oN1, did not differ between high and low predictability conditions. However, the support vector machine was able to decode the type of musical timbre during omission better than random chance in the high predictability condition but not in the low predictability condition. These results suggest that EEG signals contain information about which instrument should be played during omission, but this information is not manifested in traditional event-related potentials. The brain may specifically predict not only the pitch but also other musical dimensions, such as the timbre, of the upcoming tone.