Dominance of Audio Features in Audience Emotion: A Multimodal EEG Analysis of Music Videos

IF 1.5 4区心理学 0 HUMANITIES, MULTIDISCIPLINARY

Empirical Studies of the Arts Pub Date : 2026-04-15 DOI:10.1177/02762374261426246

Jiaxuan Wang, Yuqing Liu, Yao Song

{"title":"Dominance of Audio Features in Audience Emotion: A Multimodal EEG Analysis of Music Videos","authors":"Jiaxuan Wang, Yuqing Liu, Yao Song","doi":"10.1177/02762374261426246","DOIUrl":null,"url":null,"abstract":"This study investigates how auditory, visual, and lyrical features in music videos shape emotional responses, using EEG data from 26 participants. Five widely viewed music videos were selected based on their global popularity and cross-cultural appeal. A CNN–LSTM model classified emotional states with 97.67% accuracy, and complementary regression results showed strong generalization (best model RMSE = 0.15, MAE = 0.10). Feature selection reduced 27 candidates to a sparse set dominated by auditory and visual cues, and SHAP interpretation revealed a clear modality hierarchy: pitch- and dynamics-related auditory features accounted for the largest share of predictive importance, visual color properties (hue, saturation) provided secondary influence, and lyrical sentiment contributed least. These findings support neuroaesthetic accounts in which low-level sensory structure drives rapid affective appraisal, with visual tone refining emotional meaning. Practically, the results suggest that deliberate control of pitch/dynamics can reliably steer emotional engagement in music-video creation for diverse audiences.","PeriodicalId":45870,"journal":{"name":"Empirical Studies of the Arts","volume":"12 1","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2026-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Empirical Studies of the Arts","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1177/02762374261426246","RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"HUMANITIES, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

This study investigates how auditory, visual, and lyrical features in music videos shape emotional responses, using EEG data from 26 participants. Five widely viewed music videos were selected based on their global popularity and cross-cultural appeal. A CNN–LSTM model classified emotional states with 97.67% accuracy, and complementary regression results showed strong generalization (best model RMSE = 0.15, MAE = 0.10). Feature selection reduced 27 candidates to a sparse set dominated by auditory and visual cues, and SHAP interpretation revealed a clear modality hierarchy: pitch- and dynamics-related auditory features accounted for the largest share of predictive importance, visual color properties (hue, saturation) provided secondary influence, and lyrical sentiment contributed least. These findings support neuroaesthetic accounts in which low-level sensory structure drives rapid affective appraisal, with visual tone refining emotional meaning. Practically, the results suggest that deliberate control of pitch/dynamics can reliably steer emotional engagement in music-video creation for diverse audiences.

查看原文本刊更多论文

音频特征在观众情感中的主导地位：音乐视频的多模态脑电图分析

这项研究利用26名参与者的脑电图数据，调查了音乐视频中的听觉、视觉和抒情特征是如何塑造情绪反应的。根据全球受欢迎程度和跨文化吸引力，选出了5部广受欢迎的音乐视频。CNN-LSTM模型对情绪状态的分类准确率为97.67%，互补回归结果具有较强的泛化性（最佳模型RMSE = 0.15, MAE = 0.10）。特征选择将27个候选者减少到一个由听觉和视觉线索主导的稀疏集合，SHAP解释揭示了一个清晰的模态层次：与音调和动态相关的听觉特征占预测重要性的最大份额，视觉颜色属性（色调，饱和度）提供次要影响，抒情情绪贡献最小。这些发现支持神经美学的说法，即低水平的感觉结构驱动快速的情感评估，视觉音调精炼情感含义。实际上，研究结果表明，刻意控制音高/动态可以可靠地引导不同观众在音乐视频创作中的情感投入。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Empirical Studies of the Arts Multiple-

CiteScore

2.60

自引率

0.00%

发文量

期刊介绍： Empirical Studies of the Arts (ART) aims to be an interdisciplinary forum for theoretical and empirical studies of aesthetics, creativity, and all of the arts. It spans anthropological, psychological, neuroscientific, semiotic, and sociological studies of the creation, perception, and appreciation of literary, musical, visual and other art forms. Whether you are an active researcher or an interested bystander, Empirical Studies of the Arts keeps you up to date on the latest trends in scientific studies of the arts.