Audio augmented reality using sonification to enhance visual art experiences: Lessons learned

IF 5.3 2区计算机科学 Q1 COMPUTER SCIENCE, CYBERNETICS

International Journal of Human-Computer Studies Pub Date : 2024-07-18 DOI:10.1016/j.ijhcs.2024.103329

Abhraneil Dam , YeaJi Lee , Arsh Siddiqui , Wallace Santos Lages , Myounghoon Jeon

{"title":"Audio augmented reality using sonification to enhance visual art experiences: Lessons learned","authors":"Abhraneil Dam , YeaJi Lee , Arsh Siddiqui , Wallace Santos Lages , Myounghoon Jeon","doi":"10.1016/j.ijhcs.2024.103329","DOIUrl":null,"url":null,"abstract":"<div><p>Augmenting visual art in art galleries can be an effective Audio Augmented Reality (AAR) application for indoor exploration. In the current study, eight paintings from four genres were augmented with audio through their sonification. Basic Audio was generated using a sonification algorithm by identifying the major colors of the paintings, and Enhanced Audio was generated by a musician enhancing the Basic Audio; these were presented with the paintings to compare against No Audio. Twenty-six participants viewed each painting in all three conditions; eye gaze metrics, and qualitative data were collected. Results showed that Enhanced Audio led to significantly greater engagement and positive sentiments, compared to Basic Audio. Thematic analysis showed semantic and syntactic relationships of the audio with the paintings, and a tendency to guide users’ gaze over time. Findings from this study can guide future AAR developments to improve auditory display designs to enhance visual experiences.</p></div>","PeriodicalId":54955,"journal":{"name":"International Journal of Human-Computer Studies","volume":"191 ","pages":"Article 103329"},"PeriodicalIF":5.3000,"publicationDate":"2024-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Human-Computer Studies","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1071581924001125","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, CYBERNETICS","Score":null,"Total":0}

引用次数: 0

Abstract

Augmenting visual art in art galleries can be an effective Audio Augmented Reality (AAR) application for indoor exploration. In the current study, eight paintings from four genres were augmented with audio through their sonification. Basic Audio was generated using a sonification algorithm by identifying the major colors of the paintings, and Enhanced Audio was generated by a musician enhancing the Basic Audio; these were presented with the paintings to compare against No Audio. Twenty-six participants viewed each painting in all three conditions; eye gaze metrics, and qualitative data were collected. Results showed that Enhanced Audio led to significantly greater engagement and positive sentiments, compared to Basic Audio. Thematic analysis showed semantic and syntactic relationships of the audio with the paintings, and a tendency to guide users’ gaze over time. Findings from this study can guide future AAR developments to improve auditory display designs to enhance visual experiences.

查看原文本刊更多论文

利用声化技术增强视觉艺术体验的音频增强现实技术：经验教训

增强美术馆中的视觉艺术可以成为室内探索的有效音频增强现实（AAR）应用。在当前的研究中，来自四种流派的八幅画作通过声化技术被增强了音频。基本音频是通过识别画作的主要颜色，使用声化算法生成的；增强音频是由音乐家增强基本音频生成的；这些音频与画作一起展示，与无音频进行比较。26 名参与者在所有三种条件下观看了每幅画；收集了眼球注视指标和定性数据。结果显示，与 "基本音频 "相比，"增强音频 "的参与度和积极情绪明显更高。主题分析表明，音频与画作之间存在语义和句法上的关系，并且随着时间的推移有引导用户注视的趋势。这项研究的结果可以指导未来的 AAR 开发，以改进听觉显示设计，增强视觉体验。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Journal of Human-Computer Studies 工程技术-计算机：控制论

CiteScore

11.50

自引率

5.60%

发文量

108

审稿时长

3 months

期刊介绍： The International Journal of Human-Computer Studies publishes original research over the whole spectrum of work relevant to the theory and practice of innovative interactive systems. The journal is inherently interdisciplinary, covering research in computing, artificial intelligence, psychology, linguistics, communication, design, engineering, and social organization, which is relevant to the design, analysis, evaluation and application of innovative interactive systems. Papers at the boundaries of these disciplines are especially welcome, as it is our view that interdisciplinary approaches are needed for producing theoretical insights in this complex area and for effective deployment of innovative technologies in concrete user communities. Research areas relevant to the journal include, but are not limited to: • Innovative interaction techniques • Multimodal interaction • Speech interaction • Graphic interaction • Natural language interaction • Interaction in mobile and embedded systems • Interface design and evaluation methodologies • Design and evaluation of innovative interactive systems • User interface prototyping and management systems • Ubiquitous computing • Wearable computers • Pervasive computing • Affective computing • Empirical studies of user behaviour • Empirical studies of programming and software engineering • Computer supported cooperative work • Computer mediated communication • Virtual reality • Mixed and augmented Reality • Intelligent user interfaces • Presence ...