Replicating outdoor environments using VR and ambisonics: a methodology for accurate audio-visual recording, processing and reproduction

IF 4.4 3区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Virtual Reality Pub Date : 2024-05-17 DOI:10.1007/s10055-024-01003-1

Fotis Georgiou, Claudia Kawai, Beat Schäffer, Reto Pieren

{"title":"Replicating outdoor environments using VR and ambisonics: a methodology for accurate audio-visual recording, processing and reproduction","authors":"Fotis Georgiou, Claudia Kawai, Beat Schäffer, Reto Pieren","doi":"10.1007/s10055-024-01003-1","DOIUrl":null,"url":null,"abstract":"<p>This paper introduces a methodology tailored to capture, post-process, and replicate audio-visual data of outdoor environments (urban or natural) for VR experiments carried out within a controlled laboratory environment. The methodology consists of 360<span>\\(^\\circ\\)</span> video and higher order ambisonic (HOA) field recordings and subsequent calibrated spatial sound reproduction with a spherical loudspeaker array and video played back via a head-mounted display using a game engine and a graphical user interface for a perceptual experimental questionnaire. Attention was given to the equalisation and calibration of the ambisonic microphone and to the design of different ambisonic decoders. A listening experiment was conducted to evaluate four different decoders (one 2D first-order ambisonic decoder and three 3D third-order decoders) by asking participants to rate the relative (perceived) realism of recorded outdoor soundscapes reproduced with these decoders. The results showed that the third-order decoders were ranked as more realistic.</p>","PeriodicalId":23727,"journal":{"name":"Virtual Reality","volume":null,"pages":null},"PeriodicalIF":4.4000,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Virtual Reality","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s10055-024-01003-1","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}

引用次数: 0

Abstract

This paper introduces a methodology tailored to capture, post-process, and replicate audio-visual data of outdoor environments (urban or natural) for VR experiments carried out within a controlled laboratory environment. The methodology consists of 360\(^\circ\) video and higher order ambisonic (HOA) field recordings and subsequent calibrated spatial sound reproduction with a spherical loudspeaker array and video played back via a head-mounted display using a game engine and a graphical user interface for a perceptual experimental questionnaire. Attention was given to the equalisation and calibration of the ambisonic microphone and to the design of different ambisonic decoders. A listening experiment was conducted to evaluate four different decoders (one 2D first-order ambisonic decoder and three 3D third-order decoders) by asking participants to rate the relative (perceived) realism of recorded outdoor soundscapes reproduced with these decoders. The results showed that the third-order decoders were ranked as more realistic.

Abstract Image

查看原文本刊更多论文

利用虚拟现实和环境声复制室外环境：准确记录、处理和再现视听的方法论

本文介绍了一种为在受控实验室环境中进行的 VR 实验量身定制的方法，用于捕获、后处理和复制室外环境（城市或自然）的视听数据。该方法包括 360（^\circ\）视频和高阶环境声（HOA）现场录音，以及随后使用球形扬声器阵列和视频通过头戴式显示器进行校准的空间声音再现，并使用游戏引擎和图形用户界面进行感知实验问卷回放。对环境声麦克风的均衡和校准以及不同环境声解码器的设计给予了关注。通过听力实验，对四种不同的解码器（一种二维一阶环境声解码器和三种三维三阶解码器）进行了评估，要求参与者对使用这些解码器重现的室外声景的相对（感知）逼真度进行评分。结果显示，三阶解码器的逼真度更高。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Virtual Reality COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS-COMPUTER SCIENCE, SOFTWARE ENGINEERING

CiteScore

8.30

自引率

14.30%

发文量

审稿时长

>12 weeks

期刊介绍： The journal, established in 1995, publishes original research in Virtual Reality, Augmented and Mixed Reality that shapes and informs the community. The multidisciplinary nature of the field means that submissions are welcomed on a wide range of topics including, but not limited to: Original research studies of Virtual Reality, Augmented Reality, Mixed Reality and real-time visualization applications Development and evaluation of systems, tools, techniques and software that advance the field, including: Display technologies, including Head Mounted Displays, simulators and immersive displays Haptic technologies, including novel devices, interaction and rendering Interaction management, including gesture control, eye gaze, biosensors and wearables Tracking technologies VR/AR/MR in medicine, including training, surgical simulation, rehabilitation, and tissue/organ modelling. Impactful and original applications and studies of VR/AR/MR’s utility in areas such as manufacturing, business, telecommunications, arts, education, design, entertainment and defence Research demonstrating new techniques and approaches to designing, building and evaluating virtual and augmented reality systems Original research studies assessing the social, ethical, data or legal aspects of VR/AR/MR.

文献相关原料

公司名称	产品信息	采购帮参考价格