Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia最新文献

筛选
英文 中文
An Audio-Visual Method for Room Boundary Estimation and Material Recognition 房间边界估计与材料识别的视听方法
Luca Remaggi, Hansung Kim, P. Jackson, A. Hilton
{"title":"An Audio-Visual Method for Room Boundary Estimation and Material Recognition","authors":"Luca Remaggi, Hansung Kim, P. Jackson, A. Hilton","doi":"10.1145/3264869.3264876","DOIUrl":"https://doi.org/10.1145/3264869.3264876","url":null,"abstract":"In applications such as virtual and augmented reality, a plausible and coherent audio-visual reproduction can be achieved by deeply understanding the reference scene acoustics. This requires knowledge of the scene geometry and related materials. In this paper, we present an audio-visual approach for acoustic scene understanding. We propose a novel material recognition algorithm, that exploits information carried by acoustic signals. The acoustic absorption coefficients are selected as features. The training dataset was constructed by combining information available in the literature, and additional labeled data that we recorded in a small room having short reverberation time (RT60). Classic machine learning methods are used to validate the model, by employing data recorded in five rooms, having different sizes and RT60s. The estimated materials are utilized to label room boundaries, reconstructed by a vision-based method. Results show 89% and 80% agreement between the estimated and reference room volumes and materials, respectively.","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126942489","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia 2018年沉浸式多媒体视听场景理解研讨会论文集
{"title":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","authors":"","doi":"10.1145/3264869","DOIUrl":"https://doi.org/10.1145/3264869","url":null,"abstract":"","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"59 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127046342","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Multimodal Fusion Strategies: Human vs. Machine 多模态融合策略:人与机器
Hanseok Ko
{"title":"Multimodal Fusion Strategies: Human vs. Machine","authors":"Hanseok Ko","doi":"10.1145/3264869.3264870","DOIUrl":"https://doi.org/10.1145/3264869.3264870","url":null,"abstract":"Two-hour movie or a short movie clip as its subset is intended to capture and present a meaningful (or significant) story in video to be recognized and understood by human audience. What if we substitute the task of human audience with that of an intelligent machine or robot capable of capturing and processing the semantic information in terms of audio and video cues contained in the video? By using both auditory and visual means, human brain processes the audio (sound, speech) and video (background image scene, moving video objects, written characters) modalities to extract the spatial and temporal semantic information, that are contextually complementary and robust. Smart machines equipped with audiovisual multisensors (e.g. CCTV equipped with cameras and microphones) should be capable of achieving the same task. An appropriate fusion strategy combining the audio and visual information would be a key component in developing such artificial general intelligent (AGI) systems. This talk reviews the challenges of current video analytics schemes and explores various sensor fusion techniques [1, 2, 3, 4] to combine the audio-visual information cues for video content analytics task.","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122142376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Towards Realistic Immersive Audiovisual Simulations for Hearing Research: Capture, Virtual Scenes and Reproduction 面向听觉研究的逼真沉浸式视听模拟:捕捉、虚拟场景和再现
G. Llorach, G. Grimm, Maartje M. E. Hendrikse, V. Hohmann
{"title":"Towards Realistic Immersive Audiovisual Simulations for Hearing Research: Capture, Virtual Scenes and Reproduction","authors":"G. Llorach, G. Grimm, Maartje M. E. Hendrikse, V. Hohmann","doi":"10.1145/3264869.3264874","DOIUrl":"https://doi.org/10.1145/3264869.3264874","url":null,"abstract":"Most current hearing research laboratories and hearing aid evaluation setups are not sufficient to simulate real-life situations and to evaluate future generations of hearing aids that might include gaze information and brain signals. Thus, new methodologies and technologies might need to be implemented in hearing laboratories and clinics in order to generate audiovisual realistic testing environments. The aim of this work is to provide a comprehensive review of the current available approaches and future directions to create audiovisual realistic immersive simulations for hearing research. Additionally, we present the technologies and use cases of our laboratory, as well as the pros and cons of such technologies: From creating 3D virtual simulations with computer graphics and virtual acoustic simulations, to 360º videos and Ambisonic recordings.","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132315446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 14
Audio-Visual Attention Networks for Emotion Recognition 情感识别的视听注意网络
Jiyoung Lee, Sunok Kim, Seungryong Kim, K. Sohn
{"title":"Audio-Visual Attention Networks for Emotion Recognition","authors":"Jiyoung Lee, Sunok Kim, Seungryong Kim, K. Sohn","doi":"10.1145/3264869.3264873","DOIUrl":"https://doi.org/10.1145/3264869.3264873","url":null,"abstract":"We present a spatiotemporal attention based multimodal deep neu- ral networks for dimensional emotion recognition in multimodal audio-visual video sequence. To learn the temporal attention that discriminatively focuses on emotional sailient parts within speech audios, we formulate the temporal attention network using deep neural networks (DNNs). In addition, to learn the spatiotemporal attention that selectively focuses on emotional sailient parts within facial videos, the spatiotemporal encoder-decoder network is formu- lated using Convolutional LSTM (ConvLSTM) modules, and learned implicitly without any pixel-level annotations. By leveraging the spatiotemporal attention, the 3D convolutional neural networks (3D-CNNs) is also formulated to robustly recognize the dimensional emotion in facial videos. Furthermore, to exploit multimodal infor- mation, we fuse the audio and video features to emotion regression model. The experimental results show that our method can achieve the state-of-the-art results in dimensional emotion recognition with the highest concordance correlation coefficient (CCC) on AV+EC 2017 dataset.","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126942688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Session details: Oral Session 1 会议详情:口头会议1
Bumsub Ham
{"title":"Session details: Oral Session 1","authors":"Bumsub Ham","doi":"10.1145/3286581","DOIUrl":"https://doi.org/10.1145/3286581","url":null,"abstract":"","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129189690","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Session details: Keynote Talk 2 会议详情:主题演讲2
Hansung Kim
{"title":"Session details: Keynote Talk 2","authors":"Hansung Kim","doi":"10.1145/3286582","DOIUrl":"https://doi.org/10.1145/3286582","url":null,"abstract":"","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121233133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Spatial Audio on the Web - Create, Compress, and Render 网络上的空间音频-创建,压缩和渲染
J. Skoglund
{"title":"Spatial Audio on the Web - Create, Compress, and Render","authors":"J. Skoglund","doi":"10.1145/3264869.3264871","DOIUrl":"https://doi.org/10.1145/3264869.3264871","url":null,"abstract":"The recent surge of VR and AR has spawned an interest in spatial audio beyond its traditional delivery over loudspeakers in, e.g., home theater environments, to headphone delivery over, e.g., mobile devices. In this talk we'll discuss a web-based approach to spatial audio. It will cover creating real-time spatial audio directly in the browser, data compression of the audio for immersive media, and efficient binaural rendering.","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115975661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Generation Method for Immersive Bullet-Time Video Using an Omnidirectional Camera in VR Platform 基于VR平台全向摄像机的沉浸式子弹时间视频生成方法
O. Takeuchi, Hidehiko Shishido, Y. Kameda, Hansung Kim, I. Kitahara
{"title":"Generation Method for Immersive Bullet-Time Video Using an Omnidirectional Camera in VR Platform","authors":"O. Takeuchi, Hidehiko Shishido, Y. Kameda, Hansung Kim, I. Kitahara","doi":"10.1145/3264869.3264872","DOIUrl":"https://doi.org/10.1145/3264869.3264872","url":null,"abstract":"This paper proposes a generation method of immersive bullet-time video that continuously switches the images captured by multi-viewpoint omnidirectional cameras arranged around the subject. In ordinary bullet-time processing, it is possible to observe a point of interest (POI) at the same screen position by applying projective transformation to captured multi-viewpoint images. However, the observable area is limited by the field of view of the capturing cameras. Thus, a blank region is added to the displayed image, depending on the spatial relationship between the POI and the capturing camera. This seriously harms image quality (i.e., immersiveness). We solve this problem by applying omnidirectional cameras to bullet-time video production. Furthermore, by using the virtual reality platform for calibration of multi-viewpoint omnidirectional cameras and display of bullet-time video, fast and simple processing can be realised.","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128214934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Session details: Keynote Talk 1 会议详情:主题演讲1
A. Hilton
{"title":"Session details: Keynote Talk 1","authors":"A. Hilton","doi":"10.1145/3286580","DOIUrl":"https://doi.org/10.1145/3286580","url":null,"abstract":"","PeriodicalId":378296,"journal":{"name":"Proceedings of the 2018 Workshop on Audio-Visual Scene Understanding for Immersive Multimedia","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116916823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书