VR2Gather:一个用于自适应、多方实时通信的协作、社交虚拟现实系统

IF 2.3 4区计算机科学 Q2 COMPUTER SCIENCE, HARDWARE & ARCHITECTURE

IEEE MultiMedia Pub Date : 2023-04-01 DOI:10.1109/MMUL.2023.3263943

Irene Viola, Jack Jansen, S. Subramanyam, Ignacio Reimat, Pablo César

{"title":"VR2Gather:一个用于自适应、多方实时通信的协作、社交虚拟现实系统","authors":"Irene Viola, Jack Jansen, S. Subramanyam, Ignacio Reimat, Pablo César","doi":"10.1109/MMUL.2023.3263943","DOIUrl":null,"url":null,"abstract":"Virtual reality telecommunication systems promise to overcome the limitations of current real-time teleconferencing solutions by enabling a better sense of immersion and fostering more natural interpersonal interactions. Many solutions that currently enable immersive teleconferencing employ synthetic avatars to represent their users. However, photorealistic reconstructions have been shown to increase the sense of presence with respect to synthetic avatars in teleimmersive scenarios. In this article, we present VR2Gather, a costumizable, end-to-end system to transmit volumetric contents in multiparty, real-time communication. We present the architecture and evaluate the costs and benefits of using different modules and transport mechanisms in terms of CPU usage, latency, and bandwidth. Moreover, we report the user experience based on applications the system has been used for and how it was customized to meet the requirements using different acquisition and rendering modules.","PeriodicalId":13240,"journal":{"name":"IEEE MultiMedia","volume":"30 1","pages":"48-59"},"PeriodicalIF":2.3000,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"VR2Gather: A Collaborative, Social Virtual Reality System for Adaptive, Multiparty Real-Time Communication\",\"authors\":\"Irene Viola, Jack Jansen, S. Subramanyam, Ignacio Reimat, Pablo César\",\"doi\":\"10.1109/MMUL.2023.3263943\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Virtual reality telecommunication systems promise to overcome the limitations of current real-time teleconferencing solutions by enabling a better sense of immersion and fostering more natural interpersonal interactions. Many solutions that currently enable immersive teleconferencing employ synthetic avatars to represent their users. However, photorealistic reconstructions have been shown to increase the sense of presence with respect to synthetic avatars in teleimmersive scenarios. In this article, we present VR2Gather, a costumizable, end-to-end system to transmit volumetric contents in multiparty, real-time communication. We present the architecture and evaluate the costs and benefits of using different modules and transport mechanisms in terms of CPU usage, latency, and bandwidth. Moreover, we report the user experience based on applications the system has been used for and how it was customized to meet the requirements using different acquisition and rendering modules.\",\"PeriodicalId\":13240,\"journal\":{\"name\":\"IEEE MultiMedia\",\"volume\":\"30 1\",\"pages\":\"48-59\"},\"PeriodicalIF\":2.3000,\"publicationDate\":\"2023-04-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE MultiMedia\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1109/MMUL.2023.3263943\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE MultiMedia","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1109/MMUL.2023.3263943","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, HARDWARE & ARCHITECTURE","Score":null,"Total":0}

引用次数: 0

摘要

虚拟现实通信系统有望通过实现更好的沉浸感和促进更自然的人际互动来克服当前实时电话会议解决方案的局限性。目前，许多实现沉浸式电话会议的解决方案都使用合成化身来代表用户。然而，在远程沉浸场景中，与合成头像相比，逼真的重建已经被证明可以增加存在感。在本文中，我们介绍了VR2Gather，这是一个可定制的端到端系统，用于在多方实时通信中传输容量内容。我们介绍了体系结构，并从CPU使用、延迟和带宽的角度评估了使用不同模块和传输机制的成本和收益。此外，我们还报告了基于应用程序的用户体验，以及如何使用不同的获取和呈现模块来定制它以满足需求。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

VR2Gather: A Collaborative, Social Virtual Reality System for Adaptive, Multiparty Real-Time Communication

Virtual reality telecommunication systems promise to overcome the limitations of current real-time teleconferencing solutions by enabling a better sense of immersion and fostering more natural interpersonal interactions. Many solutions that currently enable immersive teleconferencing employ synthetic avatars to represent their users. However, photorealistic reconstructions have been shown to increase the sense of presence with respect to synthetic avatars in teleimmersive scenarios. In this article, we present VR2Gather, a costumizable, end-to-end system to transmit volumetric contents in multiparty, real-time communication. We present the architecture and evaluate the costs and benefits of using different modules and transport mechanisms in terms of CPU usage, latency, and bandwidth. Moreover, we report the user experience based on applications the system has been used for and how it was customized to meet the requirements using different acquisition and rendering modules.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

IEEE MultiMedia 工程技术-计算机：理论方法

CiteScore

6.40

自引率

3.10%

发文量

审稿时长

>12 weeks

期刊介绍： The magazine contains technical information covering a broad range of issues in multimedia systems and applications. Articles discuss research as well as advanced practice in hardware/software and are expected to span the range from theory to working systems. Especially encouraged are papers discussing experiences with new or advanced systems and subsystems. To avoid unnecessary overlap with existing publications, acceptable papers must have a significant focus on aspects unique to multimedia systems and applications. These aspects are likely to be related to the special needs of multimedia information compared to other electronic data, for example, the size requirements of digital media and the importance of time in the representation of such media. The following list is not exhaustive, but is representative of the topics that are covered: Hardware and software for media compression, coding & processing; Media representations & standards for storage, editing, interchange, transmission & presentation; Hardware platforms supporting multimedia applications; Operating systems suitable for multimedia applications; Storage devices & technologies for multimedia information; Network technologies, protocols, architectures & delivery techniques intended for multimedia; Synchronization issues; Multimedia databases; Formalisms for multimedia information systems & applications; Programming paradigms & languages for multimedia; Multimedia user interfaces; Media creation integration editing & management; Creation & modification of multimedia applications.