LapisGS:用于自适应流媒体的分层渐进式 3D 高斯拼接技术

Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi
{"title":"LapisGS:用于自适应流媒体的分层渐进式 3D 高斯拼接技术","authors":"Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi","doi":"arxiv-2408.14823","DOIUrl":null,"url":null,"abstract":"The rise of Extended Reality (XR) requires efficient streaming of 3D online\nworlds, challenging current 3DGS representations to adapt to\nbandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS\nthat supports adaptive streaming and progressive rendering. Our method\nconstructs a layered structure for cumulative representation, incorporates\ndynamic opacity optimization to maintain visual fidelity, and utilizes\noccupancy maps to efficiently manage Gaussian splats. This proposed model\noffers a progressive representation supporting a continuous rendering quality\nadapted for bandwidth-aware streaming. Extensive experiments validate the\neffectiveness of our approach in balancing visual fidelity with the compactness\nof the model, with up to 50.71% improvement in SSIM, 286.53% improvement in\nLPIPS, and 318.41% reduction in model size, and shows its potential for\nbandwidth-adapted 3D streaming and rendering applications.","PeriodicalId":501480,"journal":{"name":"arXiv - CS - Multimedia","volume":"15 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming\",\"authors\":\"Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi\",\"doi\":\"arxiv-2408.14823\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The rise of Extended Reality (XR) requires efficient streaming of 3D online\\nworlds, challenging current 3DGS representations to adapt to\\nbandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS\\nthat supports adaptive streaming and progressive rendering. Our method\\nconstructs a layered structure for cumulative representation, incorporates\\ndynamic opacity optimization to maintain visual fidelity, and utilizes\\noccupancy maps to efficiently manage Gaussian splats. This proposed model\\noffers a progressive representation supporting a continuous rendering quality\\nadapted for bandwidth-aware streaming. Extensive experiments validate the\\neffectiveness of our approach in balancing visual fidelity with the compactness\\nof the model, with up to 50.71% improvement in SSIM, 286.53% improvement in\\nLPIPS, and 318.41% reduction in model size, and shows its potential for\\nbandwidth-adapted 3D streaming and rendering applications.\",\"PeriodicalId\":501480,\"journal\":{\"name\":\"arXiv - CS - Multimedia\",\"volume\":\"15 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-08-27\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Multimedia\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2408.14823\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Multimedia","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.14823","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

扩展现实(XR)的兴起要求高效地流式传输 3D 在线世界,这对当前的 3DGS 表示法适应带宽受限的环境提出了挑战。本文提出了一种支持自适应流媒体和渐进式渲染的分层 3DGS LapisGS。我们的方法为累积表示构建了分层结构,结合了动态不透明度优化以保持视觉保真度,并利用占位图来有效管理高斯飞溅。该模型提供了一种渐进式表示方法,支持适合带宽感知流的连续渲染质量。广泛的实验验证了我们的方法在平衡视觉保真度和模型紧凑性方面的有效性,SSIM 提高了 50.71%,LPIPS 提高了 286.53%,模型大小减少了 318.41%,并显示了它在带宽适应型 3D 流媒体和渲染应用方面的潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
LapisGS: Layered Progressive 3D Gaussian Splatting for Adaptive Streaming
The rise of Extended Reality (XR) requires efficient streaming of 3D online worlds, challenging current 3DGS representations to adapt to bandwidth-constrained environments. This paper proposes LapisGS, a layered 3DGS that supports adaptive streaming and progressive rendering. Our method constructs a layered structure for cumulative representation, incorporates dynamic opacity optimization to maintain visual fidelity, and utilizes occupancy maps to efficiently manage Gaussian splats. This proposed model offers a progressive representation supporting a continuous rendering quality adapted for bandwidth-aware streaming. Extensive experiments validate the effectiveness of our approach in balancing visual fidelity with the compactness of the model, with up to 50.71% improvement in SSIM, 286.53% improvement in LPIPS, and 318.41% reduction in model size, and shows its potential for bandwidth-adapted 3D streaming and rendering applications.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信