通过 SO(2)-Equivariant 高斯雕刻网络进行单视角三维重建

Ruihan Xu, Anthony Opipari, Joshua Mah, Stanley Lewis, Haoran Zhang, Hanzhe Guo, Odest Chadwicke Jenkins
{"title":"通过 SO(2)-Equivariant 高斯雕刻网络进行单视角三维重建","authors":"Ruihan Xu, Anthony Opipari, Joshua Mah, Stanley Lewis, Haoran Zhang, Hanzhe Guo, Odest Chadwicke Jenkins","doi":"arxiv-2409.07245","DOIUrl":null,"url":null,"abstract":"This paper introduces SO(2)-Equivariant Gaussian Sculpting Networks (GSNs) as\nan approach for SO(2)-Equivariant 3D object reconstruction from single-view\nimage observations. GSNs take a single observation as input to generate a Gaussian splat\nrepresentation describing the observed object's geometry and texture. By using\na shared feature extractor before decoding Gaussian colors, covariances,\npositions, and opacities, GSNs achieve extremely high throughput (>150FPS).\nExperiments demonstrate that GSNs can be trained efficiently using a multi-view\nrendering loss and are competitive, in quality, with expensive diffusion-based\nreconstruction algorithms. The GSN model is validated on multiple benchmark\nexperiments. Moreover, we demonstrate the potential for GSNs to be used within\na robotic manipulation pipeline for object-centric grasping.","PeriodicalId":501031,"journal":{"name":"arXiv - CS - Robotics","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Single-View 3D Reconstruction via SO(2)-Equivariant Gaussian Sculpting Networks\",\"authors\":\"Ruihan Xu, Anthony Opipari, Joshua Mah, Stanley Lewis, Haoran Zhang, Hanzhe Guo, Odest Chadwicke Jenkins\",\"doi\":\"arxiv-2409.07245\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This paper introduces SO(2)-Equivariant Gaussian Sculpting Networks (GSNs) as\\nan approach for SO(2)-Equivariant 3D object reconstruction from single-view\\nimage observations. GSNs take a single observation as input to generate a Gaussian splat\\nrepresentation describing the observed object's geometry and texture. By using\\na shared feature extractor before decoding Gaussian colors, covariances,\\npositions, and opacities, GSNs achieve extremely high throughput (>150FPS).\\nExperiments demonstrate that GSNs can be trained efficiently using a multi-view\\nrendering loss and are competitive, in quality, with expensive diffusion-based\\nreconstruction algorithms. The GSN model is validated on multiple benchmark\\nexperiments. Moreover, we demonstrate the potential for GSNs to be used within\\na robotic manipulation pipeline for object-centric grasping.\",\"PeriodicalId\":501031,\"journal\":{\"name\":\"arXiv - CS - Robotics\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Robotics\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.07245\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Robotics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.07245","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

本文介绍了 SO(2)-Equivariant 高斯雕刻网络(GSNs),它是一种从单视角图像观测中重建 SO(2)-Equivariant 三维物体的方法。高斯雕刻网络将单个观测数据作为输入,生成描述观测对象几何和纹理的高斯拼接表示。通过在解码高斯颜色、协方差、位置和不透明度之前使用共享特征提取器,GSN 实现了极高的吞吐量(>150FPS)。GSN 模型在多个基准实验中得到了验证。此外,我们还展示了 GSN 与机器人操纵流水线一起用于以物体为中心的抓取的潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Single-View 3D Reconstruction via SO(2)-Equivariant Gaussian Sculpting Networks
This paper introduces SO(2)-Equivariant Gaussian Sculpting Networks (GSNs) as an approach for SO(2)-Equivariant 3D object reconstruction from single-view image observations. GSNs take a single observation as input to generate a Gaussian splat representation describing the observed object's geometry and texture. By using a shared feature extractor before decoding Gaussian colors, covariances, positions, and opacities, GSNs achieve extremely high throughput (>150FPS). Experiments demonstrate that GSNs can be trained efficiently using a multi-view rendering loss and are competitive, in quality, with expensive diffusion-based reconstruction algorithms. The GSN model is validated on multiple benchmark experiments. Moreover, we demonstrate the potential for GSNs to be used within a robotic manipulation pipeline for object-centric grasping.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信