MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation

Hyunwoo Kim, Itai Lang, Noam Aigerman, Thibault Groueix, Vladimir G. Kim, Rana Hanocka
{"title":"MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation","authors":"Hyunwoo Kim, Itai Lang, Noam Aigerman, Thibault Groueix, Vladimir G. Kim, Rana Hanocka","doi":"arxiv-2408.14899","DOIUrl":null,"url":null,"abstract":"We propose MeshUp, a technique that deforms a 3D mesh towards multiple target\nconcepts, and intuitively controls the region where each concept is expressed.\nConveniently, the concepts can be defined as either text queries, e.g., \"a dog\"\nand \"a turtle,\" or inspirational images, and the local regions can be selected\nas any number of vertices on the mesh. We can effectively control the influence\nof the concepts and mix them together using a novel score distillation\napproach, referred to as the Blended Score Distillation (BSD). BSD operates on\neach attention layer of the denoising U-Net of a diffusion model as it extracts\nand injects the per-objective activations into a unified denoising pipeline\nfrom which the deformation gradients are calculated. To localize the expression\nof these activations, we create a probabilistic Region of Interest (ROI) map on\nthe surface of the mesh, and turn it into 3D-consistent masks that we use to\ncontrol the expression of these activations. We demonstrate the effectiveness\nof BSD empirically and show that it can deform various meshes towards multiple\nobjectives.","PeriodicalId":501174,"journal":{"name":"arXiv - CS - Graphics","volume":"26 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Graphics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.14899","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

We propose MeshUp, a technique that deforms a 3D mesh towards multiple target concepts, and intuitively controls the region where each concept is expressed. Conveniently, the concepts can be defined as either text queries, e.g., "a dog" and "a turtle," or inspirational images, and the local regions can be selected as any number of vertices on the mesh. We can effectively control the influence of the concepts and mix them together using a novel score distillation approach, referred to as the Blended Score Distillation (BSD). BSD operates on each attention layer of the denoising U-Net of a diffusion model as it extracts and injects the per-objective activations into a unified denoising pipeline from which the deformation gradients are calculated. To localize the expression of these activations, we create a probabilistic Region of Interest (ROI) map on the surface of the mesh, and turn it into 3D-consistent masks that we use to control the expression of these activations. We demonstrate the effectiveness of BSD empirically and show that it can deform various meshes towards multiple objectives.
MeshUp:通过混合分数蒸馏实现多目标网格变形
我们提出的 MeshUp 是一种针对多个目标概念对三维网格进行变形的技术,它可以直观地控制每个概念所表达的区域。我们可以使用一种新颖的分数蒸馏方法(称为混合分数蒸馏法(BSD))有效地控制概念的影响并将它们混合在一起。BSD 对扩散模型的去噪 U 网的每个注意层进行操作,因为它提取并将每个目标的激活状态注入统一的去噪管道,并从中计算出变形梯度。为了定位这些激活的表达,我们在网格表面创建了一个概率感兴趣区域(ROI)图,并将其转化为三维一致的掩码,用来控制这些激活的表达。我们通过经验证明了 BSD 的有效性,并表明它可以使各种网格向多个目标变形。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信