Lat3D: Generating 3D assets in industrial paradigm via lattice deformation

Abstract

Automatic generation of 3D assets is one of the most promising future applications in AIGC. However, at current time, the prevailing 3D representation in AIGC still has a huge gap with commonly-used 3D design software, which leads to incapability of coherence and collaboration between machine generation and manual operation. To address this issue, we propose a 3D asset creation framework, Lat3D, which focus on Lattice representation that is compatible across mainstream 3D design software. This framework builds on a transformer network and distance-based matching to enable differentiable generation and supervision for lattices. To resolve the problem of biased error expectation in lattice matching, we leverage Importance Sampling to convert the deformed point sets into a uniform distribution. Besides, to activate vanishing lattices during optimization, we explicitly direct the enclosed lattices towards high-error regions by a well-designed distance function. Our framework is capable of producing lattices that are semantically decomposed, systematically structured, and closely aligned with modeling convention. With our developed Blender plugin, the generated lattices could be seamlessly imported into Blender projects for further 3D workflow. We conduct experiments of shape auto-encoding and single-view reconstruction to evaluate the quality of our created 3D assets.