3D modeling from a single image via a novel dual-decoder framework for Agile design

Abstract

In the fast-paced manufacturing industry, rapid and efficient product design is essential for meeting customer demands and maintaining a competitive edge. Despite advancements, transforming 2D design concepts into accurate 3D models remains a complex challenge, primarily due to the non-differentiability of traditional rendering processes that hinder gradient-based optimizations. To address this limitation, this paper introduces an innovative dual-decoder architecture that effectively separates the shape and color components of 3D models. By assigning separate decoders for vertex positions and color assignment, our proposed model enables targeted optimization of each, leading to more refined and authentic 3D reconstructions. Moreover, we have overcome the non-differentiability issue, enabling gradient-based learning through the incorporation of differentiable rendering techniques. These techniques facilitate gradient-based optimization, paving the way for data-driven enhancements in the design process. Our empirical research has demonstrated the effectiveness of our approach in generating high-fidelity 3D models from 2D inputs. Additionally, we have shed light on the sensitivity of hyperparameters within our framework, offering valuable insights for future model refinement and optimization. In summary, our research provides valuable insights into enhancing 3D modeling frameworks, thereby contributing to incremental progress in the field of computer-aided design and manufacturing.