Automatic three-dimensional reconstruction of transparent objects with multiple optimization strategies under limited constraints

Abstract

Reconstructing transparent objects with limited constraints has long been considered a highly challenging problem. Due to the complex interaction between transparent objects and light, which involves intricate refraction and reflection relationships, traditional three-dimensional (3D) reconstruction methods are less than effective for transparent objects. To address this issue, this study proposes a 3D reconstruction method specifically designed for transparent objects. Incorporating multiple optimization strategies, the method works under limited constraints to achieve the automatic reconstruction of transparent objects with only a few transparent object images in any known environment, without the need for specific data collection devices or environments. The proposed method makes use of automatic image segmentation and modifies the network interface and structure of the PointNeXt algorithm to introduce the TransNeXt network, which enhances normal features, optimizes weight attenuation, and employs a preheating cosine annealing learning rate. We use several steps to reconstruct the complete 3D shape of transparent objects. First, we initialize the transparent shape with a visual hull reconstructed with the contours obtained by the TOM-Net. Then, we construct the normal reconstruction network to estimate the normal values. Finally, we reconstruct the complete 3D shape using the TransNeXt network. Multiple experiments show that the TransNeXt network exhibits superior reconstruction performance to other networks and can effectively perform the automatic reconstruction of transparent objects even under limited constraints.