Open3DGen

Abstract

This paper presents the first entirely open-source and cross-platform software called Open3DGen for reconstructing photorealistic textured 3D models from RGB-D images. The proposed software pipeline consists of nine main stages: 1) RGB-D acquisition; 2) 2D feature extraction; 3) camera pose estimation; 4) point cloud generation; 5) coarse mesh reconstruction; 6) optional loop closure; 7) fine mesh reconstruction; 8) UV unwrapping; and 9) texture projection. This end-to-end scheme combines multiple state-of-the-art techniques and provides an easy-to-use software package for real-time 3D model reconstruction and offline texture mapping. The main innovation lies in various Structure-from-Motion (SfM) techniques that are used with additional depth data to yield high-quality 3D models in real-time and at low cost. The functionality of Open3DGen has been validated on AMD Ryzen 3900X CPU and Nvidia GTX1080 GPU. This proof-of-concept setup attains an average processing speed of 15 fps for 720p (1280x720) RGBD input without the offline backend. Our solution is shown to provide competitive 3D mesh quality and execution performance with the state-of-the-art commercial and academic solutions.