Filling holes in 3D scanned model base on 2D image inpainting

Abstract

Nowadays 3D model scanning is widely used for many purpose. Although scanning technique had become much advanced then earliest, users may still obtain damaged model due to unfamiliar with scanning technique, then we need to repair the model by doing hole-filling and surface reconstruction. In this paper we introduce an approach to fill holes in 3D mesh by using 2D Image Inpainting technique. We create image of depth value to represent depth information and image of color to represent color of surfaces, then inpaint those images and use result images to reconstruct holes of model. The advantage of using Image Inpainting is the flexibility of using image inpainting method, we can use different inpainting method for different purpose. We obtain color and depth image by doing 3D projection and find holes in model by doing hole detection, then we inpaint these images and do 2D to 3D transformation of new added pixels back to 3D space. This work can use any inpaint algorithm without modification. In this paper we use both structural inpainting and textural inpainting, structural inpainting is used for reconstructing depth values and textural inpainting is used for reconstructing texture of surface. The results of our experiment proves that our approach is suitable for model that has uncomplicated structure and texture on the surface.