As-scanned point cloud generation using structured-light simulation and machine learning-based coverage prediction

Although several methods have been proposed for generating as-scanned point clouds, i.e. point clouds incorporating various realistic artefacts that would appear if the corresponding real objects were digitized for real, most of them still fail to take into account the complex phenomena that occur in a real acquisition devices. This paper presents a new way of artificially generating point clouds by combining simulation and machine learning. Starting from the CAD model of the object to be virtually scanned and from a scan configuration, structured light simulation first allows reconstructing a preliminary 3D point cloud. Then, a coverage prediction network is used to predict the regions that would be acquired if a real acquisition was to be done. The prediction model has been trained from a large database of scan configurations and point clouds scanned for real. Finally, filtering and cropping are performed to fine-tune the generated point cloud. Experiments confirm that this method can generate point clouds very close to those that a real scanner would acquire, as shown by several metrics characterizing both local and global similarity. Such a virtual scanning technique enables the rapid generation of large quantities of realistic point clouds, especially when compared to the time-consuming and costly processes involved in using physical acquisition systems. This opens up new perspectives in terms of access to realistic point cloud databases, in particular for the development of various AI-based approaches.