Evaluation of flaw detection algorithm using simulated X-ray computed tomography of ground truth data

A framework to generate simulated X-ray computed tomography (XCT) data of ground truth (denoted here as ‘GT’) flaws was developed for evaluation of flaw detection algorithms using image comparison metrics. The flaws are mimicking some of those found in additively manufactured parts. The simulated flaw structure gives a GT data set with which to quantitatively evaluate, by calculating exact errors, the results of flaw detection algorithms applied to simulated XCT images. The simulated data avoid time-consuming manual voxel labeling steps needed for many physical data sets to generate GT images. The voxelated pore meshes that exactly match GT images avoid approximations due to converting continuum pore meshes to voxelated GT images. Spherical pores of varying sizes were randomly distributed near the surface and interior of a cylindrical part. XCT simulation was carried out on the structure at three different signal-to-noise levels by changing the number of frames integrated for each projection. Two different local thresholding algorithms (a commercial code and the Bernsen method) and a global thresholding algorithm (Otsu) were used to segment images using varying sets of algorithm parameters. The segmentation results were evaluated with various image evaluation metrics, which showed different behaviors for the three algorithms regarding “closeness” to the GT data. An approach to optimize the thresholding parameters is demonstrated for the commercial flaw detection algorithm based on the semantic evaluation metrics. A framework to evaluate pore sizing error and binary probability of detection was further demonstrated to compare the optimization results.