Recognition and 3D Visualization of Human Body Parts and Bone Areas Using CT Images

Abstract The advent of medical imaging significantly assisted in disease diagnosis and treatment. This study introduces to a framework for detecting several human body parts in Computerised Tomography (CT) images formatted in DICOM files. In addition, the method can highlight the bone areas inside CT images and transform 2D slices into a visual 3D model to illustrate the structure of human body parts. Firstly, we leveraged shallow convolutional Neural Networks to classify body parts and detect bone areas in each part. Then, Grad-CAM was applied to highlight the bone areas. Finally, Insight and Visualization libraries were utilized to visualize slides in 3D of a body part. As a result, the classifiers achieved 98 % in F1-score in the classification of human body parts on a CT image dataset, including 1234 slides capturing body parts from a woman for the training phase and 1245 images from a male for testing. In addition, distinguishing between bone and non-bone images can reach 97 % in F1-score on the dataset generated by setting a threshold value to reveal bone areas in CT images. Moreover, the Grad-CAM-based approach can provide clear, accurate visualizations with segmented bones in the image. Also, we successfully converted 2D slice images of a body part into a lively 3D model that provided a more intuitive view from any angle. The proposed approach is expected to provide an interesting visual tool for supporting doctors in medical image-based disease diagnosis.