Augmented reality visualization of biomechanical wall stresses on abdominal aortic aneurysms using artificial intelligence

Abstract

The number of medical images taken has continued to increase year over year for an aging population in the United States. It has been shown that patients understand their diagnoses better when shown a 2D or 3D image of their respective diseases. However, clinicians do not regularly show patients their images as it requires additional time and processing. In this experiment, we demonstrate the use of augmented reality to visualize abdominal aortic aneurysms using a previously developed artificial intelligence engine. Our group further expanded the number of cases used for training the stress prediction model to a total of 274 patients (206 used for training or ∼ 5.4 million nodes, and 68 for testing or ∼1.8 million nodes). Medical images undergo automated segmentation, and wall stresses are predicted on the 3D surface of aneurysms to view a heat map. The pipeline includes introducing elements into the Microsoft HoloLens 2 ecosystem to view models and additional analytics, enabling clinicians and patients to view the biomechanical status without the need for a computational or imaging expert. The proposed clinical workflow would allow a local server to process medical imaging data, generate point clouds, predict wall stresses on individual points, and create a 3D model with a colormap to view in augmented reality. The study revealed that neural networks and ensemble boosted tress models predicted the wall stresses more accurately (when compared to ground truth finite element analysis results). The approach is not limited to the HoloLens 2 ecosystem but can be used with other emerging augmented or virtual reality hardware systems.

Summary: Patient understanding of their diagnosis improves when they are shown medical images by clinicians. An artificial intelligence-based clinical workflow has been developed to visualize the biomechanical status using augmented reality, providing additional information to clinicians and patients. Our research improves the tools that are available to clinicians and patients to help provide a better understanding of diagnosis and potentially prognosis.