Markerless Pose Estimation and Tracking on Facial Regions for In Vivo Robot-Assisted Terahertz Sensing

Abstract

Terahertz (THz) light’s sensitivity to water, non-ionizing nature, and sub-millimeter resolution make it ideal for medical imaging, particularly for diagnosing and managing skin conditions such as eczema, psoriasis, and skin cancer. Traditional handheld THz probes suffer from positional errors, tremors, and inconsistent contact pressure, which worsen with prolonged use. To address this, robotic THz acquisition systems can be leveraged to segment regions of interest on the skin surface and compute the appropriate probe pose for scanning. This paper proposes a markerless method for automatic identification and estimation of the pose for the placement of the THz probe over high-curvature facial regions. An RGB-D camera is utilized to select, recognize, and track the scanning targets on the patient using an end-to-end network-based model for 3D human face mesh prediction. We then solve for the landing pose to position the probe at normal incidence to the tissue surface at the target scan regions. We evaluated scanning area localization accuracy, motion execution accuracy, and THz acquisition capability using a mannequin head and human volunteers. Results demonstrated target registration error of 1.71 mm, scan target localization accuracy of less than 3 mm and probe landing pose estimation accuracy of $2.769~\pm 3.02$ mm in X, $6.62~\pm ~0.6$ mm in Y and $4.792~\pm ~2.18$ in Z and orientation error of $1.6\pm 0.8$ degrees.