VLM-Based Mesh Segmentation Framework for DDH Robotic Ultrasound.

Abstract

For robotic ultrasound targeting developmental dysplasia of the hip (DDH), reconstructed infant body-surface meshes should preserve the target surface while excluding bed, support, and background geometry. This study proposes a structured vision-language model (VLM)-based multi-view segmentation method for infant phantom mesh cleanup. Text-guided 2D masks provide foreground and background evidence; mask-quality statistics and structured VLM assessments generate view-specific foreground-support weights that undergo deterministic risk suppression before target/non-target classification of 3D vertices. Hard voting and Rule-weighted voting were baselines under identical projection and post-processing settings. We evaluated one physical infant phantom across 150 separate acquisition-reconstruction trials under controlled conditions, each comprising 40 voting views and 10 held-out evaluation views, yielding 1500 manually annotated reference masks. Structured VLM voting achieved a Balanced score (intersection over union minus the 2-pixel tolerant leakage rate) of 0.804 ± 0.085, versus 0.801 ± 0.084 and 0.793 ± 0.086 for Rule-weighted and Hard voting. This leakage rate was 0.087 ± 0.043, versus 0.089 ± 0.042 and 0.092 ± 0.043; mean Coverage remained similar (0.992-0.993). The Balanced-score gain and leakage-rate reduction relative to Rule-weighted voting remained significant after Holm correction (adjusted p = 0.029 and p = 0.033, respectively). Validation using real-infant data, 3D reference standards, and closed-loop robotic scanning is still required.