Seg and Ref: A Newly Developed Toolset for Artificial Intelligence-Powered Segmentation and Interactive Refinement for Labor-Saving Three-Dimensional Reconstruction.

Abstract

Traditional three-dimensional reconstruction is labor-intensive owing to manual segmentation; this can be addressed by developing artificial intelligence-driven automated segmentation. However, it is limited by a lack of user-friendly tools for morphologists. We present a workflow for three-dimensional reconstruction using our artificial intelligence-powered segmentation tool. Specifically, we developed an interactive toolset, "Seg & Ref," to overcome the abovementioned challenges by enabling artificial intelligence-powered segmentation and easy mask editing without requiring a command-line setup. We demonstrated a three-dimensional reconstruction workflow using serial sections of a Carnegie Stage 15 human embryo. Automated segmentation (Step 1) was performed using the graphical user interface, "SAM2 GUI for Img Seq," which utilizes the Segment Anything Model 2 and supports interactive segmentation through a web-based interface. Users specify target structures via box prompts, and the results are propagated across all images for batch segmentation. The segmentation masks were reviewed and corrected (Step 2) using "Segment Editor PP," a PowerPoint-based tool enabling interactive mask refinement. Finally, the corrected masks were imported into the 3D Slicer application for reconstruction (Step 3). Our three-dimensional reconstruction visualized key structures, including the spinal cord, veins, aorta, mesonephros, gut, heart, trachea, liver, and peritoneal cavity. The reconstructed models accurately represented their spatial relationships and morphologies. This provides a labor-saving approach for three-dimensional reconstruction workflows owing to their optimization for serial sections, versatility, and accessibility without programming expertise. Therefore, morphological research can be enhanced by precise segmentation using intuitive and user-friendly interfaces of "Seg & Ref."