Liver mask-guided SAM-enhanced dual-decoder network for landmark segmentation in AR-guided surgery.

Abstract

Purpose: In augmented reality (AR)-guided laparoscopic liver surgery, accurate segmentation of liver landmarks is crucial for precise 3D-2D registration. However, existing methods struggle with complex structures, limited data, and class imbalance. In this study, we propose a novel approach to improve landmark segmentation performance by leveraging liver mask prediction.

Methods: We propose a dual-decoder model enhanced by a pre-trained segment anything model (SAM) encoder, where one decoder segments the liver and the other focuses on liver landmarks. The SAM encoder provides robust features for liver mask prediction, improving generalizability. A liver-guided consistency constraint establishes fine-grained spatial consistency between liver regions and landmarks, enhancing segmentation accuracy through detailed spatial modeling.

Results: The proposed method achieved state-of-the-art performance in liver landmark segmentation on two public laparoscopic datasets. By addressing feature entanglement, the dual-decoder framework with SAM and consistency constraints significantly improved segmentation in complex surgical scenarios.

Conclusion: The SAM-enhanced dual-decoder network, incorporating liver-guided consistency constraints, offers a promising solution for 2D landmark segmentation in AR-guided laparoscopic surgery. By mutually reinforcing liver mask and landmark segmentation, the method achieves improved accuracy and robustness for intraoperative applications.