DIRA: Diffusion-Based Imitation-to-Reinforcement Adaptation for Task Automation of Surgical Robots

Abstract

The autonomy of surgical robots can enhance the safety and efficiency of surgeries. However, the diverse surgical skill expressions in expert demonstrations create multimodal patterns that can lead to suboptimal outcomes under unimodal policy learning methods. Additionally, there can be distribution shifts due to imbalances between positive and negative samples in surgical task data, and the learned policies may fail when encountering unseen states or external perturbations. To address this, we propose Diffusion-Based Imitation-to-Reinforcement Adaptation (DIRA), a three-stage framework that leverages a conditional diffusion-based implicit policy for multimodal action modeling, a value-consistent critic warm-up for stable value estimation, and an online strategic perturbation adaptation for smooth imitation-to-reinforcement learning transition. Experiments on the SurRoL platform show that the proposed DIRA achieves a higher success rate than existing methods, with an average improvement of 24 percentage points and up to 39 percentage points on complex tasks. Moreover, the experiments with Real-World Demonstrations further demonstrate the potential of DIRA for real-world surgical task automation.