Surgical Action and Instrument Detection Based on Multiscale Information Fusion

The detection of surgical actions and instruments plays a very important role in computer-assisted endoscopic surgery. However, organ deformation and narrow surgical field increase the task difficulty. Accordingly, the problems of the detection of surgical actions and instruments have not been solved yet. In this paper, we proposed a multiscale fusion feature pyramid network (MSF-FPN) to merge low-level semantic information and high-level semantic information. Firstly, the feature map effectively aggregates the information by the initial layer of the pyramid network, and then diverges after the cross-transmission of the feature information in the middle layer. Finally, a strong semantic feature map was obtained in the output layer. Experiments verified that the average precision of the proposed MSF-FPN on the public endoscopic surgeon action detection (ESAD) dataset is increased by 2.9% and 1.5% compared with the general FPN and path aggregation network (PANet), and the average precision on the proposed cataract-based object detection (COD) dataset is increased by 4.3% and 2.6%, respectively.