Multi-Object Robot Visual Servo Based on YOLOv3

Abstract

Aiming at the low robustness of image feature extractor in Image-Based Visual Servo (IBVS), a robot visual servo method based on object detection neural network YOLOv3 is proposed. By improving the output layer of YOLOv3 and adding attitude angle of camera, the pixel coordinate and depth information of feature points, the robustness of the IBVS system based on point features is improved while it can cope with multi-type and multi-instance objects, and the problem of the image Jacoby matrix falling into singularity caused by excessive rotation angle error of the optical axis is avoided. The visual servo convergence is accelerated. Meanwhile, the network training data generation algorithm of the desired image is used to replace the traditional manual data annotation, which reduces the cost of data acquisition, and the data enhancement method ensures the generalization performance of the training model.