Development of a Vision-Based Ground Target Localization System for Flapping-Wing Flying Robots

Abstract

Implementing ground target localization remains difficult for a flapping-wing flying robot (FWFR) owing to its intrinsically periodic flapping motion and low load capacity. In this article, a vision-based ground target localization system is developed for FWFRs by using two different focal length cameras. First, to counteract the image jitter during the flight of FWFRs, we design a lightweight camera stabilizer based on the motion characteristics of FWFRs and adopt the active disturbance rejection control (ADRC) method rather than the traditional PID control method to obtain smoother aerial videos. Second, a dual-camera system consisting of long- and short-focal-length cameras is designed to eliminate the impact of flight altitude on target detection performance. We combine the digital zoom algorithm with the dual-camera system and propose an improved target detection algorithm based on YOLOv8, which successfully detects ground targets captured by the dual-camera system at different flight altitudes. Then, the latitude and longitude coordinates of the ground target are estimated by fusing the information from cameras and other onboard sensors. Finally, extensive flight experiments carried out using our self-developed FWFR named USTB-Hawk demonstrate the effectiveness of the designed ground target localization system. Our experimental results show that at a flight altitude of 100 m, the average localization error is 4.4 m. At a flight altitude of 300 m, the average localization error is 6.0 m. This provides insights into performing vision-based ground target localization tasks for the FWFR.