Transformer-based aerial robot tracking system in environments with wind disturbances

Unmanned aerial vehicles (UAVs) are increasingly used in agriculture, surveillance, and search and rescue. However, maintaining stable flight and accurate navigation in dynamic environments, especially with wind disturbances, remains a challenge. Traditional navigation systems often struggle with unreliable sensor data, complicating pose estimation and tracking. This article proposes an advanced master–slave UAV system combining a transformer-based model with YOLO for enhanced tracking in wind-affected environments. YOLO performs real-time object detection, extracting feature points matched with known landmarks to estimate the UAV’s position. To address the challenges of wind disturbances, we simulate various wind conditions and train the model under different wind disturbance environments. Using transformer-based trajectory and pose predictions, we provide control compensation to counteract the effects of wind disturbances, ensuring stable flight in dynamic conditions. The pose estimation is refined by integrating visual data with inertial measurement unit (IMU) data using transformer architectures. A vision-based formation control strategy is introduced for precise relative positioning in multi-UAV formations. Initially designed for three UAVs, this strategy is extended to handle larger formations and complex geometric shapes, focusing on maintaining a triangle formation. A graph-based dynamic formation control framework enables real-time adaptation to formation changes and environmental conditions. The approach improves MPC control with a transformer model, enhancing adaptability to wind disturbances. The system’s effectiveness is validated using webots simulations, demonstrating its ability to track UAVs and adapt to challenging environmental conditions. A theorem proves the convergence of the control law using Lyapunov’s direct method, ensuring that formation errors decay over time. Comparative experiments and webots simulations confirm the approach’s feasibility, validating its robustness in maintaining precise formation control under dynamic environmental factors. Finally, we validate the reliability of our method in real-world environments, confirming its practical applicability.