Camera-enabled joint robotic-communication paradigm for UAVs mounted with mmWave radios

UAVs mounted with millimeter wave base stations will enable last-mile high bandwidth access, as well as help in rapidly deploying point to point aerial backhaul links. Because such transmitters use directional beamforming to increase capacity, UAV deployments require careful selection of the beamwidth. Even under regular hovering conditions, UAVs display minor relative rotations and displacements caused by GPS inaccuracies and environmental factors like wind. To ensure narrow beams are perfectly aligned in such practical conditions, we propose a beamforming framework that (i) fuses out-of-band information obtained from cameras and (ii) leverages antenna beam-patterns characterized online during flight. These inputs provide the UAV pair forming the link with an improved estimate of relative orientation, and furthermore, guide controlled and coordinated movements to ensure the mmWave beams remain aligned. We implement this joint robotics-communication framework within the robot operating system and evaluate the performance for emulated DJI M100 UAVs. Our results reveal 33% improvement in physical bitrate and 60.4% reduction in latency when compared to RF-only beam sweeping methods.