UAVSensor Fusion with Latent-Dynamic Conditional Random Fields in Coronal Plane Estimation

Abstract

We present a real-time body orientation estimation in a micro-Unmanned Air Vehicle video stream. This work is part ofafully autonomous UAVsystem which can maneuver to face a single individual in challenging outdoor environments. Our body orientation estimation consists of the following steps: (a) obtaining a set ofvisual appearance models for each body orientation, where each model is tagged with a set of scene information (obtained from sensors), (b) exploiting the mutual information of on-board sensors using latent-dynamic conditional random fields (WCRF), (c) Characterizing each visual appearance model with the most discriminative sensor information, (d) fast estimation ofbody orientation during the test flights given theWCRF parameters and the corresponding sensor readings. The key aspects of our approach is to add sparsity to the sensor readings with latent variables followed by long range dependency analysis. Experimental results obtained over real-time video streams demonstrate a significant improvement in both speed (l5-fps) and accuracy (72%) compared to the state of the art techniques that only rely on visual data. Video demonstration ofour autonomous flights (both from ground view and aerial view) are included in the supplementary material.