Autonomous Flying Cameraman with Embedded Person Detection and Tracking while Applying Cinematographic Rules

Abstract

Unmanned Aerial Vehicles (UAVs) enable numerous applications such as search and rescue operations, structural inspection of buildings, crop growth analysis in agriculture, performing 3D reconstruction and so on. For such applications, currently the UAV is steered manually. However, in this paper we aim to record semi-professional video footage (e.g. concerts, sport events) using fully autonomous UAVs. Evidently, this is challenging since we need to detect and track the actor on-board a UAV in real-time, while automatically – and smoothly – controlling the UAV based on these detections. For this, all four DOF (Degrees of freedom) are controlled in separate simultaneous control loops by our vision-based algorithms. Furthermore cinematographic rules need to be taken into account (e.g. the rule of thirds) which position the actor at the visually optimal location in the frame. We extensively validated our algorithms: each control loop and the overall final system is thoroughly evaluated with respect to both accuracy and control speed. We show that our system is able to efficiently control the UAV such that professional recordings are obtained.