Deep Learning Based Obstacle Awareness from Airborne Optical Sensors

Abstract

Aviation statistics identify collision with terrain and obstacles as a leading cause of helicopter accidents. Assisting helicopter pilots in detecting the presence of obstacles can partly mitigate the risk of collisions. However, only a limited number of helicopters in operation have an installed helicopter terrain awareness and warning system (HTAWS), while the cost of active obstacle warning systems remains prohibitive for many civil operators. In this work, we apply machine learning to automate obstacle detection and classification in combination with any commercially-available airborne optical sensor. While numerous techniques for learning-based object detection have appeared in the literature, many of them are data- and computation-intensive. Our approach seeks to balance the performance in regards to the detection and classification accuracy on the one hand, and the amount of training data and runtime performance on the other hand. Specifically, our approach combines the invariant feature extraction ability of pre-trained deep Convolutional Neural Networks (CNNs) and the high-speed training and classification ability of a novel, proprietary frequency-domain Support Vector Machine (SVM) method. In this paper, we present the CNN+SVM method for efficient obstacle detection and classification. We describe the experimental setup comprising datasets of pre-defined classes of obstacles – pylons, chimneys, antennas, towers, wind turbines, flying aircraft – from airborne video sequences of low-altitude helicopter flight. We analyze the performance results using average precision, average recall, and runtime performance metrics on representative test data. Finally, we present a simple architecture for a real-time, on-board evaluation of automatic vision-based obstacle detection.