Empowering UAV scene perception by semantic spatio-temporal features

The use of unmanned aerial vehicles (UAVs) is becoming a key asset in different application domains: from the military to surveillance tasks; to filming and journalism to shipping and delivery; to disaster monitoring to rescue operation and healthcare. One of the most desirable UAV capabilities is a human-like scenario understanding, i.e., the object recognition and interactions with other objects and with the environment, through the scene evolution, in order to get a high-view scenario description. The paper presents a semantic-enhanced approach for UAV-based surveillance systems. The video analysis is extended and enriched with semantic high level data to provide a global view of the video scenes. Semantic Web technologies provide the expressive power to describe semantically scenes appearing in the videos. The synergy between the video tracking methods and the semantic web technologies provides a new high-level human-like interpretation of the scenario. The approach focuses on the event understanding at semantic level: it is coded as spatio-temporal relation which joins fixed or mobile objects, with respect to a given temporal sequence of video frames. The system is composed of two macro components: one devoted to the tracking activities, i.e., the object identification and classification, the other enriches tracking data semantically, where the ontology-based scenario model is the bridge between the components. A reasoning component applied to the semantic knowledge, extracted from the scenario, infers new statements that describe the detected events occurring in the video.