Semantic Segmentation of UAV Videos based on Temporal Smoothness in Conditional Random Fields

Abstract

Video semantic segmentation is increasingly becoming a vital factor in many Unmanned Aerial Vehicle (UAV) drone-based applications such as surveillance, scene understanding etc. However, the accuracy of video semantic segmentation systems are greatly dependent on temporal consistent labelling. In this regard, a new approach for semantic segmentation of UAV videos is proposed by utilizing U-Net and Conditional Random Field. This algorithm incorporates temporal information to ensure temporal consistency in labelling. This work shows that Conditional Random Field algorithm along with temporal cues reduces the false positives and increases the accuracy of semantic segmentation. Moreover, the proposed method is quantitatively evaluated on ManipalUAVid dataset and achieved a mIoU of 0.88 which is significantly greater than traditional image based segmentation method such as U-Net.