Spatiotemporal uncertainty guided non maximum suppression for video event detection.

Abstract

In recent years, several research hotspots have emerged, including autonomous driving, intelligent surveillance, microscopic video analysis, and so on. Since detecting events in video streams is one of the core requirements for these applications, Video Event Detection (VED) has received increased interest in the field of computer vision. Existing methods have focused on introducing and designing novel deep network architectures to improve detection precision or broaden the VED's application to new tasks. However, uncertainty estimation for video event detection has not been thoroughly investigated, which may reduce decision-making mistakes in practical applications. Specifically, the assessment of uncertainty can alert decision-making systems and decision-makers when the detection results are unreliable. In this paper, we propose an end-to-end VED neural network that incorporates spatial and temporal uncertainty. Furthermore, the estimated spatial and temporal uncertainty is considered to guide and improve the procedure of Non-Maximum Suppression (NMS), termed Spatio-Temporal Uncertainty guided NMS (STU-NMS). Extensive experiments on J-HMDB-21, UCF101-24 and AVA datasets demonstrate integration of STU is superior than existing techniques without modeling uncertainty. Meanwhile, the experimental results also indicate that the proposed STU-NMS can further improve the detection performance on three above datasets.