Two Stream Dynamic Threshold Network for Weakly-Supervised Temporal Action Localization

Abstract

The current mainstream temporal action localization methods are fully-supervised, which needs a lot of time to annotate the required frame-level labels. The emergence of weakly-supervised methods greatly alleviates such problem, as they only require video-level labels to train the models. In order to generate accurate action localization boundary, recent two stream consensus network (TSCN) proposes an attention normalization loss to explicitly force the attention values approach extreme values to avoid ambiguity. However, most previous methods including TSCN use a fixed threshold applied on attention loss to polarize the attention values, which lacks flexibility for different videos. In this paper, we propose a Dynamic Threshold Weakly-supervised action Localization (DH-WTAL) method to address this problem. The proposed DH-WTAL features a dynamic attention threshold decision for the attention mechanism. Specifically, the dynamic threshold can dynamically control the number of snippets selected for different videos, which further adjust the extreme values of the attention mechanism for different videos accordingly. Extensive experiments demonstrate that the proposed DH-WTAL outperforms the TSCN baseline, and ablation study validates the effectiveness of this method.