CMVCG: Non-autoregressive Conditional Masked Live Video Comments Generation Model

Abstract

The blooming of live comment videos leads to the need of automatic live video comment generating task. Previous works focus on autoregressive live video comments generation and can only generate comments by giving the first word of the target comment. However, in some scenes, users need to generate comments by their given prompt keywords, which can't be solved by the traditional live video comment generation methods. In this paper, we propose a Transformer based non-autoregressive conditional masked live video comments generation model called CMVCG model. Our model considers not only the visual and textual context of the comments, but also time and color information. To predict the position of the given prompt keywords, we also introduce a keywords position predicting module. By leveraging the conditional masked language model, our model achieves non-autoregressive live video comment generation. Furthermore, we collect and introduce a large-scale real-world live video comment dataset called Bili-22 dataset. We evaluate our model in two live comment datasets and the experiment results present that our model outperforms the state-of-the-art models in most of the metrics.