Construction of a machine learning dataset for multiple AI tasks using Korean commercial multimodal video clips

Abstract

Accordingly a lot of broadcasting medias pursuing various concepts have been appeared and the major type of contents consumed on the web has been changed to multimodal contents, the attempt to actively utilize multimedia content in artificial intelligence research is also starting. This paper introduces a study that constructs a converged information dataset in an integrated form by analyzing various types of multimodal information on video clips. The constructed dataset was released with various semantic labels for artificial intelligence research about various information classification. The labels and descriptions in this dataset include various context, intention and emotion information describing with vision, speech and language in each video clips. The constructed dataset can be resolved the problem of lack of public data for multimodal interaction research with Korean. It is expected that this dataset can be applied in the constructions of various artificial intelligence services like Korean dialogue processing, visual information extractions and various multimodal data analysis tasks.