Clustering scenes in cooking video guided by object access

Abstract

We propose a method in which scenes in a cooking video are clustered for every type of food processing, such as cutting or stir-frying. To extract motion feature, the method first divides the video into segments. The obtained segments are then clustered based on the similarity of the extracted motion feature. The key point is how to divide the video at the first step of the method. Though a simple approach is to divide the video into segments with the same length, this approach cannot deal with the difference of food processing techniques in cooking. Instead, we propose an approach based on object access, namely the moments when a chef picks up or puts down objects. It is expected to obtain segments reflecting such difference. We compare our method with methods using fixed lengths on three cooking videos in the KUSK Dataset, and evaluate the performance for clustering.