A Study on Machine Learning-Based Image Identification Towards Assitive Automation of Commentary on Animation Characters

This study aims to assist visually impaired people as well as animation novices by focusing on problems that arise at the time of viewing animation videos and images. We focus on the following problems: (1) difficulty of understanding behaviors and situations, (2) difficulty of discriminating animation characters, and (3) confusion caused by animation characters with similarities. We use deep neural networks to identify animation characters as preliminary verification by training a customized convolutional neural network (CNN) from scratch on a small class of data based on the original database of animation characters. The results show that some combinations of characters are difficult to discriminate in cross validation. To resolve this problem, we performed transfer learning based on the CNN variants pre-trained on the natural image database ImageNet. We confirmed that the learning proceeded steadily with a gradual learning curve, resulting in high accuracy. The results indicate that the bottleneck features of the CNN variants pre-trained on ImageNet are effective in identifying animation characters. Furthermore, we verified the operation speed of the inference of our trained CNN on a microcomputer board with a machine learning accelerator Intel Movidius and confirmed that the speed is sufficient in real-time execution.