Advertisement Effectiveness Estimation Based on Crowdsourced Multimodal Affective Responses

In this paper, we estimate the effectiveness of an advertisement using online data collection and the remote measurement of facial expressions and physiological responses. Recently, the online advertisement market has expanded, and the measurement of advertisement effectiveness has become very important. We collected a significant number of videos of Japanese faces watching video advertisements in the same scenario in which media is normally used via the Internet. Facial expression and physiological responses such as heart rate and gaze were remotely measured by analyzing facial videos. By combining the measured responses into multimodal features and using machine learning, we show that ad liking can be predicted (ROC AUC = 0.93) better than when only single-mode features are used. Furthermore, intent to purchase can be estimated well (ROC AUC = 0.91) using multimodal features.