Development of pixel-based facial thermal image analysis for emotion sensing

Abstract

Thermal imaging technology, known for its noncontact and noninvasive nature, offers distinct advantages in computerized emotion sensing. In the literature, a decrease in nose-tip temperature has been associated with dynamic subjective arousal. However, these studies were limited by their focus on a few regions of interest, neglecting a comprehensive analysis of the entire face, and not accounting for the temporal dynamics of thermal changes. To overcome these limitations, we propose an analytical method for facial thermal images using statistical parametric mapping (SPM), which was developed for functional brain image analysis. We developed semiautomated preprocessing protocols to effectively realign and standardize facial thermal images. To validate these analyses, we recorded the thermal images of participants’ faces and assessed dynamic valence and arousal ratings while they observed emotional films. The proposed SPM analyses revealed significant negative associations with dynamic arousal ratings at the nose tip and forehead. The analyses incorporating temporal disparity revealed more forehead clusters than the analyses assuming no delay. These findings validate the proposed pixel-based facial thermal image analysis method using SPM. The results suggest that computerized pixel-based analysis of facial thermal images can be used to estimate dynamic emotional states, with potential applications in various human behavioral fields, including mental health diagnosis and marketing research.