Assessment of smile intensity using an ordinal scale toward QOL monitoring

Abstract

Facial expressions are the most common resource for estimating the internal state of a person and are widely employed as a method for monitoring in care settings. In daily life, peak or high-intensity expressions appear for only short durations of time, whereas neutral or low-to-mid-intensity expressions are far more common. Existing algorithms for facial expression recognition (FER) have focused on detecting high-intensity expressions, which limits the detection of low-to-mid expressions and prevents detailed intensity assessments. To address this problem, we propose a framework for facial expression assessment based on comparison and ordinal scale. Using this framework, we also propose methods for determining the number of ordinal ranks based on discriminability and for selecting reference images for intensity assessment. Experimental results demonstrate that the proposed framework outperforms both human annotators and conventional FER systems.