Use of Active Appearance Models for analysis and synthesis of naturally occurring behavior

Significant efforts have been made in the analysis and understanding of naturally occurring behavior. Active Appearance Models (AAM) are an especially exciting approach to this task for facial behavior. They may be used both to measure naturally occurring behavior and to synthesize photo-realistic real-time avatars with which to test hypotheses made possible by those measurements. We have used both of these capabilities, analysis and synthesis, to investigate the influence of depression on face-to-face interaction. With AAMs we have investigated large datasets of clinical interviews and successfully modeled and perturbed communicative behavior in a video conference paradigm to test causal hypotheses. These advances have lead to new understanding of the social functions of depression and dampened affect in dyadic interaction. Key challenges remain. These include automated detection and synthesis of subtle facial actions; hybrid methods that optimally integrate automated and manual processing; computational modeling of subjective states from multimodal input; and dynamic models of social and affective behavior.