Applying affective feedback to reinforcement learning in ZOEI, a comic humanoid robot

Abstract

As robotic technologies of varying shapes and forms continue to make their way into our everyday lives, the significance of a humanoid robot's ability to make a human interaction feel natural, engaging and entertaining becomes an area of keen interest in sociable robotics. In this paper, we present our findings on how affective feedback can be used to drive reinforcement learning in human-robot interactions (HRI) and other dialogue systems. We implemented a system where a humanoid robot, named ZOEI, acts as a standup comedian by entertaining a human audience in a bid to generate humor and positively influence the emotional state of the humans. The mood rating of the audience is recorded prior to the interaction session. Using a survey, the eventual emotional state of the human participant is captured after the HRI session. For each audience member, we capture feedback regarding how funny each joke was. We present the implementation of the content selection framework. We share our findings to substantiate the idea that by using expressive behaviors of the humanoid to influence the delivery of content (in this case, jokes) as well as employing reinforcement learning techniques for driving targeted content selection, the robot was able to improve the human mood score progressively across the 16 people who engaged in the study.