Learning Generative Models of Social Interactions with Humans-in-the-Loop

The development of agents that can engage in human social interaction has become critical in an increasingly wide range of applications. The focus of this work is modeling agents for social skills training where learners can interact with the autonomous characters in social scenarios and thereby acquire skills that can be applied in the real world. A key goal in the design of these systems is to allow users to explore various actions and tactics while still having the agents generate consistent and diverse responses. Providing the ability to explore different tactics raises a significant content challenge for the design of agents. To tackle the creative content creation problem, this paper introduces a humans-in-the-loop iterative process to automatically generate rich, varied content from a small amount of vignettes provided by online crowd workers. Specifically, this process uses the crowd to iteratively refine and improve an ensemble of generative models. The results show that the iterative, ensemble based approach generates more coherent and novel interactions than alternative non-ensemble, non-iterative approaches. The results presented in this paper can potentially provide the basis for flexible agent-based training systems.