Towards Human-Like Emergent Communication via Utility, Informativeness, and Complexity.

Abstract

Two prominent, yet contrasting, theoretical views are available to characterize the underlying drivers of language evolution: on the one hand, task-specific utility maximization; on the other hand, task-agnostic communicative efficiency. The latter has recently been grounded in an information-theoretic tradeoff between communicative complexity and informativeness, known as the Information Bottleneck (IB) principle. Here, we integrate these two views and propose an information-constrained emergent communication framework that trades off utility, informativeness, and complexity. To train agents within our framework, we develop a method, called Vector-Quantized Variational Information Bottleneck (VQ-VIB), that allows agents to interact using information-constrained discrete communication embedded in a continuous vector space. We test this approach in three domains and show that pressure for informativeness facilitates faster learning and better generalization to novel domains. At the same time, limiting complexity yields better alignment with actual human languages. Lastly, we find that VQ-VIB outperforms previously proposed emergent communication methods; we posit that this is due to the semantically-meaningful communication embedding space that VQ-VIB affords. Overall, our work demonstrates the role of cognitively-motivated optimality principles in inducing aspects of human-like communication among artificial agents.