Self-Governance by Transfiguration: From Learning to Prescription Changes

Abstract

Reinforcement learning is a widespread mechanism for adapting the individual behaviour of autonomous agents, while norms are a well-established means for organising the common conduct of these agents. Therefore, norm-governed reinforcement learning agents appear to be a powerful bio-inspired, as well as socio-inspired, paradigm for the construction of decentralised, self-adapting, self-organising systems. However, the convergence of learning and norms is not as straightforward as it appears: learning can 'misguide' the development of norms, while norms can 'stall' the learning of optimal behaviour. In this paper, we investigate the self-governance of learning agents, or more specifically the domain-independent (de)construction at run-time of prescriptive systems from scratch, for and by learning agents, without any agent having complete information of the system. Most importantly, because prescriptions may also misguide agents, we allow them to repeal any misguiding prescriptions that have previously been enacted. Simulations illustrate the approach with experimental insights regarding scalability and timeliness in the construction of prescriptive systems.