Control architecture for embedding reinforcement learning frameworks on industrial control hardware

Abstract

Using reinforcement learning to find new control strategies for manufacturing processes is a promising approach. However, in order to use reinforcement learning in a manufacturing environment, industrial requirements must be met. In this paper, a new control architecture is proposed that allows reinforcement learning frameworks to be executed on programmable logic controllers, which implies requirements regarding real-time execution. An agent exchange module is presented, that allows to automatically load agents from a non real-time learning environment to a control program running in a real-time environment of a programmable logic controller. The proposed architecture is evaluated on an experimental setup with a non-linear process and commercial off-the-shelf control hardware, to meet industrial requirements. A genetic algorithm is used to learn new control strategies without having prior knowledge of the system.