Efficiently Learning a Distributed Control Policy in Cyber-Physical Production Systems Via Simulation Optimization

Abstract

The manufacturing industry is becoming more dynamic than ever. The limitations of non-deterministic network delays and real-time requirements call for decentralized control. For such dynamic and complex systems, learning methods stand out as a transformational technology to have a more flexible control solution. Using simulation for learning enables the description of highly dynamic systems and provides samples without occupying a real facility. However, it requires prohibitively expensive computation. In this paper, we argue that simulation optimization is a powerful tool that can be applied to various simulation-based learning processes for tremendous effects. We proposed an efficient policy learning framework, ROSA (Reinforcement-learning enhanced by Optimal Simulation Allocation), with unprecedented integration of learning, control, and simulation optimization techniques, which can drastically improve the efficiency of policy learning in a cyber-physical system. A proof-of-concept is implemented on a conveyer-switch network, demonstrating how ROSA can be applied for efficient policy learning, with an emphasis on the industrial distributed control system.