Deep Q-Network Based Rotary Inverted Pendulum System and Its Monitoring on the EdgeX Platform

Abstract

A rotary inverted pendulum is an unstable and highly nonlinear device and is used as a common model for engineering applications in linear and nonlinear control. In this study, we created a cyber physical system (CPS) to demonstrate that a deep reinforcement learning agent using a rotary inverted pendulum can successfully control a remotely located physical device. The device we created is composed of a cyber environment and physical environment using the Message Queuing Telemetry Transport (MQTT) protocol with an Ethernet connection to connect the cyber environment and the physical environment. The reinforcement learning agent controls the physical device, which is located remotely from the controller and a classical proportional integral derivative (PID) controller is utilized to implement imitation and reinforcement learning and facilitate the learning process. In addition, the control and monitoring system is built on the open source EdgeX platform, so that learning tasks performed near the source of data generation and real-time data emitted from the physical device can be observed while reinforcement learning is performed. From our CPS experimental system, we verify that a deep reinforcement learning agent can control a remotely located real-world device successfully.