Recovering From Cyber-Manufacturing Attacks by Reinforcement Learning

Abstract

A Cyber-Manufacturing systems (CMS) is an integration of informational and operational entities that are synchronized with manufacturing processes to increase productivity. However, this integration enlarges the scope for cyber attackers to intrude manufacturing processes, which are called cyber-manufacturing attacks. They can have significant impacts on physical operations within a CMS, such as shutting down plants, production interruption, premature failure of products, and fatal accidents. Although research activities in this emerging problem have been increased recently, existing research has been limited to detection and prevention solutions. However, these strategies cannot ensure a continuous function of an attacked CMS. To ensure continuous functioning of a CMS, a robust recovery strategy must be developed and employed. Current research in recovery has been limited to feedback controllers with an assumption of a complete knowledge of a system model. To overcome this limitation, a recovery agent augmented by reinforcement learning was developed. This is to utilize the ability of reinforcement learning to handle sequential decisions and to proceed even without a complete knowledge of a system model. A virtual environment for recovery agents has been developed to assist efforts needed to obtain sample data, experiment various scenarios, and explore with reinforcement learning. Two cyber-manufacturing attack scenarios have been developed: (i) spoofing a stepper motor controlling additive manufacturing processes, (ii) disrupting the sequence of the pick and place robot. The recovery agent takes random actions by exploring its environment and receives rewards from the actions. After many iterations, it learns proper actions to take.