Reconfigurable Embedded Devices Using Reinforcement Learning to Develop Action-Policies

Abstract

The size of sensor networks supporting smart cities is ever increasing. Sensor network resiliency becomes vital for critical networks such as emergency response and waste water treatment. One approach is to engineer ‘self-aware’ sensors that can proactively change their component composition in response to changes in work load when critical devices fail. By extension, these devices could anticipate their own termination, such as battery depletion, and offload current tasks onto connected devices. These neighboring devices can then reconFigure themselves to process these tasks, thus avoiding catastrophic network failure. In this article, we present an array of self-aware sensors who use Q-learning to develop a policy that guides device reaction to various environmental stimuli. The novelty lies in the use of field programmable gate arrays embedded on the sensors that take into account internal system state, configuration, and learned state-action pairs, that guide device decisions in order to meet system demands. Experiments show that even relatively simple reward functions develop Q-learning policies that yield positive device behaviors in dynamic environments.