A comparison of reinforcement learning based approaches to appliance scheduling

Reinforcement learning is often proposed as a technique for intelligent control in a smart home setup with dynamic real-time energy pricing and advanced sub-metering infrastructure. In this paper, we introduce a variation of State Action Reward State Action (SARSA) as an optimization algorithm for appliance scheduling in smart homes with multiple appliances and compare it with the popular reinforcement learning method Q-learning. A simple, intuitive and unique treelike Markov decision process (MDP) structure of appliances is proposed which takes into account the states, such as on/off/runtime status, of all schedulable appliances but does not require the knowledge of the state to state transition probabilities.