Learning acceleration by policy sharing

Abstract

Reinforcement learning is one of the more prominent machine learning technologies due to its unsupervised learning structure and ability to continually learn, even in a dynamic operating environment. Applying this learning to cooperative multi-agent systems not only allows each individual agent to learn from its own experience, but also offers the opportunity for the individual agents to learn from the other agents in the system to increase the speed of learning can be accelerated. In the proposed learning algorithm, an agent store its experience in terms of state aggregation implemented with a decision tree, such that policy sharing between multi-agent is eventually accomplished by merging different decision trees between peers. Unlike lookup tables which have homogeneous structure for state aggregations, decision trees carried in agents are with heterogeneous structure. This work executes policy sharing between cooperative agents by means of forming a hyper structure from their trees instead of merging whole trees violently. The proposed scheme initially translates the whole decision tree from an agent to others. Based on the evidence, only partial leaf nodes hold helpful experience for policy sharing. The proposed method inducts a hyper decision tree by a great mount of samples which are sampled from the shared nodes. Results from simulations in multi-agent cooperative domain illustrate that the proposed algorithms perform better than the one without sharing.