Learning Strategies Based on Fuzzy Set Rules for the Ideal Opponent Model

Abstract

RoboCup Soccer is a rich domain for the study of multiagent learning issues. Not only must the players learn low-level skills, but they must also learn to work together and to adapt to the behaviors of different opponents. Dynamic behaviour learning in the face of adversarial opponents involves a) learning a basic set of strategies, and b) tuning these strategies for the specific opponents involved. Iterative approaches to dynamic learning are often slow for large state spaces, especially since in many dynamic situations, the reward is not obvious immediately, but may need to be temporally apportioned over multiple time epochs. In this work, we construct a reinforcement learning model based on a radial basis function network which may be interpreted as a set of fuzzy rules, and which are capable of real-time online learning. We test this method on the soccer-server domain that has emerged as an important testbed for learning dynamic behaviours. In addition to relatively simple behaviours such as goal scoring, we also learn multi-epoch behaviours such as pass interception in the presence of multiple opponents.