Learning regions for building a world model from clusters in probability distributions

Abstract

A developing agent learns a model of the world by observing regularities occurring in its sensory inputs. In a continuous domain where the model is represented by a set of rules, a significant part of the task of learning such a model is to find appropriate intervals within the continuous state variables, such that these intervals can be used to define rules whose predictions are reliable. We propose a technique to find such intervals (or regions) by means of finding clusters on approximate probability distributions of sensory variables. We compare this cluster-based method with an alternative landmark-based algorithm. We evaluate both techniques on a data log recorded in a simulation based on OpenArena, a three-dimensional first-person-perspective computer game, and demonstrate the results of how the techniques can learn rules which describe walking behaviour. While both techniques work reasonably well, the clustering approach seems to give more “natural” regions which correspond more closely to what a human would expect; we speculate that such regions should be more useful if they are to form a basis for further learning of higher order rules.