Incremental Markov-model planning

Abstract

This paper presents an approach to building plans using partially observable Markov decision processes. The approach begins with a base solution that assumes full observability. The partially observable solution is incrementally constructed by considering increasing amounts of information from observations. The base solution directs the expansion of the plan by providing an evaluation function for the search fringe. We show that incremental observation moves from the base solution towards the complete solution, allowing the planner to model the uncertainty about action outcomes and observations that are present in real domains.