Navigating image space

Abstract

Navigation means getting from here to there. Unfortunately, for biological navigation, there is no agreed definition of what we might mean by ‘here’ or ‘there’. Computer vision (‘Simultaneous Localisation and Mapping’, SLAM) uses a 3D world-based coordinate frame but that is a poor model for biological spatial representation. Another possibility is to use an image-based rather than a map-based representation. The image-based strategy is made simpler if the observer maintains fixation on a stationary point in the scene as they move. This strategy would require a system for relating different fixation points to one another as the observer moves through the environment. I describe how this can be done by, first, relating fixations to an egocentric representation of visual direction and, second, encoding egocentric representations in a coarse-to-fine hierarchy. The coarsest level of this hierarchy is, in some sense, a world-based frame as it does not vary with eye rotation or observer translation. This representation could be implemented as a ‘policy’, a term used in reinforcement learning to describe a set of states and associated actions, or a ‘graph’ that describes how images or sensory states can be connected by actions. I discuss some of the psychophysical evidence relating to these differing hypotheses about spatial representation and navigation. I argue that this evidence supports image-based rather than map-based representation.