General interaction battery: Simple object navigation and affordances (GIBSONA)

Abstract

Perception of affordances is an agent’s capability to identify what action-possibilities exist with a particular object or set of objects, based on its own physical properties and capacities. This capability has been well explored in psychology because perception of affordances provides the basis for understanding and interacting with the world. For the same reason, affordance perception is also crucial for AI research. Most approaches to evaluating AI are task-oriented which means that they are geared towards evaluating aggregate performance on a specific set of tasks, rather than focusing on the nature and degree of underlying capabilities that drive task performance. An alternative approach to measuring performance in AI is capability-oriented evaluation, which aims to measure robust, task-independent capabilities across different conditions and difficulties. This approach allows not only measurement of performance but also prediction of performance on novel challenges that share the same fundamental demands. In the context of affordances, there are currently no clear guidelines as to how such capability-oriented approach should best be implemented; for example, there is much variation in what perception of affordances entails. Perhaps for this reason, no comprehensive battery of affordances tasks for AI currently exists. Building on this gap, the aims of this paper are to first, lay out some candidate guidelines for the construction of capability-oriented task batteries for embodied AI and second, to construct and present a battery GIBSONA that takes a step towards this goal: Assessing perception of a set of affordances in AI, directly following these guidelines.