A Deep Convolutional Neural Network Model for Sense of Agency and Object Permanence in Robots

This work investigates the role of predictive models in the implementation of basic cognitive skills in robots, such as the capability to distinguish between self-generated actions and those generated by other individuals and the capability to maintain an enhanced internal visual representation of the world, where objects covered by the robot's own body in the original image may be visible in the enhanced one. A developmental approach is adopted for this purpose. In particular, a humanoid robot is learning, through a self-exploration behaviour, the sensory consequences (in the visual domain) of self-generated movements. The generated sensorimotor experience is used as training data for a deep convolutional neural network that maps proprioceptive and motor data (e.g. initial arm joint positions and applied motor commands) onto the visual consequences of these actions. This forward model is then used in two experiments. First, for generating visual predictions of self-generated movements, which are compared to actual visual perceptions and then used to compute a prediction error. This error is shown to be higher when there is an external subject performing actions, compared to situations where the robot is observing only itself. This supports the idea that prediction errors may serve as a cue for distinguishing between self and other, a fundamental prerequisite for the sense of agency. Secondly, we show how predictions can be used to attenuate self-generated movements, and thus create enhanced visual perceptions, where the sight of objects - originally occluded by the robot body - is still maintained. This may represent an important tool both for cognitive development in robots and for the understanding of the sense of object permanence in humans.