Self-supervised transformers predict dynamics of object-based attention in humans.

Abstract

Spread of attention within objects has been proposed as a mechanism for how humans group features to segment objects. However, such a mechanism has not yet been implemented and tested in naturalistic images. Here, we leverage the feature maps from self-supervised vision transformers and propose a model of human object-based attention spreading and segmentation. The attention spreads within an object through the affinity signal between different patches of the image. We show that this model predicts reaction time patterns of people grouping objects in natural images by judging whether two dots are on the same object or on two different objects.