Random image masking and in-batch feature mixing for self-supervised learning

Abstract

Contrastive learning has already had a significant impact in the field of self-supervised learning. The contrast of positive and negative samples is critical for contrastive learning. Recently, a revolutionary breakthrough has revealed that it is possible to learn meaningful representations without the need of negative samples. Nevertheless, using two randomly augmented views of the same instance as positive samples inevitably leads to its limitations. The design of positive samples becomes critically important and often necessitates domain-specific expertise. These methods invariably emphasize maximizing the similarity between two views, with a focus on global information invariance: different views of the same image yield approximately similar representations after encoding. Hence, we propose a novel methodology aimed at generating more intricate positive instances at both the image and feature level, termed random image masking and in-batch feature mixing. The former introduces local information loss through random masking of images, compelling the model to learn generalizable representations that focus on local information. The latter generates virtual positive samples by mixing samples within the batch in feature space, breaking free from the limitations of traditional data augmentation. We validate the superiority of our proposed method through experiments on several public datasets, the proposed method significantly enhances the self-supervised learning performance for downstream tasks, particularly in classification and object detection tasks.