Gain From Give Up: Intuitive Data Augmentation Framework for Image Retrieval

Abstract

Modern deep hashing methods rely on data augmentation to fully realize their potential in the face of large-scale retrieval galleries and over-parameterized visual models. However, this work observes that mainstream label-preserving augmentation methods are unreliable for information retrieval because they lead to incomplete alignment between data and labels. This misalignment impairs metric losses in distinguishing original/augmented data during same-class clustering, compromising nearest-neighbor search efficacy. To address these issues, we propose an innovative plug-and-play data augmentation framework tailored for retrieval tasks, based on the concept of Gaining robust features by randomly Giving up parts of the image (GG). Inspired by the ease with which visual changes induced by discard transformations can be estimated, we design two intuitive augmentation methods along with corresponding semantic shift estimators to measure the semantic changes introduced by each operation. Additionally, we optimize the metric loss based on the semantic retention scores, guiding the metric objective to properly allocate gradients for generated samples. This adjustment mitigates the adverse effects caused by incomplete alignment, optimizing the intra-class distance of both original and augmented data in the Hamming space, while ensuring the relevance and accuracy of the retrieval results. Extensive experiments conducted on six datasets demonstrate the effectiveness and robustness of our proposed framework.