Robust Dataset Pruning via Joint Noise-Aware Discrimination and Re-Labeling

Abstract

Deep neural networks have achieved success, but require enormous computational costs and storage space on large-scale datasets. Dataset pruning is an effective method to construct compact subsets from large-scale datasets to reduce computational costs while preserving model performance. Existing dataset pruning methods predominantly focus on clean benchmark datasets, overlooking the critical challenges of real-world scenarios with prevalent noisy label samples. In this paper, we find that previous methods have the limitation of being unable to distinguish noisy label samples from hard samples, resulting in overfitting to noisy label samples. To address this, we propose Robust Pruning (RoP), a two-stage dataset pruning framework that consists of noisy label discrimination and re-labeling. Specifically, RoP employs the divergence in label distributions between given labeled samples and their neighboring predictions as an importance metric to identify noisy samples. Compared to other neighborhood-based methods, RoP corrects uncertain neighboring samples through feature and label propagation. Subsequently, RoP applies re-labeling techniques to the selected subset to prevent overfitting of noisy labels. Finally, RoP evaluates easy and hard samples through empirical analysis of re-labeling accuracy, guaranteeing the coverage of hard and easy samples in the selected subset. Extensive experiments demonstrate the effectiveness of the proposed method, not only on real-world benchmarks but also on synthetic datasets, highlighting its suitability for noisy label scenarios. The code is available at https://github.com/successhaha/RoP