OPUPO: Defending Against Membership Inference Attacks With Order-Preserving and Utility-Preserving Obfuscation

Abstract

In this work, we present OPUPO to protect machine learning classifiers against black-box membership inference attacks by alleviating the prediction difference between training and non-training samples. Specifically, we apply order-preserving and utility-preserving obfuscation to prediction vectors. The order-preserving constraint strictly maintains the order of confidence scores in the prediction vectors, guaranteeing that the model's classification accuracy is not affected. The utility-preserving constraint, on the other hand, enables adaptive distortions to the prediction vectors in order to protect their utility. Moreover, OPUPO is proved to be adversary resistant that even well-informed defense-aware adversaries cannot restore the original prediction vectors to bypass the defense. We evaluate OPUPO on machine learning and deep learning classifiers trained with four popular datasets. Experiments verify that OPUPO can effectively defend against state-of-the-art attack techniques with negligible computation overhead. In specific, the inference accuracy could be reduced from as high as 87.66% to around 50%, i.e., random guess, and the prediction time will increase by only 0.44% on average. The experiments also show that OPUPO could achieve better privacy-utility trade-off than existing defenses.