Adapting Across Domains via Target-Oriented Transferable Semantic Augmentation Under Prototype Constraint

Abstract

The demand for reducing label annotation cost and adapting to new data distributions gives rise to the emergence of domain adaptation (DA). DA aims to learn a model that performs well on the unlabeled or scarcely labeled target domain by transferring the rich knowledge from a related and well-annotated source domain. Existing DA methods mainly resort to learning domain-invariant representations with a source-supervised classifier shared by two domains. However, such a shared classifier may bias towards source domain, limiting its generalization capability on target data. To alleviate this issue, we present a target-oriented transferable semantic augmentation (T\(^2\)SA) method, which enhances the generalization ability of the classifier by training it with a target-like augmented domain, constructed by semantically augmenting source data towards target at the feature level in an implicit manner. Specifically, to equip the augmented domain with target semantics, we delicately design a class-wise multivariate normal distribution based on the statistics estimated from features to sample the transformation directions for source data. Moreover, we achieve the augmentation implicitly by minimizing the upper bound of the expected Angular-softmax loss over the augmented domain, which is of high efficiency. Additionally, to further ensure that the augmented domain can imitate target domain nicely and discriminatively, the prototype constraint is enforced on augmented features class-wisely, which minimizes the expected distance between augmented features and corresponding target prototype (i.e., average representation) in Euclidean space. As a general technique, T\(^2\)SA can be easily plugged into various DA methods to further boost their performances. Extensive experiments under single-source DA, multi-source DA and domain generalization scenarios validate the efficacy of T\(^2\)SA.