Adversarial Training on Weights for Graph Neural Networks

Despite the fact that Graph Neural Networks (GNNs) have been extensively used for graph embedding representation, it is challenging to train well-performing GNNs on graphs with good generalization due to the limitation of overfitting. Previous research in Computer Vision (CV) has shown that the lack of generalization usually corresponds to the convergence of model parameters to sharp local minima. However, there is still a lack of related research in the field of graph analysis. In this paper, we investigate the loss landscape of models from the weight change perspective and show that the vanilla training method tends to cause GNNs to fall into sharp local minima with poor generalization. To tackle this problem, we propose a method named Adversarial Training on Weights (ATW) to flatten the weight loss landscape using adversarial training, thus improving the generalization of GNNs. Extensive experiments with multiple backbones on various datasets demonstrate the effectiveness of our method.