Cross-domain inductive applications with unsupervised (dynamic) Graph Neural Networks (GNN): Leveraging Siamese GNN and energy-based PMI optimization

Abstract

The existing body of work in graph embedding has primarily focused on Graph Neural Network (GNN) models designed for transductive settings, meaning these models can only be applied to the specific graph on which they were trained. This limitation restricts the applicability of GNN models, as training them on large graphs is computationally expensive. Additionally, there is a significant research gap in applying these models to cross-domain inductive prediction, where the goal is to train on a smaller graph and generalize to larger or different domain graphs. To address these challenges, this study proposes a novel GNN model capable of generating node representations not only within the same domain but also across different domains. To achieve this, we have explored state-of-the-art Graph Neural Networks (GNNs), including Graph Convolutional Networks, Graph Attention Networks, Graph Isomorphism Networks, and Position-Aware Graph Neural Networks. Furthermore, to effectively learn parameters from smaller graphs, we developed a Siamese Graph Neural Network trained using a novel loss function specifically designed for Graph Siamese Neural Networks. Additionally, to handle real-world sparse graphs efficiently, we provide TensorFlow code optimized for sparse graph operations, significantly reducing spatial complexity. To evaluate the performance of the proposed model, we utilized five real-world dynamic graphs. The model was trained on a smaller dataset in an unsupervised manner, and the pre-trained model was then used to generate both inter- and intra-domain graph node representations. The framework demonstrates robustness, as any state-of-the-art GNN method can be integrated into the Siamese neural network framework to learn parameters using the proposed hybrid cost function. The implementation code is publicly available online to ensure reproducibility of the model: https://github.com/khushnood/UnsupervisedPretrainedCrossdomainInductive NodeRepresentationLearning.