Two-level attention mechanism with contrastive learning for heterogeneous graph representation learning

Abstract

This paper introduces a novel method, M2CHGNN, for generating node representations in heterogeneous graphs by combining an attention mechanism with contrastive learning. The approach leverages meta-structures and meta-paths to capture complex hidden structures within heterogeneous graphs. While meta-paths identify diverse interaction patterns between nodes, meta-structures uncover intricate structural arrangements, revealing detailed relationships and capturing both local and higher-order structures simultaneously. Each meta-path or meta-structure results in a homogeneous graph to obtain node representations. Within each homogeneous subgraph, two views of node representation are then employed. In the first view, a node attention mechanism assesses the influence of neighboring nodes during embedding extraction, emphasizing the features of influential neighbors. Concurrently, a second set of embeddings is derived using the graph-topology view, highlighting structural relationships and further enriching the representations. Then, an additional attention layer is applied to determine the significance of each homogeneous subgraph within each view, resulting in a weighted, aggregated node representation. To learn robust and informative node representations across these two views, we use contrastive learning to align and distinguish representations between the node-attention and graph-topology views, alongside intra-view contrastive learning to refine each view individually. To train the node representations, we combine contrastive loss with a cross-entropy loss function, which enhances the model’s ability to generate high-quality node representations for heterogeneous graphs. The effectiveness of M2CHGNN was evaluated in three different applications, including link prediction, data classification, and clustering. The experimental results demonstrated the superiority of the proposed method in comparison with baseline and state-of-the-art graph embedding methods.