Mitigating over-smoothing in Graph Neural Networks for node classification through Adaptive Early Embedding and Biased DropEdge procedures

Abstract

Graph Neural Networks (GNNs) are widely used for tasks involving graph-structured data across various fields, including computer vision, biology, social media, and traffic prediction. Despite their substantial success, increasing the depth of GNNs can impair the discriminability of node representations, leading to a decline in performance for node classification tasks. This challenge is partly attributed to a phenomenon known as over-smoothing. This paper introduces an Adaptive Early Embedding (AEE) procedure between Graph Convolutional Network (GCN) layers. This method adaptively halts the aggregation of neighboring nodes before the final layer of the main network. By reducing the over-smoothing of node embeddings, we enhance the distinguishability of the data. Another important contribution of this work is using the inter-class Biased DropEdge (BDE) procedure, which effectively propagates beneficial information. The proposed model based on AEE+BDE can be integrated with baseline message-passing GNN models to mitigate over-smoothing challenges. Our experiments show that the proposed model outperforms baseline models. Additionally, we provide theoretical evidence supporting the effectiveness of the AEE and BDE procedures for node classification tasks.