{"title":"Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization","authors":"","doi":"10.1016/j.neunet.2024.106704","DOIUrl":null,"url":null,"abstract":"<div><div>Graph Neural Networks (GNNs) have drawn great attention in handling graph-structured data. To characterize the message-passing mechanism of GNNs, recent studies have established a unified framework that models the graph convolution operation as a graph signal denoising problem. While increasing interpretability, this framework often performs poorly on heterophilic graphs and also leads to shallow and fragile GNNs in practice. The key reason is that it encourages feature smoothness, but ignores the high-frequency information of node features. To address this issue, we propose a general framework for GNNs via relaxation of the smoothness regularization. In particular, it employs an information aggregation mechanism to learn the low- and high-frequency components adaptively from data, offering more flexible graph convolution operators compared to the smoothness-promoted framework. Theoretical analyses demonstrate that our framework can capture both low- and high-frequency information of node features, effectively. Experiments on nine benchmark datasets show that our framework achieves the state-of-the-art performance in most cases. Furthermore, it can be used to handle deep models and adversarial attacks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0000,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0893608024006282","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Graph Neural Networks (GNNs) have drawn great attention in handling graph-structured data. To characterize the message-passing mechanism of GNNs, recent studies have established a unified framework that models the graph convolution operation as a graph signal denoising problem. While increasing interpretability, this framework often performs poorly on heterophilic graphs and also leads to shallow and fragile GNNs in practice. The key reason is that it encourages feature smoothness, but ignores the high-frequency information of node features. To address this issue, we propose a general framework for GNNs via relaxation of the smoothness regularization. In particular, it employs an information aggregation mechanism to learn the low- and high-frequency components adaptively from data, offering more flexible graph convolution operators compared to the smoothness-promoted framework. Theoretical analyses demonstrate that our framework can capture both low- and high-frequency information of node features, effectively. Experiments on nine benchmark datasets show that our framework achieves the state-of-the-art performance in most cases. Furthermore, it can be used to handle deep models and adversarial attacks.
期刊介绍:
Neural Networks is a platform that aims to foster an international community of scholars and practitioners interested in neural networks, deep learning, and other approaches to artificial intelligence and machine learning. Our journal invites submissions covering various aspects of neural networks research, from computational neuroscience and cognitive modeling to mathematical analyses and engineering applications. By providing a forum for interdisciplinary discussions between biology and technology, we aim to encourage the development of biologically-inspired artificial intelligence.