{"title":"ConUMIP:通过表征空间上的不确定性掩蔽混合进行连续时间动态图学习","authors":"Haoyu Zhang, Xuchu Jiang","doi":"10.1016/j.knosys.2024.112748","DOIUrl":null,"url":null,"abstract":"<div><div>Representation learning on continuous-time dynamic graphs has garnered substantial attention for its capacity to model evolving entity relationships. However, existing methods exhibit pronounced overfitting, particularly in complex and sparse data scenarios. We empirically substantiate this overfitting through multiple indicators: (1) a significant performance discrepancy between training and validation/test sets, especially for long-term interaction predictions; (2) an inverse correlation between model complexity and generalization performance; (3) a widening temporal generalization gap as the prediction horizons extend; and (4) rapid performance deterioration under data-sparse conditions. These phenomena collectively demonstrate the overfitting issue, limiting the applicability of current approaches in cold-start scenarios and dynamic environments. To address this, we propose <strong>Con</strong>tinuous-Time Dynamic Graph Learning via <strong>U</strong>ncertainty <strong>M</strong>asked M<strong>I</strong>x-U<strong>P</strong> (ConUMIP), a novel data augmentation method operating in the representation space of continuous-time dynamic graphs. Unlike conventional techniques that perturb raw graph data, ConUMIP adaptively captures temporal evolution patterns and generates diverse augmented samples. This approach effectively mitigates overfitting while enhancing long-term dependency modeling. By eschewing predefined time windows and integrating both local and global structures, ConUMIP demonstrates superior adaptation to complex dynamic evolution patterns. Comprehensive evaluations across five real-world datasets validate ConUMIP's efficacy in substantially improving both the performance and generalizability of existing continuous-time dynamic graph models, particularly in long-term predictions and data-sparse scenarios, without incurring additional computational complexity, thus offering a robust solution to the overfitting challenge in this domain.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112748"},"PeriodicalIF":7.2000,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"ConUMIP: Continuous-time dynamic graph learning via uncertainty masked mix-up on representation space\",\"authors\":\"Haoyu Zhang, Xuchu Jiang\",\"doi\":\"10.1016/j.knosys.2024.112748\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Representation learning on continuous-time dynamic graphs has garnered substantial attention for its capacity to model evolving entity relationships. However, existing methods exhibit pronounced overfitting, particularly in complex and sparse data scenarios. We empirically substantiate this overfitting through multiple indicators: (1) a significant performance discrepancy between training and validation/test sets, especially for long-term interaction predictions; (2) an inverse correlation between model complexity and generalization performance; (3) a widening temporal generalization gap as the prediction horizons extend; and (4) rapid performance deterioration under data-sparse conditions. These phenomena collectively demonstrate the overfitting issue, limiting the applicability of current approaches in cold-start scenarios and dynamic environments. To address this, we propose <strong>Con</strong>tinuous-Time Dynamic Graph Learning via <strong>U</strong>ncertainty <strong>M</strong>asked M<strong>I</strong>x-U<strong>P</strong> (ConUMIP), a novel data augmentation method operating in the representation space of continuous-time dynamic graphs. Unlike conventional techniques that perturb raw graph data, ConUMIP adaptively captures temporal evolution patterns and generates diverse augmented samples. This approach effectively mitigates overfitting while enhancing long-term dependency modeling. By eschewing predefined time windows and integrating both local and global structures, ConUMIP demonstrates superior adaptation to complex dynamic evolution patterns. Comprehensive evaluations across five real-world datasets validate ConUMIP's efficacy in substantially improving both the performance and generalizability of existing continuous-time dynamic graph models, particularly in long-term predictions and data-sparse scenarios, without incurring additional computational complexity, thus offering a robust solution to the overfitting challenge in this domain.</div></div>\",\"PeriodicalId\":49939,\"journal\":{\"name\":\"Knowledge-Based Systems\",\"volume\":\"306 \",\"pages\":\"Article 112748\"},\"PeriodicalIF\":7.2000,\"publicationDate\":\"2024-11-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Knowledge-Based Systems\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0950705124013820\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Knowledge-Based Systems","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0950705124013820","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
ConUMIP: Continuous-time dynamic graph learning via uncertainty masked mix-up on representation space
Representation learning on continuous-time dynamic graphs has garnered substantial attention for its capacity to model evolving entity relationships. However, existing methods exhibit pronounced overfitting, particularly in complex and sparse data scenarios. We empirically substantiate this overfitting through multiple indicators: (1) a significant performance discrepancy between training and validation/test sets, especially for long-term interaction predictions; (2) an inverse correlation between model complexity and generalization performance; (3) a widening temporal generalization gap as the prediction horizons extend; and (4) rapid performance deterioration under data-sparse conditions. These phenomena collectively demonstrate the overfitting issue, limiting the applicability of current approaches in cold-start scenarios and dynamic environments. To address this, we propose Continuous-Time Dynamic Graph Learning via Uncertainty Masked MIx-UP (ConUMIP), a novel data augmentation method operating in the representation space of continuous-time dynamic graphs. Unlike conventional techniques that perturb raw graph data, ConUMIP adaptively captures temporal evolution patterns and generates diverse augmented samples. This approach effectively mitigates overfitting while enhancing long-term dependency modeling. By eschewing predefined time windows and integrating both local and global structures, ConUMIP demonstrates superior adaptation to complex dynamic evolution patterns. Comprehensive evaluations across five real-world datasets validate ConUMIP's efficacy in substantially improving both the performance and generalizability of existing continuous-time dynamic graph models, particularly in long-term predictions and data-sparse scenarios, without incurring additional computational complexity, thus offering a robust solution to the overfitting challenge in this domain.
期刊介绍:
Knowledge-Based Systems, an international and interdisciplinary journal in artificial intelligence, publishes original, innovative, and creative research results in the field. It focuses on knowledge-based and other artificial intelligence techniques-based systems. The journal aims to support human prediction and decision-making through data science and computation techniques, provide a balanced coverage of theory and practical study, and encourage the development and implementation of knowledge-based intelligence models, methods, systems, and software tools. Applications in business, government, education, engineering, and healthcare are emphasized.