{"title":"Incorporating Template-Based Contrastive Learning into Cognitively Inspired, Low-Resource Relation Extraction","authors":"Yandan Zheng, Luu Anh Tuan","doi":"10.1007/s12559-024-10343-8","DOIUrl":null,"url":null,"abstract":"<p>From an unstructured text, relation extraction (RE) predicts semantic relationships between pairs of entities. The process of labeling tokens and phrases can be very expensive and require a great deal of time and effort. The low-resource relation extraction (LRE) problem comes into being and is challenging since there are only a limited number of annotated sentences available. Recent research has focused on minimizing the cross-entropy loss between pseudo labels and ground truth or on using external knowledge to make annotations for unlabeled data. Existing methods, however, fail to take into account the semantics of relation types and the information hidden within different relation groups. By drawing inspiration from the process of human interpretation of unstructured documents, we introduce a <b>Temp</b>late-based <b>C</b>ontrastive <b>L</b>earning ( <span>TempCL</span> ). Through the use of <i>template</i>, we limit the model’s attention to the semantic information that is contained in a relation. Then, we employ a <i>contrastive learning</i> strategy using both <i>group-wise</i> and <i>instance-wise</i> perspectives to leverage shared semantic information within the same relation type to achieve a more coherent semantic representation. Particularly, the proposed group-wise contrastive learning minimizes the discrepancy between the template and original sentences in the same label group and maximizes the difference between those from separate label groups under limited annotation settings. Our experiment results on two public datasets show that our model <span>TempCL</span> achieves state-of-the-art results for low-resource relation extraction in comparison to baselines. The relative error reductions range from 0.68 to 1.32%. Our model encourages the feature to be aligned with both the original and template sentences. Using two contrastive losses, we exploit shared semantic information underlying sentences (both original and template) that have the same relation type. We demonstrate that our method reduces the noise caused by tokens that are unrelated and constrains the model’s attention to the tokens that are related.</p>","PeriodicalId":51243,"journal":{"name":"Cognitive Computation","volume":null,"pages":null},"PeriodicalIF":4.3000,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cognitive Computation","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s12559-024-10343-8","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
From an unstructured text, relation extraction (RE) predicts semantic relationships between pairs of entities. The process of labeling tokens and phrases can be very expensive and require a great deal of time and effort. The low-resource relation extraction (LRE) problem comes into being and is challenging since there are only a limited number of annotated sentences available. Recent research has focused on minimizing the cross-entropy loss between pseudo labels and ground truth or on using external knowledge to make annotations for unlabeled data. Existing methods, however, fail to take into account the semantics of relation types and the information hidden within different relation groups. By drawing inspiration from the process of human interpretation of unstructured documents, we introduce a Template-based Contrastive Learning ( TempCL ). Through the use of template, we limit the model’s attention to the semantic information that is contained in a relation. Then, we employ a contrastive learning strategy using both group-wise and instance-wise perspectives to leverage shared semantic information within the same relation type to achieve a more coherent semantic representation. Particularly, the proposed group-wise contrastive learning minimizes the discrepancy between the template and original sentences in the same label group and maximizes the difference between those from separate label groups under limited annotation settings. Our experiment results on two public datasets show that our model TempCL achieves state-of-the-art results for low-resource relation extraction in comparison to baselines. The relative error reductions range from 0.68 to 1.32%. Our model encourages the feature to be aligned with both the original and template sentences. Using two contrastive losses, we exploit shared semantic information underlying sentences (both original and template) that have the same relation type. We demonstrate that our method reduces the noise caused by tokens that are unrelated and constrains the model’s attention to the tokens that are related.
期刊介绍:
Cognitive Computation is an international, peer-reviewed, interdisciplinary journal that publishes cutting-edge articles describing original basic and applied work involving biologically-inspired computational accounts of all aspects of natural and artificial cognitive systems. It provides a new platform for the dissemination of research, current practices and future trends in the emerging discipline of cognitive computation that bridges the gap between life sciences, social sciences, engineering, physical and mathematical sciences, and humanities.