{"title":"Generative commonsense knowledge subgraph retrieval for open-domain dialogue response generation","authors":"","doi":"10.1016/j.neunet.2024.106666","DOIUrl":null,"url":null,"abstract":"<div><p>Grounding on a commonsense knowledge subgraph can help the model generate more informative and diverse dialogue responses. Prior <em>Traverse-based</em> works explicitly retrieve a subgraph from the external knowledge base (eKB). Notably, the available knowledge is strictly restricted by the eKB. To break this restriction, <em>Generative Retrieval</em> methods externalize knowledge from the language model. However, they always generate boring knowledge due to their one-pass externalization procedure. This work proposes a novel TiLM <em>Traverse in Language Model (TiLM)</em>, which uses three ‘Chain-of-Thought’ sub-tasks, i.e., <em>Query Entity Production</em>, <em>Topic Entity Prediction</em>, and <em>Knowledge Subgraph Completion</em>, to build a high-quality knowledge subgraph to ground the next <em>Response Generation</em> without explicitly accessing the eKB in inference. Experimental results on both Chinese and English datasets demonstrate <em>TiLM</em>’s outstanding performance even only with a small scale of parameters.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0000,"publicationDate":"2024-08-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0893608024005902","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Grounding on a commonsense knowledge subgraph can help the model generate more informative and diverse dialogue responses. Prior Traverse-based works explicitly retrieve a subgraph from the external knowledge base (eKB). Notably, the available knowledge is strictly restricted by the eKB. To break this restriction, Generative Retrieval methods externalize knowledge from the language model. However, they always generate boring knowledge due to their one-pass externalization procedure. This work proposes a novel TiLM Traverse in Language Model (TiLM), which uses three ‘Chain-of-Thought’ sub-tasks, i.e., Query Entity Production, Topic Entity Prediction, and Knowledge Subgraph Completion, to build a high-quality knowledge subgraph to ground the next Response Generation without explicitly accessing the eKB in inference. Experimental results on both Chinese and English datasets demonstrate TiLM’s outstanding performance even only with a small scale of parameters.
期刊介绍:
Neural Networks is a platform that aims to foster an international community of scholars and practitioners interested in neural networks, deep learning, and other approaches to artificial intelligence and machine learning. Our journal invites submissions covering various aspects of neural networks research, from computational neuroscience and cognitive modeling to mathematical analyses and engineering applications. By providing a forum for interdisciplinary discussions between biology and technology, we aim to encourage the development of biologically-inspired artificial intelligence.