Chenchen Zhang , Zhan Su , Qiuchi Li , Dawei Song , Prayag Tiwari
{"title":"Quantum-inspired neural network with hierarchical entanglement embedding for matching","authors":"Chenchen Zhang , Zhan Su , Qiuchi Li , Dawei Song , Prayag Tiwari","doi":"10.1016/j.neunet.2024.106915","DOIUrl":null,"url":null,"abstract":"<div><div>Quantum-inspired neural networks (QNNs) have shown potential in capturing various non-classical phenomena in language understanding, e.g., the emgerent meaning of concept combinations, and represent a leap beyond conventional models in cognitive science. However, there are still two limitations in the existing QNNs: (1) Both storing and invoking the complex-valued embeddings may lead to prohibitively expensive costs in memory consumption and storage space. (2) The use of entangled states can fully capture certain non-classical phenomena, which are described by the tensor product with powerful compression ability. This approach shares many commonalities with the process of word formation from morphemes, but such connection has not been further exploited in the existing work. To mitigate these two limitations, we introduce a Quantum-inspired neural network with Hierarchical Entanglement Embedding (QHEE) based on finer-grained morphemes. Our model leverages the <em>intra-word</em> and <em>inter-word</em> entanglement embeddings to learn a multi-grained semantic representation. The intra-word entanglement embedding is employed to aggregate the constituent morphemes from multiple perspectives, while the inter-word entanglement embedding is utilized to combine different words based on unitary transformation to reveal their non-classical correlations. Both the number of morphemes and the dimensionality of the morpheme embedding vectors are far smaller than the counterparts of words, which would compress embedding parameters efficiently. Experimental results on four benchmark datasets of different downstream tasks show that our model outperforms strong quantum-inspired baselines in terms of effectiveness and compression ability.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106915"},"PeriodicalIF":6.0000,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S089360802400844X","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Quantum-inspired neural networks (QNNs) have shown potential in capturing various non-classical phenomena in language understanding, e.g., the emgerent meaning of concept combinations, and represent a leap beyond conventional models in cognitive science. However, there are still two limitations in the existing QNNs: (1) Both storing and invoking the complex-valued embeddings may lead to prohibitively expensive costs in memory consumption and storage space. (2) The use of entangled states can fully capture certain non-classical phenomena, which are described by the tensor product with powerful compression ability. This approach shares many commonalities with the process of word formation from morphemes, but such connection has not been further exploited in the existing work. To mitigate these two limitations, we introduce a Quantum-inspired neural network with Hierarchical Entanglement Embedding (QHEE) based on finer-grained morphemes. Our model leverages the intra-word and inter-word entanglement embeddings to learn a multi-grained semantic representation. The intra-word entanglement embedding is employed to aggregate the constituent morphemes from multiple perspectives, while the inter-word entanglement embedding is utilized to combine different words based on unitary transformation to reveal their non-classical correlations. Both the number of morphemes and the dimensionality of the morpheme embedding vectors are far smaller than the counterparts of words, which would compress embedding parameters efficiently. Experimental results on four benchmark datasets of different downstream tasks show that our model outperforms strong quantum-inspired baselines in terms of effectiveness and compression ability.
期刊介绍:
Neural Networks is a platform that aims to foster an international community of scholars and practitioners interested in neural networks, deep learning, and other approaches to artificial intelligence and machine learning. Our journal invites submissions covering various aspects of neural networks research, from computational neuroscience and cognitive modeling to mathematical analyses and engineering applications. By providing a forum for interdisciplinary discussions between biology and technology, we aim to encourage the development of biologically-inspired artificial intelligence.