Discovering Invariance From Variations: Invariant-Specific Bi-Graph Neural Network for Multisource Transfer Learning With Application to Industrial Soft Sensor

Abstract

Conventional transfer learning offers a feasible solution to address distribution discrepancy by extracting and transferable knowledge from source domains to the target domain. Integrating information from multiple source domains while mitigating the negative transfer caused by distribution discrepancy poses a significant challenge. Instead of aligning the data distribution among different domains, it is observed that enduring invariance lies in the relation among variables. In this article, an invariant-specific Bi-graph neural network (IS-BiGNN) model is proposed to address the aforementioned issue by designing an invariant relation alignment scheme. It is built upon the subsequent observations: 1) despite discrepancy among multidomains, we propose a novel concept wherein the invariant intervariable relation is put forth as the invariant characteristic within each domain and 2) the mentioned relation can be extracted through the learning of graph representations among the variables. To address the existence of both cross-domain invariant and varying relation among variables, a Bi-graph network architecture is designed. The invariant graph network extracts cross-domain transferable relations, whereas the specific graph network captures domain-specific relation among variables. Invariant relation alignment and specific information filtering modules are developed to implement the extraction of invariant relation collaboratively, facilitating knowledge transfer from multiple source domains to the target domain. Furthermore, with theoretical support, the proposed method provides a tighter generalization error bound. The effectiveness of IS-BiGNN is verified by soft sensor as the downstream task on industrial processes.