Switching fuzzy transfer learning for multimodal reservoir fuzzy echo state networks

Abstract

In this study, an interesting yet previously uncared concept—switching fuzzy transfer learning for multimodal reservoir echo state networks—is proposed from three overwhelming phenomena under multimodal reservoir transfer learning environments. Due to the use of reservoir transition, the first phenomenon that the discrimination between different classes along each modal may perhaps be weakened actually encourages the use of fuzzy classification. And then the second one is that source and target domains may occasionally have more overlapping caused by the randomness of reservoir computing, which actually encourages the use of fuzzy similarity for these two domains. The third one that the similarity between source and target domains along each modal and even across different modals may perhaps be distorted inspires the switching of transfer learning across modals. This study explores a novel switching fuzzy transfer learning (SFTL) framework for multimodal reservoir echo state networks. SFTL begins with the calculation of the theoretically derived fuzzy reservoir Stein discrepancies (fuzzy RSDs) between target domain and source domains in the same and even different modals. After that, SFTL trains each modal’s fuzzy transfer learning classifier by taking the proposed adaptive multimodal source switching strategy for an appropriate source domain selection. Finally, SFTL achieves promising multimodal learning through moving from linear aggregation level of each fuzzy transfer learning classifier to the mixture level of both this linear aggregation and the switching ensemble of multimodal source domains. The comprehensive experiments on 31 adopted datasets demonstrate the superiority of SFTL, achieving an average classification accuracy of 85.00 % in the focused multimodal reservoir transfer learning scenario.