A Dual-Loop Task-Orthogonal Framework for Sequential Multitask Learning in Echo State Networks.

Abstract

Sequential multitask learning is a major problem for Echo State Networks (ESNs) because of catastrophic forgetting and interference between the tasks. To solve this, a Dual-Loop Task-Orthogonal Framework of Echo State Networks (DLTOF-ESN) model is proposed. It has a well-scaling architecture based on orthogonal parameter updates and a new mechanism named Adaptive Stability Regulation (ASR). DLTOF-ESN is based on dual-loop learning. The inner loop uses orthogonal updates to minimize gradient overlap. The outer loop adapts regularization strength based on changes in the relationships of tasks. Also, in this loop the weight sensitivity and decay-weighted similarity are measured using an adaptive relevance mask. This dual-loop interactive mechanism enables the related tasks to be flexible whilst maintaining important information in the tasks that have a temporal distance. In contrast to the traditional Elastic Weight Consolidation (EWC), which uses a fixed regularization, this DLTOF-ESN dynamically varies the penalties according to the relevance of the task and time. This research integrated SHAP-driven layers of interpretability to the framework, which enables explainability and is useful in high-stakes areas. Model evaluations are performed on three benchmark continual learning datasets, Permuted MNIST, Split CIFAR 100 and CIFAR-10, which have shown that DLTOF-ESN shows improvement in tasks recall of up to 18%, accuracy of up to 17%, and interference of up to 44% compared to EWC + ESN and OL + ESN architectures. Ablation studies confirm the synergy between orthogonal updates and ASR methods, while scalability tests reveal reduced training time and memory overhead with increasing dataset sizes.