Improving Deep Random Vector Functional Link Networks through computational optimization of regularization parameters

Deep Random Vector Functional Link (dRVFL) networks are a class of randomization-based deep neural networks known for their rapid learning capabilities and universal approximation potential. Despite these advantages, the dRVFL model faces substantial computational and memory challenges as training data, hidden layers, or input feature dimensions grow. These issues are intensified by regularization, as optimal performance requires repeatedly solving large-scale linear systems to tune the regularization parameter. Traditional matrix inversion methods for this task are computationally intensive, memory-demanding, and prone to numerical instability. This study introduces a computationally efficient two-stage approach to address these limitations. In the first stage, a randomized algorithm-based low-rank approximation computes a compressed representation of the feature matrix in $O\left(mnk\right)$ for an $m\times n$ matrix ($k\ll min(m,n)$), requiring only two passes over the data, significantly reducing costs. In the second stage, the derived solution is used to optimize $\lambda$ via the Golden Section search and Menger curvature-based technique, enabling efficient exploration of the continuous domain with minimal function evaluations. Furthermore, the theoretical framework derives an upper bound on the error of the approximated regularized solution, combining deterministic and probabilistic insights. The proposed method, validated on multiple classification datasets, demonstrates up to 6% improvement in accuracy compared to state-of-the-art feedforward neural network algorithms. Statistical evaluations further confirm its robustness and superior performance over seven baseline algorithms. Overall, the proposed method enhances the scalability, robustness, and efficiency of dRVFL for large-scale applications.