Learning Error Refinement in Stochastic Gradient Descent-Based Latent Factor Analysis via Diversified PID Controllers

In Big Data-based applications, high-dimensional and incomplete (HDI) data are frequently used to represent the complicated interactions among numerous nodes. A stochastic gradient descent (SGD)-based latent factor analysis (LFA) model can process such data efficiently. Unfortunately, a standard SGD algorithm trains a single latent factor relying on the stochastic gradient related to the current learning error only, leading to a slow convergence rate. To break through this bottleneck, this study establishes an SGD-based LFA model as the backbone, and proposes six proportional-integral-derivative (PID)-incorporated LFA models with diversified PID-controllers with the following two-fold ideas: a) refining the instant learning error in stochastic gradient by the principle of six PID-variants, i.e., a standard PID, an integral separated PID, a gearshift integral PID, a dead zone PID, an anti-windup PID, and an incomplete differential PID, to assimilate historical update information into the learning scheme in an efficient way; b) making the hyper-parameters adaptation by utilizing the mechanism of particle swarm optimization for acquiring high practicality. In addition, considering the diversified PID-variants, an effective ensemble is implemented for the six PID-incorporated LFA models. Experimental results on industrial HDI datasets illustrate that in comparison with state-of-the-art models, the proposed models obtain superior computational efficiency while maintaining competitive accuracy in predicting missing data within an HDI matrix. Moreover, their ensemble further improves performance in terms of prediction accuracy.