Neural Networks-Incorporated Latent Factor Analysis for High-Dimensional and Incomplete Data

high-dimensional and incomplete (HDI) matrices are commonly encountered in a variety of big data-related industrial applications, which describe complex interactions between entities. The complete interaction relationship in the HDI matrix is essential to deal with various problems such as pattern recognition in industrial applications. Therefore, estimating the missing data in the HDI matrix is crucial. latent factor analysis (LFA) models have achieved advanced results in solving such problems. However, the existing LFA models cannot model the nonlinear structure hidden in the HDI matrix. neural networks (NNs) can handle the nonlinearity in the HDI data, but their high estimation accuracy relies on high computation cost and storage burden. To address the aforementioned problems, this article proposes a novel NNLFA model. It contains the following primary ideas: 1) it can model the nonlinear structure of the HDI matrix efficiently through NNs and 2) it incorporates the NNs into the LFA model to improve estimation accuracy while maintaining high computational and storage efficiency. To validate the superiority of the NNLFA model, experiments with six state-of-the-art models are conducted on six practical industrial application datasets. The experimental results indicate that the NNLFA model enhances estimation accuracy by up to 33.3%. In addition, NNLFA model shows strong competitiveness in terms of both time and storage efficiency when compared to baseline models.