Rec-PF: Data-Driven Large-Scale Deep Learning Recommendation Model Training Optimization Based on Tensor-Train Embedding Table With Photovoltaic Forecast

Abstract

Photovoltaic (PV) power forecasting is important for promoting the integration of renewable energy sources. However, neural network-based methods, particularly deep learning for PV power forecasting, face challenges with computational and memory requirements when dealing with industry-scale datasets. To address this, we introduce Rec-PF, a robust computational framework employing the tensor-train (TT) technique. This framework aims to streamline the training process of massive deep learning recommendation models (DLRMs) on constrained resources. Rec-PF employs a high-performance compressed embedding table, enhancing TT decomposition using key computing primitives. It serves as a drop-in replacement for the PyTorch API. Additionally, Rec-PF utilizes an index reordering technique to leverage local and global information from training inputs, thereby enhancing performance. Furthermore, Rec-PF adopts a pipeline training model, eliminating the need for communication between training workers and host memory. We are pioneers in applying DLRM to PV power prediction to reduce training time without compromising accuracy. Our approach demonstrates a twofold improvement in training time compared to methods that do not incorporate our approach. To better demonstrate the enhanced performance of the algorithm, we specifically compare its efficiency with other frameworks using datasets commonly employed in recommender systems. Comprehensive experiments indicate that Rec-PF is capable of processing the largest publicly accessible DLRM and PV datasets on a single GPU, offering a threefold acceleration compared to state-of-the-art DLRM and PV frameworks.