Singular Value Decomposition-based lightweight LSTM for time series forecasting

Long–short-term memory (LSTM) neural networks are known for their exceptional performance in various domains, particularly in handling time series data and managing long-term dependencies. However, deploying LSTM often faces challenges due to limitations in memory and computational resources, especially in edge computing and real-time processing scenarios. To maximize the advantages of LSTM in resource-constrained environments, this paper presents a lightweight LSTM method that uses weight matrix decomposition. Specifically, it employs Singular Value Decomposition (SVD) to decompose the weight matrices within the LSTM Cell and fully connected layers. Then, an optimization method is addressed to enable the efficient development of a lightweight model by dynamically assessing and enhancing storage and computational efficiency through adjustments of the learning rate and weight parameters. The experimental results indicate that this method reduces the parameters of the LSTM model by 45%, compresses the model size to 45% of its original size, and maintains prediction accuracy without decline. It means that the proposed method based on weight matrix decomposition allows LSTM to operate with less computational power and memory, making them more feasible for deploying resource-constrained devices.