ALSTM: Adaptive LSTM for Durative Sequential Data

Abstract

Long short-term memory (LSTM) network is an effective model architecture for deep learning approaches to sequence modeling tasks. However, the current LSTMs can't use the property of sequential data when dealing with the sequence components, which last for a certain period of time. This may make the model unable to benefit from the inherent characteristics of time series and result in poor performance as well as lower efficiency. In this paper, we present a novel adaptive LSTM for durative sequential data which exploits the temporal continuance of the input data in designing a new LSTM unit. By adding a new mask gate and maintaining span, the cell's memory update is not only determined by the input data but also affected by its duration. An adaptive memory update method is proposed according to the change of the sequence input at each time step. This breaks the limitation that the cells calculate the cell state and hidden output for each input always in a unified manner, making the model more suitable for processing the sequences with continuous data. The experimental results on various sequence training tasks show that under the same iteration epochs, the proposed method can achieve higher accuracy, but need relatively less training time compared with the standard LSTM architecture.