An end-to-end optimized feature specific data imputation for recurrent neural networks under missing data

Abstract

We investigate regression and classification of time series under missing data, which happens in most real-life applications and severely degrades the performance of most, if not all, machine learning algorithms. We introduce a novel missing-valued time series processing algorithm involving a set of different imputation models to complete these missing values. We formulate the imputation selection in a multi-armed bandit framework, where imputation functions are selected specifically for each feature. Particularly, we simultaneously select an imputation model for each feature/component of the input vector among a set of imputation algorithms and train these imputation models along with the network for the target task in an end-to-end manner. Since the individual features may have widely distinct characteristics and temporal behaviors, a single imputation algorithm may show less than adequate performance for the imputation of all of the features. Our method is generic so that the set of imputation models can straightforwardly be extended by the additional imputation methods, and is also equally applicable to recurrent neural network architectures, even when the data arrival times of the feature vectors are non-uniform. In our experiments, we achieved significant performance improvements with respect to the state-of-the-art methods in well-known real-life datasets under different missing data regimes.