Failure Prediction with Adaptive Multi-scale Sampling and Activation Pattern Regularization

We treat failure prediction in a supervised learning framework using a convolutional neural network (CNN). Due to the nature of the problem, learning a CNN model on this kind of dataset is generally associated with three primary problems: 1) negative samples (indicating a healthy system) outnumber positives (indicating system failures) by a great margin; 2) implementation design often requires chopping an original time series into sub-sequences, defining a segmentation window size with sufficient data augmentation and avoiding serious multiple-instance learning issue is non-trivial; 3) positive samples may have a common underlying cause and thus present similar features, negative samples can have various latent characteristics which can "distract" CNN in the learning process. While the first problem has been extensively discussed in literatures, the last two issues are less explored in the context of deep learning using CNN. We mitigate the second problem by introducing a random variable on sample scaling parameters, whose distribution's parameters are jointly learnt with CNN and leads to what we call adaptive multi-scale sampling (AMS). To address the third problem, we propose activation pattern regularization (APR) on only positive samples such that the CNN focuses on learning representations pertaining to the underlying common cause. We demonstrate the effectiveness of our proposals on a past Kaggle contest dataset that predicts seizures from EEG data. Compared to the baseline method with a CNN trained in traditional scheme, we observe significant performance improvement for both proposed methods. When combined, our model without any sophisticated hyper-parameter tuning or ensemble methods shows a near 10% relative improvement on AUROC and is able to send us to the 14th place on the contest's leaderboard while the highest rank the baseline can reach is 77th.