Universal Representations for Well-Logging Data via Ensembling of Self-Supervised Models

Abstract

Time series representation learning is crucial in applications requiring sophisticated data analysis. In some areas, like the Oil and Gas industry, the problem is particularly challenging due to missing values and anomalous samples caused by sensor failures in highly complex manufacturing environments. Self-supervised learning is one of the most popular solutions for obtaining data representation. However, being either generative or contrastive, these methods suffer from the limited applicability of obtained embeddings, – so general usage is more often declared than achieved.

This study introduces and examines various generative self-supervised architectures for complex industrial time series. Moreover, we propose a new way to ensemble several generative approaches, leveraging the best advantages of each method. The suggested procedure is designed to tackle a wide range of scenarios with missing and multiscale data.

For numerical experiments, we use various-scale datasets of well logs from diverse oilfields. Evaluation includes change point detection, clustering, and transfer learning, with the last two problems being introduced for the first time. It shows that variational autoencoders excel in clustering, autoregressive models better detect change points, and the proposed ensemble succeeds in both tasks.