Bayesian Predictive Performance Assessment of Rate-Time Models for Unconventional Production Forecasting

Abstract

A common industry practice is to select a particular model from a set of models to history match oil production and estimate reserves by extrapolation. Future production forecasting is usually done in this deterministic way. However, this approach neglects: a) model uncertainty, and b) quantification of uncertainty of future production forecasts. The current study evaluates the predictive accuracy of rate-time models to forecast production over a set of tight oil wells of West Texas. We present the application of an accuracy metric that evaluates the uncertainty of our models' estimates: the expected log predictive density (elpd). This work assesses the predictive performance of two empirical models—the Arps hyperbolic and the logistic growth models—and two physics-based models—scaled slightly compressible single-phase and scaled two-phase (oil and gas) solutions of the diffusivity equation. These models are arbitrarily selected for the purpose of illustrating the statistical procedure shown in this paper. First, we perform classical regression with the models and evaluate their predictive performance using frequentist (point estimates) metrics such as R2, the Akaike information criteria (AIC), and hindcasting. Second, we generate probabilistic production forecasts using Bayesian inference for each model. Third, we evaluate the predictive accuracy of the models using the elpd accuracy metric. This metric evaluates a measure of out-of-sample predictive performance. We apply both adjusted-within-sample and cross-validation techniques. The adjusted within-sample method is the widely applicable information criteria (WAIC). The cross-validation techniques are hindcasting and leave-one-out (LOO-CV) method. The results of this research are the following. First, we illustrate that the assessment of a model's predictive accuracy depends on whether we use frequentist or Bayesian approaches. This is an important finding in this work. The frequentist approach relies on point estimates while the Bayesian approach considers the uncertainty of our models' estimates. From a frequentist or classical standpoint, all of the models under study yielded very similar results which made it difficult to determine which model yielded the best predictive performance. From a Bayesian standpoint, however, we determined that the logistic growth model yielded a best match in 81 of 130 wells in our sample play and the two-phase physics-based model yielded a best match in 39 of the wells. In addition, we show that WAIC and LOO-CV present similar results for each model, a thing to expect because of their asymptotical equivalence. Finally, Our observations regarding the different models are subject to the dataset under study wherein a majority of the wells are in transient flow. The present study provides tools to evaluate the predictive accuracy of models used to forecast (extrapolate) production of tight oil wells. The elpd is an accuracy metric useful to evaluate the uncertainty of our models' estimates and compare their predictive performance since it assesses distributions instead of point estimates. To our knowledge, the proposed approach is a novel and an appropriate technique to evaluate the predictive accuracy of models to forecast hydrocarbon production.