Nonparametric Bayesian machine learning and signal processing

Abstract

We have seen that stochastic processes play an important foundational role in a wide range of methods in DSP. For example, we treat a discrete-time signal as a Gaussian process, and thereby obtain many mathematically simplified algorithms, particularly based on the power spectral density. At the same time, in machine learning, it has generally been observed that nonparametric methods outperform parametric methods in terms of predictive accuracy since they can adapt to data with arbitrary complexity. However, these techniques are not Bayesian so we are unable to do important inferential procedures such as draw samples from the underlying probabilistic model or compute posterior confidence intervals. But, Bayesian models are often only mathematically tractable if parametric, with the corresponding loss of predictive accuracy. An alternative, discussed in this section, is to extend the mathematical tractability of stochastic processes to Bayesian methods. This leads to so-called Bayesian nonparametrics exemplified by techniques such as Gaussian process regression and Dirichlet process mixture modelling that have been shown to be extremely useful in practical DSP and machine learning applications.